Food animation video generation method, program, food animation video generation server, and food animation video providing system

Abstract

To properly represent the sensation when eating a food product by another form of sensory expression.SOLUTION: One or more embodiments of the present invention includes the steps of: determining a graphic and a color representing a feeling at each time from when eating a target food product on the basis of feeling change information IN2 representing a time-series change in the feeling when eating the target food product; determining an arrangement state of the color and the graphic in an image including the determined graphic and color on the basis of the feeling when eating the target food product; generating the image; repeatedly executing the determination of the graphic and color, the determination of the arrangement state of the color and graphic and the generation of the image at each predetermined time from when eating the target food product to generate a plurality of images; and generating a food animation by arranging the plurality of generated images in chronological order.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to a method, a program, a server, and a system for generating a video that expresses the sensation of eating food. [Background technology]

[0002] Currently, food descriptions, promotional advertisements, etc. are carried out using, for example, photographs of the food, videos, or text that verbally describes the sensations experienced when eating the food. These methods may not accurately express the sensations experienced when eating the food (e.g., taste, aroma, etc.). Furthermore, for example, when the sensations of food are expressed in text, people who are not experts in food sensations may not be able to understand the meaning of the text and may not be able to recognize the sensations of the food. One possible way to solve this problem is to use technology that converts the sensations of food so that they can be recognized by a different sense (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-76118 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional methods for converting the sensation of food so that it can be perceived by another sense, the optimal method for converting the sensation of food into another sense has not been established, and therefore, conventional methods have not been able to adequately express the sensation of food through another sensory expression.

[0005] Therefore, an object of the present invention is to appropriately express the sensation experienced when eating food using a different sensory expression. [Means for solving the problem]

[0006] [1] A step of determining a graphic and a color representing the sensation at each time point after eating a target food based on sensory change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation when eating the target food; generating the image; a step of repeatedly executing the steps of determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; generating a food animation video by arranging the generated plurality of images in time series; A food animation video generation method comprising:

[0007] [2] A food animation video generation method according to [1], in which the size of the figure in the image is determined based on the sensation experienced when eating the target food.

[0008] [3] A food animation video generation method described in [1] or [2], in which the number of figures to be placed in the image is determined based on the sensation when eating the target food.

[0009] [4] The sensations include the aroma and taste of the food of interest; A food animation video generation method described in any of [1] to [3], wherein the figure representing the aroma of the target food is placed at the top of the image, and the figure representing the taste of the target food is placed at the bottom of the image.

[0010] [5] A food animation video generation method described in any of [1] to [4], wherein the sensory change information includes information regarding the type of sensation that appears at each time after eating the target food and information regarding the intensity of the sensation at each time.

[0011] [6] the sensation includes the taste of the food of the subject; The food animation moving image generating method according to any one of [1] to [5], wherein the shape and color of the figure are determined based on the taste of the target food.

[0012] [7] A food animation video generation method described in [6], which determines the size, color intensity, number of figures to be displayed in the image, and / or the arrangement of multiple figures in the image based on the intensity of the taste of the target food.

[0013] [8] A food animation video generation method according to [6] or [7], in which the placement position of the figure is determined based on the impression of the taste of the target food.

[0014] [9] the sensation includes the aroma of the food of the subject; The food animation moving image generating method according to any one of [1] to [8], wherein a background color of the image is determined based on the aroma of the target food.

[0015]

[10] A food animation video generation method as described in [9], which determines the background color intensity of the image based on the intensity of the aroma of the target food.

[0016]

[11] A food animation video generation method according to any one of claims [1] to

[10] , wherein the style of the graphics is determined based on the attributes of the user who watches the food animation video.

[0017]

[12] A program that causes a computer to execute the food animation video generation method described in any one of [1] to

[11] .

[0018]

[13] A storage unit and an information processing unit, The memory unit stores sensory change information representing a time-series change in sensation when eating a target food product, The information processing unit determining a graphic and a color representing the sensation at each time point after eating the target food based on the sensation change information; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food. Food animation video generation server.

[0019]

[14] A system including a server and a terminal, The server determining a graphic and a color representing the sensation at each time point after eating the target food based on sensory change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; Sending the food animation video to the terminal; The terminal displays the food animation video received from the server. Food animation video provision system. [Effects of the Invention]

[0020] The food animation video generation method, program causing a computer to execute the method, food animation video generation server, and food animation video provision system disclosed in this specification determine the colors, shapes, etc. to be displayed in each frame of a food animation video and determine the arrangement of these colors, shapes, etc. based on the sensations experienced when eating food. In food animation videos that visually represent the sensations experienced when eating food, not only the colors, shapes, etc. displayed but also how they are arranged in each frame greatly influences the impression given to the user. Therefore, the food animation video generation method, program causing a computer to execute the method, food animation video generation server, and food animation video provision system disclosed in this specification can generate food animation videos that can appropriately visually represent the sensations experienced when eating food. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a diagram illustrating the configuration of a food animation video generation system. [Figure 2A] FIG. 10 is a diagram showing an example of a graphic representing a scent. [Figure 2B] FIG. 10 is a diagram showing an example of a graphic representing a scent. [Figure 2C] FIG. 10 is a diagram showing an example of a graphic representing a scent. [Figure 2D] FIG. 10 is a diagram showing an example of a graphic representing a scent. [Figure 2E] FIG. 10 is a diagram showing an example of a graphic representing a scent. [Figure 3A] FIG. 10 is a diagram showing an example of a graphic representing sweetness. [Figure 3B] FIG. 10 is a diagram showing an example of a graphic representing sweetness. [Figure 3C] FIG. 10 is a diagram showing an example of a graphic representing sweetness. [Figure 4] FIG. 10 is a diagram showing an example of a graphic representing umami. [Figure 5A] FIG. 10 is a diagram showing an example of a graphic representing bitterness. [Figure 5B] FIG. 10 is a diagram showing an example of a graphic representing bitterness. [Figure 5C] FIG. 10 is a diagram showing an example of a graphic representing bitterness. [Figure 5D] FIG. 10 is a diagram showing an example of a graphic representing bitterness. [Figure 6] FIG. 10 is a diagram showing an example of a graphic representing saltiness. [Figure 7A] FIG. 10 is a diagram showing an example of a graphic representing sourness. [Figure 7B] FIG. 10 is a diagram showing an example of a graphic representing sourness. [Figure 8] FIG. 10 is a diagram showing an example of a graphic representing spiciness. [Figure 9] FIG. 10 is a diagram showing an example of a graphic representing a taste like yogurt. [Figure 10] FIG. 10 is a diagram showing an example of a graphic representation of astringency and minerality. [Figure 11] FIG. 10 is a diagram illustrating an example of sensation change information. [Figure 12] 10 is a flowchart showing a food animation video generation operation. [Figure 13] FIG. 10 is a diagram showing an example of a flavor waveform diagram in which participants were asked to fill in shapes, colors, etc. that best represent their sensations. [Figure 14] FIG. 10 is a diagram showing an example of sensory change information obtained when eating a certain food. [Figure 15] FIG. 10 is a diagram showing an example of a frame image displayed at time T1 in generation example 1. [Figure 16] FIG. 10 is a diagram showing an example of a frame image displayed at time T2 in generation example 1. [Figure 17] FIG. 10 is a diagram showing an example of a frame image displayed at time T3 in generation example 1. [Figure 18] FIG. 10 is a diagram showing an example of a frame image displayed at time T4 in generation example 1. [Figure 19] FIG. 10 is a diagram showing another example of sensation change information. [Figure 20] FIG. 10 is a diagram showing an example of a frame image displayed at time T5 in generation example 2. [Figure 21] FIG. 10 is a diagram showing an example of a frame image displayed at time T6 in generation example 2. [Figure 22]FIG. 10 is a diagram showing an example of a frame image displayed at time T7 in generation example 2. [Figure 23] FIG. 10 is a diagram showing an example of a frame image displayed at time T8 in generation example 2. [Figure 24] FIG. 10 is a diagram showing an example of a frame image displayed at time T8 in generation example 2. DETAILED DESCRIPTION OF THE INVENTION

[0022] 1. Food animation video generation system The food animation video creation system 100 will be described below. The food animation video creation system 100 is a system that creates animations (videos) that express the sensations experienced when eating a target food. In the following description, these animations (videos) will be referred to as "food animation videos."

[0023] The foods targeted by the food animation video generation system 100 include all kinds of foodstuffs, such as beverages such as juice, coffee, alcohol, wine, and soft drinks; sweets such as chocolate, gummy candy, and ramune; and foods such as curry, stew, hayashi rice, hamburger steak, and gratin. Food animation videos visually express the sensation of eating food by primarily selecting and changing visual information such as color and shape (shape, size, etc.) according to the sensation of the food. Food animation videos allow users to experience the sensation of eating food without actually eating it, for example. Food animation videos may also include text information, audio, etc.

[0024] The food sensations expressed by food animation videos include various sensations obtained through taste, smell, hearing, sight, touch, emotions, etc., such as the taste, aroma, smell, texture, the feeling when touching the food, the sound when eating the food, and the appearance of the food. The food sensations also include information about the timing of the above sensations. Information about the timing of the sensations is information that represents temporal fluctuations in sensations, such as the timing of the occurrence of a certain sensation or the temporal change in the intensity of a certain sensation.

[0025] The configuration of the food animation video generation system 100 will be described below with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the food animation video generation system 100. The food animation video generation system 100 includes a server 1 and a terminal 3.

[0026] The server 1 is a computer system configured with a CPU, a storage device (storage devices such as RAM, ROM, HDD, and SSD), an input / output interface, a network interface, and other various interfaces. The server 1 generates a food animation video of a target food and provides it to the terminal 3. The server 1 has a storage unit 11 and an information processing unit 13.

[0027] The storage unit 11 is a part or all of the storage area of ​​the storage device constituting the server 1, and stores various information necessary for generating food animation videos, programs that realize the functions of the server 1, etc. The storage unit 11 stores, for example, conversion rule information IN1 and sensory change information IN2 as information necessary for generating food animation videos.

[0028] Conversion rule information IN1 is information that associates the sensation when eating food with information that indicates what elements (video components such as shapes, colors, and sounds) should be displayed in relation to that sensation. In other words, conversion rule information IN1 is information that indicates what components such as shapes and colors should be arranged in a food animation video in relation to the sensation of food. In other words, conversion rule information IN1 is information that defines rules for converting the sensation of food into components to be arranged in a food animation video.

[0029] For example, the aroma of food can be represented by color (e.g., the background color of a food animation video, the overall hue of the video, etc.). Specifically, for example, a lemon-like aroma can be represented by yellowish or greenish colors. A strawberry-like aroma can be represented by reddish colors. A vanilla-like aroma can be represented by white colors. The aroma of a fragrant flavor (e.g., the aroma when a specific food such as bread is baked) can be represented by the brown color of the bread (e.g., a brown color such as golden brown). Furthermore, the strength of the aroma can be represented by the shade of the color. For example, a strong aroma can be represented by a dark color, and a weak aroma can be represented by a light color.

[0030] Furthermore, the graphics displayed in the food animation video can also represent the aroma of the food. Specifically, for example, a berry-like aroma (blueberry, strawberry, grape, etc.) can be represented by a graphic representing a berry. More specifically, a strawberry-like aroma can be represented by a graphic representing the shape of a strawberry, as shown in FIG. 2A. A grape-like aroma can be represented by a graphic representing the shape of a grape, as shown in FIG. 2B. A lychee-like aroma can be represented by a graphic representing the shape of a lychee, as shown in FIG. 2C. A rose-like aroma can be represented by a graphic representing the shape of a rose composed of rose flowers, leaves, and / or stems, as shown in FIG. 2D. A bread-like aroma can be represented by a graphic representing the shape of bread, as shown in FIG. 2E. In this way, an "aroma like A" can be represented by a graphic representing the shape of the letter "A." FIGS. 2A to 2E are diagrams showing examples of graphics representing aromas.

[0031] For example, a yeast-like aroma can be represented by a circle or ellipse, which is based on the shape of yeast. In this way, the aroma and / or taste can be expressed using a color and / or shape that evokes the bacteria, substance, etc. that is the basis of the aroma and / or taste to be expressed.

[0032] A graphic representing the aroma of a food can also be colored. For example, a graphic representing the shape of a strawberry representing a strawberry-like aroma can be colored red. When representing a grape-like aroma with a purple color (or when purple grapes are used as an ingredient), a graphic representing the shape of grapes can be colored purple. When representing a grape-like aroma with a green color (or when green grapes are used as an ingredient), a graphic representing the shape of grapes can be colored green. A graphic representing the shape of bread representing a bread-like aroma can be colored brown (e.g., a color similar to the brown color of bread). In this way, when an "aroma like A" is represented by a graphic representing the shape of "A," the color of "A" and / or a color representing the aroma from "A" can be colored to the graphic. The area to be colored on the graphic may be the entire graphic, or a part of the graphic (e.g., the center of the graphic).

[0033] As described above, information about the type of fragrance, information about the color representing the fragrance, information about the figure representing the fragrance, and information about the color to be applied to the figure can be stored in association with each other in conversion rule information IN1. Information about the type of fragrance can be information that can identify the type of fragrance, such as a linguistic expression of the type of fragrance (e.g., lemon-like scent, strawberry-like scent, etc.) or an identification number representing the type of fragrance. A fragrance can be expressed by combining multiple other fragrances. Specifically, a certain fragrance can be expressed by the types and intensities of multiple other fragrances. More specifically, information about the type of fragrance can also be expressed as a (multidimensional) vector represented by multiple coordinate values, such as (intensity of fragrance 1, intensity of fragrance 2, ...).

[0034] The information about the figure representing the scent can be, for example, information that can identify the type of figure, such as information that represents the shape of each figure (e.g., vertex positions, radius of curvature of rounded parts, side lengths, angles, etc.), and figure identification information (e.g., file name, save destination, etc.). Each figure may be created in advance as image data and stored in the storage unit 11. Each figure may be generated by an artificial intelligence model such as a generative AI. In this case, the information about the figure representing the scent may include, for example, instructions (e.g., prompts) for generating the figure by the artificial intelligence model such as a generative AI. The artificial intelligence model can be, for example, a neural network composed of an input layer, an output layer, and (if necessary) an intermediate layer. It can also be an artificial intelligence model that includes a transformer.

[0035] When making an artificial intelligence model generate a figure, for example, the artificial intelligence model can be trained using training data in which basic elements (basic scents) that make up a scent are input and a figure corresponding to the basic elements is output. By inputting a vector representation of a scent into a trained model trained as described above, the trained model can generate a figure representing the scent. Alternatively, the artificial intelligence model can be trained using training data in which information about the scent is input directly and a figure corresponding to the scent is output.

[0036] The information relating to the color representing the scent and the information relating to the color to be added to the figure can be, for example, numerical information expressing the color such as the intensity, contrast, brightness, color temperature, etc. of each color element of the RGB system.

[0037] For example, the taste of a food can be represented by the shape and / or color of a graphic. Specifically, sweetness can be represented by a rounded, soft-shaped graphic or a graphic that looks like something is melting, as shown in FIGS. 3A to 3C. Sweetness can also be represented by warm colors such as red, orange, pink, salmon, and cream yellow. A rounded graphic, as shown in FIG. 3A, can represent a natural sweetness. A cloud-shaped graphic, as shown in FIG. 3B, can represent a dissolving or melting sweetness. A melting graphic, as shown in FIG. 3C, can represent a sweet, sticky, or sticky sweetness. FIGS. 3A to 3C are diagrams showing examples of graphics that represent sweetness.

[0038] For example, umami can be represented by a rounded and wide-shaped figure as shown in Figure 4. Furthermore, umami can be represented by a light orange, a dark orange, a brown, etc. Figure 4 is a diagram showing an example of a figure representing umami.

[0039] For example, bitterness can be represented by a complex, angular figure, as shown in Figures 5A to 5D. Bitterness can also be represented by a dark brown, such as a dark green, or a greenish color. A polygonal figure (e.g., a square, pentagon, or hexagon) as shown in Figure 5A can represent a preferred bitterness, such as the bitterness of tea. A spiky figure as shown in Figure 5B can represent an undesirable bitterness, such as a tingly bitterness. A rounded figure with many protrusions as shown in Figure 5C can represent an undesirable bitterness, such as a thick bitterness or a spreading, sticky bitterness. A figure with intertwined lines, such as shown in Figure 5D, can represent a complex bitterness or a vague bitterness. Figures 5A to 5D are examples of figures representing bitterness.

[0040] For example, saltiness can be represented by a geometrical crystal-like shape (e.g., a square) as shown in Figure 6. Saltiness can also be represented by white, dark blue, grayish blue, light gray, etc. Figure 6 shows an example of a shape representing saltiness.

[0041] For example, sourness can be represented by shapes with acute angles, such as a diamond or teardrop shape, or a shape with multiple sharp shapes extending radially, as shown in Figures 7A and 7B. Sourness can also be represented by yellow-based colors, such as yellow, yellow-green, and orange. Shapes with shapes such as a diamond or teardrop shape, as shown in Figure 7A, can represent a refreshing sourness. Shapes with shapes extending radially, as shown in Figure 7B, can represent a sour (sharp) sourness. Figures 7A and 7B are diagrams showing examples of shapes representing sourness.

[0042] For example, spiciness can be represented by a sharp triangle, a spiky shape, or a jagged shape, as shown in Figure 8. Also, spiciness can be represented by a mixture of red as the base color and orange, black, yellow, etc. Figure 8 is a diagram showing an example of a shape representing spiciness.

[0043] For example, a yogurt-like taste (whey-like taste) can be represented by a slightly angular flower-like figure as shown in Figure 9. A color representing the yogurt-like taste (for example, a pale white color) can be added to the flower. Figure 9 is a diagram showing an example of a figure representing a yogurt-like taste.

[0044] Furthermore, if another flavor (such as berry or strawberry) is added to the yogurt-like flavor, the flower shape representing the yogurt-like flavor can be partially colored with that other flavor (for example, a light red color for strawberry flavor).

[0045] For example, the astringency and mineral taste of black tea can be represented by a graphic representing the shape of a dull leaf, as shown in Figure 10. This graphic can be given a dull color (e.g., a grayish color), for example. Figure 10 is a diagram showing an example of a graphic representing astringency and mineral taste.

[0046] The above-mentioned information about the type of taste, information about the figure representing the taste, and information about the color (to be assigned to the figure) can be stored in association with each other in the conversion rule information IN1. The information about the type of taste can be information that can identify the type of taste, such as a linguistic expression of the type of taste (e.g., sweet, umami, bitter, salty, sour, spicy, yogurt-like, astringent, mineral, etc.), or an identification number representing the type of taste. A taste can be expressed by combining multiple other tastes. Specifically, a certain taste can be expressed by the types and intensities of multiple other tastes. More specifically, the information about the type of taste can also be expressed as a (multidimensional) vector represented by multiple coordinate values, such as (intensity of taste 1, intensity of taste 2, ...).

[0047] The information about the figures representing flavors can be information that can identify the type of figure, such as information that represents the shape of each figure (e.g., vertex positions, radius of curvature of rounded parts, side lengths, angles, etc.), and figure identification information (e.g., file name, save destination, etc.). Each figure may be created in advance as image data and stored in the storage unit 11. Each figure may be generated by an artificial intelligence model such as a generative AI. In this case, the information about the figure representing the aroma may include, for example, a command (e.g., a prompt) for generating the figure by an artificial intelligence model such as a generative AI.

[0048] When generating a graphic from an artificial intelligence model, for example, the artificial intelligence model can be trained using training data in which basic elements (basic flavors) that make up a flavor are input and a graphic corresponding to the basic element is output. By inputting a vector representation of a flavor into the trained model trained as described above, the trained model can generate a graphic representing the flavor. Alternatively, the artificial intelligence model can be trained using training data in which information about a flavor is input directly and a graphic corresponding to the flavor is output.

[0049] The information relating to the color that represents the taste can be, for example, numerical information that represents the color such as the intensity, contrast, brightness, and color temperature of each color element of the RGB system.

[0050] The sensory change information IN2 is information that represents changes in the sensory sensation over time when the target food is eaten. Specifically, the sensory change information IN2 includes information about the type of sensory sensation experienced at each time point after eating the target food, and information about the intensity of the sensory sensation at each time point. More specifically, the sensory change information IN2 is information that associates the time elapsed since eating the target food, information about the type of sensory sensation experienced at each time point, and information about the intensity of the sensory sensation at each time point. The information about the type of sensory sensation is, for example, information that can identify the type of sensory sensation, such as linguistic information that represents the sensory sensation or a sensory identification number. The information about the intensity of the sensory sensation is, for example, a numerical value that represents the strength of the sensory sensation.

[0051] If the above-mentioned sensory change information IN2 is plotted on a coordinate system with time on the horizontal axis and the intensity of the sensory change on the vertical axis, a graph like that shown in FIG. 11 will be obtained. The sensory change information IN2 shown in FIG. 11 is sensory change information IN2 for a certain orange juice. A graph like that shown in FIG. 11 can also be called a "flavor waveform diagram" because it shows the temporal change in the sensory (flavor) of a food product as a waveform. FIG. 11 is a diagram showing an example of sensory change information IN2.

[0052] Since each food produces a different sensation and the sensation changes over time differently, the sensation change information IN2 is generated for each food for which a food animation video is generated. On the other hand, the conversion rule information IN1 defines a unified rule for converting sensations into visual information, so it is generally common to all foods. However, if it is preferable to have different conversion rules for each type of food or each user, for example, multiple copies of conversion rule information IN1 may be stored.

[0053] The storage unit 11 stores the food animation video generated by the server 1. This allows the food animation video to be provided to users of the terminals 3 as many times as necessary.

[0054] The information processing unit 13 is configured by the CPU constituting the server 1, a part of the storage area of ​​the storage device, and various interfaces, and executes various information processing in the server 1. The information processing unit 13 generates a food animation video of the target food using the above conversion rule information IN1 and sensory change information IN2. The information processing unit 13 stores the generated food animation video in the storage unit 11 and / or transmits it to a terminal 3 used by a user who desires the food animation video. The information processing unit 13 realizes part or all of the above information processing by executing a program stored in the storage unit 11. Furthermore, part or all of the information processing may be realized by hardware.

[0055] The terminal 3 is a computer system configured with a CPU, storage devices (storage devices such as RAM, ROM, HDD, and SSD), and various interfaces, and is used by a user, etc. The terminal 3 is, for example, a computer system such as a smartphone, a tablet terminal, a PDA, a portable game console, a portable terminal such as a notebook computer, or a personal computer.

[0056] A user or the like uses a terminal 3 to request the server 1 to provide a food animation video of a desired food. The server 1, which receives this request, generates the requested food animation video and transmits it to the terminal 3, or transmits the desired food animation video stored in the storage unit 11 to the terminal 3. The terminal 3, which receives the food animation video, displays the received food animation video on its display device (for example, a display).

[0057] 2. Operation of the food animation video generation system The operation of generating a food animation video in the food animation video generation system 100 having the above configuration will be described below with reference to Fig. 12. Fig. 12 is a flowchart showing the operation of generating a food animation video in the food animation video generation system 100.

[0058] To generate a food animation video, first, the necessary information, i.e., conversion rule information IN1 and sensory change information IN2, is generated and stored in the storage unit 11 of the server 1 (step S1). This information is generated based on various data related to the sensory perception of the target food, such as sensory evaluation by a user or the like eating the target food, video data capturing the target food, the target food packaging, facial expressions and voices of the user or the like eating the target food, measurement results of the target food, the composition of ingredients of the target food, and recipes for producing the target food.

[0059] A sensory evaluation by a user or the like eating a target food includes, for example, having the user or the like eat the target food and create a flavor waveform diagram as shown in FIG. 11. Also, as shown in FIG. 13, the user or the like is asked to fill in the flavor waveform diagram with a shape, color, etc. that best represents each sensation. FIG. 13 is a diagram showing an example of a flavor waveform diagram in which the user or the like has been asked to fill in the shape, color, etc. that best represents each sensation. Such a sensory evaluation can provide, for example, information about the taste, aroma, and texture of the target food, as well as the timing at which these sensations occur.

[0060] It is preferable that the sensory evaluation be conducted with the participation of a sensory evaluation expert (a user who is familiar with the sensory aspects of food). This allows for the acquisition of not only an abstract sensation such as "sweet," but also more specific and detailed sensations, such as "refreshing and modest sweetness," "clear flavor without impurities," "citrus-like aroma," "smooth mouthfeel," "top scent of ●● + last scent of △△," and "yellow color like the peel of a young lemon," i.e., information on slightly different sensations. On the other hand, for luxury items consumed by many users, such as wine, sake, chocolate, and coffee, there is often already a common language used to describe flavor, so sensory evaluation by an expert may not be necessary.

[0061] Video data can be obtained, for example, by capturing the appearance of the target food, the packaging of the target food (including the label portion containing product information), a user eating the target food, etc. The video data can provide information such as the image the user gets from the food, the aroma and taste of the food, the user's impressions of the food (e.g., whether the food is perceived as delicious or unappealing), the strength of these impressions, the timing at which these impressions are felt (the timing of the aroma, taste, etc., and changes over time), information about the sounds made when the food packaging is opened (e.g., the sound of the bag being opened), information about the sounds made by the user while eating the target food (e.g., the "crunch" sound of eating hard food), the ingredients of the target food (the combination of ingredients), and a recipe for making the target food. When the video data is obtained, the user is asked to enter shapes, colors, sounds, etc. that represent the sensations felt by the user in each scene of the video data.

[0062] The measurement results of the target food can be obtained, for example, by measuring the target food with various measuring devices. By measuring the target food with various measuring devices, measurement results such as information about the temperature, viscosity, hardness, and nutritional components of the target food can be obtained.

[0063] The ingredient combination and recipe of the target food can be obtained, for example, from video data of the target food's packaging, instructions for producing the target food, etc. From the ingredient combination and recipe of the target food, information on the aroma, taste, nutritional components, etc. of the target food can be obtained.

[0064] Specifically, the conversion rule information IN1 and the sensory change information IN2 can be generated as follows. First, the various data related to the sensory sensation of the target food are collected. This data is subjected to statistical processing, analysis, etc. to extract the sensations experienced when eating the food and the changes in each sensation over time. Then, based on the extracted sensations and their changes over time, the time elapsed since eating a certain food, information representing the type of sensation extracted for that food, and information representing the intensity of that sensation at each time are calculated, and these pieces of information are associated to generate the sensory change information IN2 related to the certain food. As explained above, sensory change information IN2 is generated for each target food.

[0065] Furthermore, by statistically processing and analyzing the figures, colors, sounds, etc. representing sensations entered by the user or the like when collecting the various data described above, the figure, color, sound, etc. that best represents each sensation extracted by the analysis of the various data is determined. Information representing the extracted type of sensation is associated with information regarding the figure, color, sound, etc. that has been determined to best represent the sensation, and conversion rule information IN1 is generated.

[0066] In addition, if a large amount of the above-mentioned various data regarding food can be obtained, an artificial intelligence model can be trained using the large amount of various data, and the conversion rule information IN1 and sensory change information IN2 can be generated using the trained model trained using the various data.

[0067] After generating the conversion rule information IN1 and the sensory change information IN2, the information processing unit 13 uses this information to generate a food animation video. First, the information processing unit 13 determines the components such as shapes, colors, and sounds to be included in the images (hereinafter referred to as frame images) to be displayed at each time point of the food animation video, and their arrangement (size, number, density, shading, placement position, etc.) (step S2). The information processing unit 13 can determine the components to be arranged and their arrangement based on information representing the type of sensation and information representing the intensity of the sensation associated with each time point in the sensory change information IN2. Furthermore, the information processing unit 13 determines the shapes, colors, sounds, etc. to be arranged in the frame images based on the conversion rule information IN1.

[0068] Furthermore, the information processing unit 13 can determine the colors (background color, hue, etc.) in the frame images and the style of the figures displayed in the frame images based on the attributes of the user watching the food animation video. For example, the style can be determined based on the purpose of the user watching the video, the user demographics (e.g., age, gender, income, etc.) who often eat the target food, the image that the user expects of the target food (luxury, common, etc.), etc.

[0069] Specifically, for example, when targeting users with impaired taste, more realistic and / or geometric shapes, such as those using many polygons, or colors and shapes that allow for a clearer visual recognition of the aroma and flavor, can be used. For example, when targeting users of working age who have a relatively high income and desire a luxurious experience, more artistic colors and shapes (including art elements) can be used. For example, when targeting young children, more cute (cartoon-like) colors and shapes can be used.

[0070] The attributes of users who watch food animation videos can be determined, for example, by having the user enter various information in a questionnaire using terminal 3 before playing the food animation video, and based on the analysis results of the questionnaire. For example, a comment section can be provided on the screen (user interface) displaying the food animation video, and the attributes can be determined based on the content of comments posted in the comment section. For example, the user's face can be photographed using a camera provided on terminal 3, and the attributes can be determined based on the results of facial recognition of the photographed face. In addition, the target food can be associated in advance with the attributes of users who are the main consumers of the food.

[0071] The rules for determining what style of colors and figures to use for a user's attributes may be stored in advance in the conversion rule information IN1, for example, or may be included as an algorithm in a program for generating food animation videos.

[0072] After determining the colors and shapes to be arranged in the frame images as described above, the information processing unit 13 determines the arrangement state of the colors and shapes determined in step S2 (step S3). The information processing unit 13 can determine the arrangement positions of the colors, shapes, etc. in the frame images of the food animation video based on the feeling at the timing when the frame image is displayed when the target food is eaten.

[0073] Specifically, the information processing unit 13 can arrange colors and / or shapes representing aromas in the upper part of the frame image. On the other hand, the information processing unit 13 can arrange colors and / or shapes representing tastes in the lower part of the frame image. This corresponds to the relative positions of the user's nose and mouth. That is, humans sense aromas with their noses, which are located in the center of their faces, and sense tastes (taste food) with their mouths, which are located below their noses (lower part of their faces). Therefore, by arranging colors and / or shapes representing aromas in the upper part of the frame image and colors and / or shapes representing tastes in the lower part of the frame image, the user can intuitively recognize which colors and / or shapes in the frame image represent aromas or tastes.

[0074] Furthermore, aromas are composed of gaseous molecules, and tastes are perceived when something with a shape (i.e., something composed mainly of molecules that do not vaporize) is placed in the mouth. That is, aromas are composed of molecules with relatively small molecular weights, and tastes are composed of molecules with relatively large molecular weights. Molecules with relatively small molecular weights tend to rise or continue to float in the atmosphere, while molecules with relatively large molecular weights tend to sink. Therefore, the position of the color and / or shape representing the aroma / taste of the molecule (whether to place it at the top or bottom of the frame image) can be determined depending on the molecular weight of the molecule contained in the target food. The molecules contained in the target food can be identified, for example, by various measurements of the food.

[0075] Furthermore, the information processing unit 13 can determine the placement position of the color and / or graphic based on the impression of the taste of the target food. For example, a graphic representing a taste that lasts for only a short time, a light taste, or a taste that spreads over a wide area can be placed at the top of the frame image. For example, a graphic representing a taste that is coated with an upper taste can be placed around the middle of the frame image. On the other hand, a graphic representing a taste that lasts for a long time, a base taste, a heavy taste, or a taste that gradually builds up can be placed at the bottom of the frame image.

[0076] The information processing unit 13 can determine the number and / or density of the figures to be displayed in the frame image based on the sensations experienced when eating the target food at the time the frame image is displayed. For example, the number and / or density of the figures can be determined based on the strength of the aroma and / or taste. Specifically, a weak aroma and / or taste, or an aroma and / or taste that is not noticeable overall, can be represented by reducing the number of figures and / or decreasing the density of the figures displayed. On the other hand, a strong aroma and / or taste, or an aroma and / or taste that is noticeable overall, can be represented by increasing the number of figures and / or increasing the density of the figures displayed.

[0077] The information processing unit 13 can determine the size of the graphic to be displayed in the frame image based on the sensation at the time when the frame image is displayed when the target food is eaten. For example, the size of the graphic can be determined based on the strength of the aroma and / or taste. Specifically, a weak aroma and / or taste, or an aroma and / or taste that is not noticeable overall, can be represented by making the graphic smaller. On the other hand, a strong aroma and / or taste, or an aroma and / or taste that is noticeable overall, can be represented by making the graphic larger.

[0078] The information processing unit 13 can determine the colors of the frame images (background color, hue, etc.) and the colors to be applied to the figures displayed in the frame images based on the sensations experienced when eating the target food. For example, the colors can be determined based on the strength of the aroma and / or taste. Specifically, a weak aroma and / or taste, or an aroma and / or taste that is not noticeable overall, can be represented by a light color. On the other hand, a strong aroma and / or taste, or an aroma and / or taste that is noticeable overall, can be represented by a dark color.

[0079] The rules for determining the placement state of colors and / or shapes in frame images, such as the placement position, color shade, number of shapes, and size of shapes, may be stored in advance in the conversion rule information IN1, for example, or may be included as an algorithm in a program for generating food animation videos.

[0080] After determining the color and graphic placement positions, color shades, number of graphics, and graphic sizes, the information processing unit 13 generates a frame image by placing the graphics in the frame image according to the determined rules and coloring the frame image (step S4).

[0081] After generating a frame image for a certain time, the information processing unit 13 determines whether or not frame images for the playback time of the food animation video have been generated (step S5). If frame images for the playback time have not been generated ("No" in step S5), the information processing unit 13 repeats steps S2 to S4 above to continue generating frame images for each time.

[0082] On the other hand, if frame images for the playback time have been generated ("Yes" in step S5), the information processing unit 13 generates a video file of the food animation video by arranging the generated frame images in chronological order (step S6). The information processing unit 13 stores the generated video file in the storage unit 11.

[0083] 3. Food animation video generation example 1 Hereinafter, examples of generating food animation videos using the above-mentioned food animation video generation method will be described. First, a generation example (generation example 1) of a food animation video of a food (orange juice) represented by sensation change information IN2 shown in Figs. 11 and 13 will be described.

[0084] The food (orange juice) represented by the sensory change information IN2 shown in Figures 11 and 13 develops a strong sweetness for a short period of time when the user starts eating (drinking), followed by a strong sourness for a short period of time. On the other hand, a relatively strong saltiness develops over a long period of time after the user starts eating (drinking). Furthermore, a relatively weak bitterness develops that lasts for a long period of time after a predetermined time has elapsed since the user started eating (drinking). Furthermore, a relatively strong orange aroma (orange flavor) develops over a long period of time after the user started eating (drinking).

[0085] In Generation Example 1, the attributes of users who watch food animation videos are assumed to be users who need to visually clearly show the aroma and taste of food, such as users with taste disorders. For this reason, the colors and shapes used in Generation Example 1 are those that can visually clearly show the aroma and taste of food, such as primary colors and geometric shapes that make extensive use of polygons (for example, shapes shown in Figures 3A to 8).

[0086] The information processing unit 13 references the sensation change information IN2 shown in FIG. 14 and determines that a strong sweetness, a weak saltiness, and a weak orange scent are present at the time T1 has elapsed since the food was eaten. Therefore, the information processing unit 13 references the conversion rule information IN1 and determines to place a large graphic representing sweetness and a small graphic representing saltiness in the frame image at time T1. The information processing unit 13 also determines to use a light orange color representing a weak orange scent as the background color of the frame image. Furthermore, since the strong sweetness lasts for only a short time, it is determined to place it at the top of the frame image. On the other hand, since the weak saltiness lasts for a relatively long time, it is determined to place it at the bottom of the frame image. As a result, an image such as that shown in FIG. 15 is generated as the frame image to be displayed at time T1. FIG. 14 is a diagram illustrating an example of sensation change information IN2 obtained when a certain food is eaten. FIG. 15 is a diagram illustrating an example of a frame image to be displayed at time T1 in generation example 1.

[0087] The information processing unit 13 references the sensation change information IN2 shown in FIG. 14 and determines that a strong sourness, a moderate saltiness, a moderate orange scent, and a weak sweetness are present at the time T2 after eating the food. Therefore, the information processing unit 13 references the conversion rule information IN1 and determines to place a large graphic representing sourness, a medium-sized graphic representing saltiness, and a small graphic representing sweetness in the frame image at time T2. The information processing unit 13 also determines to use a medium-dark orange color representing a moderate orange scent as the background color of the frame image. Furthermore, since strong sourness and weak sweetness last only for a short time, the information processing unit 13 determines to place the graphics representing these at the top of the frame image. On the other hand, since saltiness lasts relatively long, the information processing unit 13 determines to place the graphics representing this at the bottom of the frame image. As a result, an image such as that shown in FIG. 16 is generated as the frame image to be displayed at time T2. FIG. 16 is a diagram illustrating an example of a frame image to be displayed at time T2 in generation example 1.

[0088] The information processing unit 13 references the sensation change information IN2 shown in FIG. 14 and determines that a relatively strong salty taste, a relatively strong orange scent, a weak bitter taste, and a lingering weak sweet taste are present at the time T3 after eating the food. Therefore, the information processing unit 13 references the conversion rule information IN1 and determines that a relatively large figure representing the salty taste, a relatively small figure representing the bitter taste, and a small figure representing the lingering sweet taste should be placed in the frame image at time T3. The information processing unit 13 also determines that a relatively deep orange color representing a relatively strong orange scent should be used as the background color of the frame image. Furthermore, the information processing unit 13 determines that the figures representing the salty taste, the bitter taste, and the lingering sweet taste should be placed at the bottom of the frame image. As a result, an image such as that shown in FIG. 17 is generated as the frame image to be displayed at time T3. FIG. 17 is a diagram illustrating an example of a frame image to be displayed at time T3 in generation example 1.

[0089] The information processing unit 13 references the sensation change information IN2 shown in FIG. 14 and determines that a moderate salty taste, a weak orange scent, a weak bitter taste, and a lingering weak sweet taste are present at the time T4 after eating the food. Therefore, the information processing unit 13 references the conversion rule information IN1 and determines that a medium-sized figure representing saltiness, a small figure representing bitterness, and a small figure representing lingering sweetness should be placed in the frame image at time T4. The information processing unit 13 also determines that a light orange color representing the orange scent should be used as the background color of the frame image. Furthermore, the information processing unit 13 determines that the figures representing saltiness, bitterness, and lingering sweetness should be placed at the bottom of the frame image. As a result, an image such as that shown in FIG. 18 is generated as the frame image to be displayed at time T4. FIG. 18 is a diagram illustrating an example of a frame image to be displayed at time T4 in generation example 1.

[0090] As described above, the food animation video of Generation Example 1 is generated by generating frame images for the playback time and arranging the generated frame images in chronological order.

[0091] 4. Food animation video generation example 2 Next, a description will be given of a generation example (generation example 2) of a food animation video of a food (wine) represented by sensation change information IN2 shown in Fig. 19. Fig. 19 is a diagram showing another example of sensation change information IN2.

[0092] The food (wine) represented by the sensory change information IN2 in FIG. 19 emits a strong grape-like aroma and a moderate strawberry-like aroma at the beginning of eating (drinking), then emits a strong lychee-like aroma within a short period of time, and then emits a relatively strong rose-like aroma within a short period of time thereafter. Around the middle of the meal, a yeast-like aroma emerges, which then changes to a bread-like aroma. The grape-like aroma and strawberry-like aroma linger until the end. Regarding the taste of the food (wine) represented by the sensory change information IN2 in FIG. 19, a yogurt (whey)-like taste gradually increases, and then the yogurt (whey)-like taste gradually decreases. Finally, a black tea-like astringency emerges.

[0093] In Generation Example 2, the attributes of users who watch food animation videos are assumed to be those who like wine as a preference and want to enjoy food as art. Therefore, the colors and shapes used in Generation Example 2 are highly artistic (for example, shapes shown in Figures 2A to 2E, 9, and 10).

[0094] The information processing unit 13 refers to the sensation change information IN2 shown in FIG. 19 and determines that a strong grape-like scent, a medium strawberry-like scent, and a weak lychee-like scent are present at the time T5 has elapsed since eating this food. Therefore, the information processing unit 13 refers to the conversion rule information IN1 and determines to place multiple (two) large figures representing grapes and (one) large figure representing a strawberry in the frame image at time T5. Furthermore, multiple (three) figures representing the shapes of lychees drawn with thin lines are displayed. Note that the figure representing the shape of the lychee is drawn with thick lines in the frame image at a later time, indicating that the lychee-like scent has become stronger. The information processing unit 13 places the figures representing the scents at the top of the frame image.

[0095] Furthermore, this food has a refreshing and light flavor in the early stages of eating. Therefore, the information processing unit 13 determines to use a bright, vibrant, and pop color that evokes a tropical atmosphere, such as cherry blossom pink or orange, as the background color of the frame image at time T5.

[0096] As a result of the above, an image such as that shown in Fig. 20 is generated as the frame image to be displayed at time T5. Fig. 20 is a diagram showing an example of the frame image to be displayed at time T5 in generation example 2.

[0097] The information processing unit 13 references the sensation change information IN2 shown in FIG. 19 and can confirm that, at the time T6 after eating the food, the lychee-like scent has disappeared and the grape-like scent and strawberry-like scent continue to be present. It can also confirm that a relatively strong rose-like scent and a medium-strength yogurt (whey) taste are present. Therefore, the information processing unit 13 references the conversion rule information IN1 and determines that the frame image at time T6 should include multiple (two) medium-sized figures representing grapes, one medium-sized figure representing a strawberry, and multiple (four) medium-sized figures representing roses. The information processing unit 13 places these figures representing the scents at the top of the frame image. The information processing unit 13 colors the rose-shaped figure to indicate a strong rose-like scent. The information processing unit 13 also places a slightly angular flower-like figure representing a yogurt-like taste at the bottom of the frame image.

[0098] As a result of the above, an image such as that shown in Fig. 21 is generated as the frame image to be displayed at time T6. Fig. 21 is a diagram showing an example of the frame image to be displayed at time T6 in generation example 2.

[0099] The information processing unit 13 references the sensation change information IN2 shown in FIG. 19 and determines that, at the time T7 after eating the food, the rose-like scent has disappeared and the grape-like scent and strawberry-like scent continue to be present. It also confirms the presence of a moderate yeast-like scent and a relatively strong yogurt (whey)-like taste. Therefore, the information processing unit 13 references the conversion rule information IN1 and determines that the frame image at time T7 should include multiple (two) medium-sized figures representing grapes, one medium-sized figure representing a strawberry, and multiple (eight) small circular and oval figures. The information processing unit 13 places these scent-representing figures at the top of the frame image.

[0100] From the middle stage onwards, when a certain amount of time has passed since the start of eating, the food's color appears to match the color of its exterior. Therefore, information processing unit 13 determines that the background color of the frame images from time T7 onwards will be a bright, vibrant, and pop color of red, which is the color of the food (wine).

[0101] Furthermore, the information processing unit 13 places a slightly angular flower-like figure representing a yogurt-like taste at the bottom of the frame image. The size of the slightly angular flower-like figure at time T7 gradually increases from time T6, and it comes to occupy almost the entire bottom of the frame image. In this way, the slightly angular flower-like figure representing a yogurt-like taste is displayed so that its proportion of the frame image gradually increases over time from a small proportion (changing from weak to strong taste) or gradually decreases over time from a large proportion (changing from strong to weak taste), depending on the strength of the taste indicated in the sensation change information IN2.

[0102] As a result of the above, an image such as that shown in Fig. 22 is generated as the frame image to be displayed at time T7. Fig. 22 is a diagram showing an example of a frame image to be displayed at time T7 in generation example 2.

[0103] The information processing unit 13 references the sensation change information IN2 shown in FIG. 19 and can confirm that the grape-like and strawberry-like aromas continue to be present at time T8 after the food is eaten. It can also be confirmed that the medium yeast-like aroma has changed to a bread-like aroma. Furthermore, it can be confirmed that the yogurt (whey)-like taste reaches its maximum between time T7 and time T8 and is present in a slightly reduced state at time T8. Therefore, the information processing unit 13 references the conversion rule information IN1 and arranges, in the frame image at time T8, multiple (two) medium-sized figures representing grapes, one medium-sized figure representing a strawberry, and a figure representing bread. Because the yeast-like aroma has changed to a bread-like aroma, the information processing unit 13 generates a figure representing the shape of bread by, for example, merging multiple circles and ellipses representing the yeast-like aroma into one to form the shape of bread between time T7 and time T8. It also applies a color reminiscent of the toasted color of bread (e.g., golden brown) to the figure, circle, and ellipse representing the shape of bread. The information processing unit 13 places these figures representing the scents at the top of the frame image.

[0104] Furthermore, information processing unit 13 places a slightly angular flower-like figure representing a yogurt-like taste at the bottom of the frame image. The yogurt (whey)-like taste reaches its maximum between time T7 and time T8 and then decreases slightly at time T8. Accordingly, the size of the slightly angular flower-like figure gradually increases from time T7 to time T8 and reaches its maximum, then gradually decreases, but still occupies almost the entire bottom of the frame image at time T8.

[0105] As a result of the above, an image such as that shown in Fig. 23 is generated as the frame image to be displayed at time T7. Fig. 23 is a diagram showing an example of a frame image to be displayed at time T8 in generation example 2.

[0106] The information processing unit 13 references the sensation change information IN2 shown in FIG. 19 and can confirm that at the point in time T9 after eating the food, the bread-like aroma has disappeared, and the grape-like aroma and strawberry-like aroma continue to be present. These aromas that remain until the end constitute the "aftertaste." It can be confirmed that the yogurt (whey)-like taste gradually decreases from time T8 to time T9. Furthermore, the tea-like astringency gradually increases from time T8 to time T9.

[0107] Therefore, the information processing unit 13 refers to the conversion rule information IN1 and places multiple (two) large figures representing the shape of grapes and (one) large figure representing the shape of a strawberry in the frame image at time T9. At time T9, a long time has passed since the food was eaten, the grape-like scent and strawberry-like scent that were emitted from the beginning remain as "lingering aromas." Therefore, the information processing unit 13 displays these figures representing the scents in an emphasized manner. Specifically, the figure representing the shape of grapes and the figure representing the shape of strawberries are displayed larger compared to the strength of the scent indicated in the sensation change information IN2, and the interiors of these figures are colored (the figure representing the shape of grapes is colored purple, and the figure representing the shape of strawberries is colored red). The information processing unit 13 places these figures representing the scents at the top of the frame image.

[0108] The information processing unit 13 places a slightly angular flower-like figure representing the yogurt-like taste at the bottom of the frame image. Because the yogurt (whey)-like taste decreases from time T8 to time T9, the size of the slightly angular flower-like figure is correspondingly reduced from that at time T8. In addition, a leaf-shaped figure representing the astringency that occurred from time T8 to time T9 is placed at the bottom of the frame image. In addition, a dull color (e.g., a grayish color) is applied to this leaf-shaped figure. Because the astringency gradually increases from time T8 to time T9, the dull leaf-shaped figure is displayed so that it gradually becomes larger from time T8 to time T9 accordingly.

[0109] As a result of the above, an image such as that shown in Fig. 24 is generated as the frame image to be displayed at time T9. Fig. 24 is a diagram showing an example of a frame image to be displayed at time T8 in generation example 2.

[0110] By generating frame images for the playback time as described above and arranging the generated frame images in chronological order, the food animation video of Generation Example 2 is generated.

[0111] 5. Summary In the food animation video generation system 100, the frame images corresponding to each frame of the food animation video include shapes, colors, sounds, etc. that represent the sensations at each time (the display time of each frame) after eating the target food. The shapes, colors, sounds, etc. that represent the sensations at each time are determined based on sensory change information IN2 that represents the time-series changes in the sensations when eating the target food. The arrangement of the determined shapes, colors, etc. is also determined based on the sensations when eating the target food. In food animation videos that visually represent the sensations when eating food, not only the displayed colors, shapes, etc. but also how they are arranged in each frame greatly influences the impression given to the user. Therefore, each frame of the food animation video generated by the present disclosure can appropriately express the sensations of the target food at the timing corresponding to the display time of the frame using shapes, colors, sounds, etc.

[0112] Furthermore, a food animation video is generated by generating a plurality of frame images corresponding to each frame of the food animation video for a predetermined time period after eating the target food, and arranging the generated plurality of frame images in chronological order. As described above, the frame images corresponding to each frame of the food animation video contain shapes and colors that correspond to the sensation of the food at the timing corresponding to the display time of the image. Therefore, the food animation video generated by arranging a plurality of such frame images in chronological order visually represents the time-series changes in the sensation when eating the target food. Since the time-series changes in the sensation when eating food are an important element in representing the sensation of food, the food animation video generated by the above method can appropriately represent the sensation when eating food.

[0113] The food animation videos generated as described above can be used in the following situations. For example, when developing a new food product, it can be difficult to get others to understand the sensation of the new food product even if you explain it in words. When promoting or presenting a new food product, the audience will be a client or other person. In food development, the audience will be other developers.

[0114] In this case, by showing the other person a food animation video of the new food, the other person can intuitively understand the sensation of eating the new food. In other words, by using a food animation video, the other person can easily understand the sensation of eating the food. As a result, a great advertising effect can be obtained for the new food. Also, if it is in the development stage, a unified sensation can be easily shared among multiple developers.

[0115] For example, even if you are unable to go out due to the spread of an epidemic or the like and are unable to go to the store and sample the food in the store, you can still experience the same sensation of eating the food as if you were tasting it in the store by simply viewing the food animation video at home using terminal 3.

[0116] For example, when food is displayed in a store or on a website, if only the food, a photo of the food, and / or the food packaging are displayed, or if only a description is presented along with the food, it is difficult to convey the sensation of eating the food, and customers may not be able to decide whether to purchase the food. In other words, if only the food, a photo of the food, the food packaging, and a description are displayed, it is difficult to increase customers' purchasing motivation. Therefore, when food is displayed in a store or on a website, a food animation video can be played and displayed together with the food, customers can easily understand the sensation of eating the food. As a result, customers' purchasing motivation for the food can be increased.

[0117] For example, if you want to purchase food based not only on its appearance but also on its taste, if a food animation video is played and displayed along with the food, you can decide whether or not to purchase the food without tasting it.

[0118] For example, if you are considering purchasing a food you have never tried, if an animation video is played and displayed along with the food, you can get a feel for the food and easily decide whether or not to buy it. This reduces the chances of buying a food you have never tried and then not getting the experience you expected.

[0119] For example, if a user is unable to sense food due to a disease such as tongue cancer or a congenital disorder, they may not be able to enjoy their meal even if they eat it because they cannot sense the food. By showing a food animation video to a user who is unable to sense food, the user can feel the sensation of the food they are eating, and so the user can enjoy their meal.

[0120] For example, when food is shown in a game, television, movie, etc., a food animation video can be played and displayed at the same time, making it easier for users watching the game, television, movie, etc. to recognize the texture of the food being shown.

[0121] For example, by showing a food animation video to a picky eater, the child can enjoy eating the food and come to like the food, which can reduce the child's food preferences.

[0122] 6. Other Embodiments Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible within the scope of the gist of the invention. In particular, the multiple embodiments and modifications described in this specification can be arbitrarily combined as necessary. (A) The processing content in each step of the flowchart shown in FIG. 9 and / or the execution order of each step can be changed as appropriate without departing from the spirit of the invention.

[0123] (B) The conversion rule information IN1 and sensory change information IN2 that have been generated may be modified. For example, when a large number of sensory evaluation results, measurement results from a measuring device, video data, etc. are obtained for a certain food, an artificial intelligence model can learn from this large number of data and the information obtained based on them, and the learned artificial intelligence model can output more optimal conversion rule information IN1 and sensory change information IN2. By replacing the conversion rule information IN1 and sensory change information IN2 stored in memory unit 11 with the conversion rule information IN1 and sensory change information IN2 output from the artificial intelligence model, the generated conversion rule information IN1 and sensory change information IN2 can be modified to be more optimal.

[0124] Furthermore, for example, a user may be asked to evaluate (for example, evaluate whether or not the food animation video adequately represents the sensation of the food) a food animation video generated based on the current conversion rule information IN1 and sensory change information IN2, and the current conversion rule information IN1 and sensory change information IN2 may be modified based on the evaluation results.

[0125] (C) Conversion rule information IN1 and sensory change information IN2 may be generated for each user. For the same sensation obtained when eating food, the shape, color, sound, etc. that is considered to best represent that sensation may differ depending on the user. Therefore, by generating conversion rule information IN1 and sensory change information IN2 for each user, it is possible to generate a food animation video that is optimal for each user.

[0126] (D) The food animation video is not limited to video data. For example, the food animation video may be composed of a plurality of frame images and a program that displays the frame images in chronological order. In this case, the information processing unit 13 of the server 1 generates the plurality of frame images and the program as the food animation video, and then transmits the program to the terminal 3. The terminal 3 executes the program, whereby the plurality of frame images are sequentially transmitted from the server 1 to the terminal 3. The terminal 3 sequentially receives the frame images and displays them on the display device, thereby allowing the food animation video to be played and displayed on the terminal 3.

[0127] (E) The food animation video may be configured by a program that generates a plurality of frame images and displays the generated frame images in chronological order. In this case, after the program is generated by the information processing unit 13 of the server 1, the server 1 transmits the program to the terminal 3, and the terminal 3 executes the received program, whereby the terminal 3 or the server 1 generates a plurality of frame images and displays the generated frame images in chronological order, thereby playing and displaying the food animation video on the terminal 3.

[0128] (F) The food animation video may end with a still image, such as an image that comprehensively expresses the sensation of eating the food, after the animation video has been played. [Industrial Applicability]

[0129] The present invention can be widely applied to the generation of videos that express the sensations experienced when eating food. [Explanation of symbols]

[0130] 1: Server 3: Terminal 11: Storage section 13: Information Processing Department 100: Food animation video generation system IN1: Conversion rule information IN2: Sensory change information

Claims

1. a step in which a computer determines a graphic and a color representing the sensation at each time point after eating the target food based on sensation change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation when eating the target food; generating the image; a step of repeatedly executing the steps of determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; generating a food animation video by arranging the generated plurality of images in time series; Equipped with the sensations include the smell and taste of the food of interest; The graphic representing the aroma of the target food is placed at the top of the image, and the graphic representing the taste of the target food is placed at the bottom of the image; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; How to generate food animation videos.

2. a step in which a computer determines a graphic and a color representing the sensation at each time point after eating the target food based on sensation change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation when eating the target food; generating the image; a step of repeatedly executing the steps of determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; generating a food animation video by arranging the generated plurality of images in time series; Equipped with determining a background color for the image based on the sensory characteristics of the target food; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; How to generate food animation videos.

3. a step in which a computer determines a graphic and a color representing the sensation at each time point after eating the target food based on sensation change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation when eating the target food; generating the image; a step of repeatedly executing the steps of determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; generating a food animation video by arranging the generated plurality of images in time series; Equipped with the sensations include the smell and taste of the food of interest; The graphic representing the aroma of the target food is placed at the top of the image, and the graphic representing the taste of the target food is placed at the bottom of the image; determining a background color for the image based on the sensory characteristics of the target food; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; How to generate food animation videos.

4. 4. The food animation moving image generating method according to claim 1, wherein the size of the figure in the image is determined based on the sensation when eating the target food.

5. The food animation moving image generating method according to any one of claims 1 to 3, wherein the number of the figures to be arranged in the image is determined based on the sensation when eating the target food.

6. A food animation video generation method described in any one of claims 1 to 3, wherein the sensory change information includes information regarding the type of sensory sensation that appears at each time after eating the target food, and information regarding the intensity of the sensory sensation at each time.

7. the sensation includes the taste of the food of the subject; 4. The food animation moving image generating method according to claim 1, wherein the shape and color of the figure are determined based on the taste of the target food.

8. The food animation video generation method of claim 7, wherein the size, color intensity, number of figures to be displayed in the image, and / or the arrangement of multiple figures in the image are determined based on the intensity of the taste of the target food.

9. The food animation moving image generating method according to claim 2 or 3, wherein the intensity of the background color of the image is determined based on the intensity of the aroma of the target food.

10. 4. The food animation video generation method according to claim 1, wherein the color and the style of the figure are determined based on attributes of a user who watches the food animation video.

11. A program that causes a computer to execute the food animation video generation method according to any one of claims 1 to 3.

12. The device includes a storage unit and an information processing unit, The memory unit stores sensory change information representing a time-series change in sensation when eating a target food product, The information processing unit determining a graphic and a color representing the sensation at each time point after eating the target food based on the sensation change information; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; the sensations include the smell and taste of the food of interest; The graphic representing the aroma of the target food is placed at the top of the image, and the graphic representing the taste of the target food is placed at the bottom of the image; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; Food animation video generation server.

13. The device includes a storage unit and an information processing unit, The memory unit stores sensory change information representing a time-series change in sensation when eating a target food product, The information processing unit determining a graphic and a color representing the sensation at each time point after eating the target food based on the sensation change information; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; determining a background color for the image based on the sensory characteristics of the target food; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; Food animation video generation server.

14. The device includes a storage unit and an information processing unit, The memory unit stores sensory change information representing a time-series change in sensation when eating a target food product, The information processing unit determining a graphic and a color representing the sensation at each time point after eating the target food based on the sensation change information; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; the sensations include the smell and taste of the food of interest; The graphic representing the aroma of the target food is placed at the top of the image, and the graphic representing the taste of the target food is placed at the bottom of the image; determining a background color for the image based on the sensory characteristics of the target food; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; Food animation video generation server.

15. A server and a terminal are provided, The server determining a graphic and a color representing the sensation at each time point after eating the target food based on sensory change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; Sending the food animation video to the terminal; The terminal displays the food animation video received from the server; the sensations include the smell and taste of the food of interest; The graphic representing the aroma of the target food is placed at the top of the image, and the graphic representing the taste of the target food is placed at the bottom of the image; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; Food animation video provision system.

16. A server and a terminal are provided, The server determining a graphic and a color representing the sensation at each time point after eating the target food based on sensory change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; Sending the food animation video to the terminal; The terminal displays the food animation video received from the server; determining a background color for the image based on the sensory characteristics of the target food; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; Food animation video provision system.

17. A server and a terminal are provided, The server determining a graphic and a color representing the sensation at each time point after eating the target food based on sensory change information representing a time-series change in the sensation when eating the target food; determining an arrangement of the color and the graphic in an image including the determined graphic and color based on the sensation experienced when eating the target food; generating said image; repeatedly determining the shape and the color, determining the arrangement of the color and the shape, and generating the image for a predetermined time period after eating the target food, thereby generating a plurality of the images; By arranging the generated plurality of images in chronological order, a food animation video is generated that visually represents the time-series changes in sensation when eating the target food; Sending the food animation video to the terminal; The terminal displays the food animation video received from the server; the sensations include the smell and taste of the food of interest; The graphic representing the aroma of the target food is placed at the top of the image, and the graphic representing the taste of the target food is placed at the bottom of the image; determining a background color for the image based on the sensory characteristics of the target food; placing the graphic representing the short-lasting and / or spreading taste of the target food toward the top of the image, and placing the graphic representing the long-lasting and / or lingering taste of the target food toward the bottom of the image; Food animation video provision system.

Images