Stand, information processing method, and program

Abstract

To provide pleasure relating to food and drink.SOLUTION: A game device acquires a plurality of pieces of image information relating to a surface of a drink when another liquid is poured into the drink stored in a container, and executes processing relating to a game by using transition of the other liquid appearing on the surface of the drink contained in the plurality of pieces of image information. An electronic device acquires a plurality of pieces of image information relating to a surface of a drink when another liquid is poured into the drink stored in a container, and executes processing for outputting content from an output unit by using transition of the other liquid appearing on the surface of the drink contained in the plurality of pieces of image information.SELECTED DRAWING: Figure 44

Description

[Technical field]

[0001] The present invention relates to a game machine, an electronic device, an information processing method, and a program, and more particularly to a game machine, an electronic device, an information processing method, and a program that use information obtained about food and drink. [Background technology]

[0002] Various beverages such as coffee, black tea, and green tea are consumed all over the world. There are many devices that brew these beverages (extraction devices). For example, there is a coffee dripper for extracting coffee (see, for example, Patent Document 1). There are also various foods such as curry rice, ramen, and meat dishes all over the world. There are many devices that cook these foods (for example, pots and frying pans). There are also many devices that serve these foods (for example, plates and rice bowls). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-104155 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, various devices can be used to cook and serve food according to personal preferences. It is believed that the enjoyment of food will increase if some information can be provided to the person who prepares or eats the food using information that can be obtained at the time of cooking or serving the food.

[0005] Therefore, an object of the present invention is to provide enjoyment in food and drink. [Means for solving the problem]

[0006] One aspect of the present invention is an electronic device that acquires a plurality of pieces of image information relating to the surface of a drink stored in a container, and executes a process relating to a game by utilizing the transition of a natural phenomenon appearing on the surface of the drink contained in the plurality of pieces of image information. Also, the present invention is a processing method for controlling the electronic device, and a program for causing a computer to execute the method. Also, one aspect of the present invention is an electronic device that acquires information about a drink stored in a container using at least one of a temperature sensor capable of measuring temperature, an odor sensor capable of measuring odor, a sound sensor capable of measuring sound, a weight sensor capable of measuring weight, and a taste sensor capable of measuring taste, and executes a process related to a game using the information. Also, a processing method for controlling the electronic device and a program for causing a computer to execute the method. Also, one aspect of the present invention is an electronic device that acquires a plurality of pieces of information related to a drink stored in a container using at least one of a temperature sensor capable of measuring temperature, an odor sensor capable of measuring odor, a sound sensor capable of measuring sound, a weight sensor capable of measuring weight, and a taste sensor capable of measuring taste, and executes a process related to a game using transitions of the plurality of pieces of information. Also, the present invention is a processing method for controlling the electronic device and a program for causing a computer to execute the method. Another aspect of the present invention is an electronic device that executes a process related to a game by utilizing information about a drink stored in a container that is acquired at a plurality of predetermined timings when a user manually extracts the drink, and also a processing method for controlling the electronic device and a program for causing a computer to execute the method. Another aspect of the present invention is an electronic device that executes a process related to a game by utilizing the presence of a container newly placed at a predetermined location, the presence of a container lowered from the predetermined location, and the transition of food and drink in each container among a plurality of images of a container placed at a predetermined location and food and drink contained in the container, which are captured at a plurality of predetermined timings. Also, the present invention is a processing method for controlling the electronic device and a program for causing a computer to execute the method. Another aspect of the present invention is an electronic device that acquires a plurality of image information related to a container placed in a predetermined location and food and drink to be placed in the container, and executes a process related to a game using the presence of a container newly placed in the predetermined location, the presence of a container lowered from the predetermined location, and the transition of food and drink in each container among the containers and the food and drink included in the plurality of image information. Also, the present invention is a processing method for controlling the electronic device and a program for causing a computer to execute the method. One aspect of the present invention is an electronic device that controls output of content from an output unit by utilizing transitions of image information related to food and drink. Also, a processing method for performing the control and a program for causing a computer to execute the method. Another aspect of the present invention is an electronic device that uses a transition of image information about food and drink to control the execution of at least one process from among a plurality of processes different from the process about the evaluation and production of the food and drink, and a processing method for performing the control and a program for causing a computer to execute the method. In these embodiments, the electronic device may include an image acquisition unit that acquires the image information. In these embodiments, the image acquisition unit may acquire image information relating to the surface of the food or drink placed in a container, and the electronic device may perform the control based on a transition in the surface of the food or drink.

[0007] One aspect of the present invention is an information processing device that acquires image information about foods and beverages for each of the foods and beverages contained in a plurality of containers, and controls the provision of content determined based on the acquired image information to electronic devices of people associated with the plurality of containers. Also, the present invention is a processing method for performing the control, and a program for causing a computer to execute the method. Another aspect of the present invention is an information processing device that acquires image information about foods and beverages for each of the foods and beverages contained in a plurality of containers and controls electronic devices of each person associated with the plurality of containers to execute processing determined based on the acquired image information. Also, the present invention is a processing method for performing the control and a program for causing a computer to execute the method. In these embodiments, the information processing device may obtain judgment information for each of the foods and drinks based on image information relating to a surface of the foods and drinks, and perform the control using the judgment information.

[0008] One aspect of the present invention is an extractor having a dripper that drips liquid poured into a substance for extracting a beverage, a container that collects the liquid dripping from the dripper, and an electronic device that is disposed between the dripper and the container and that obtains information about the liquid collected in the container and uses the information to output content from an output unit. Also provided is a processing method for controlling the extractor and a program for causing a computer to execute the method.

[0009] In addition, in this form, the electronic device may have a hole for passing liquid dripped from the dripper, and may have a ring-shaped portion corresponding to the shape of the opening of the container, and when in use, may be sandwiched between the dripper and the container and supported by the opening of the container while supporting the dripper.

[0010] In addition, in this form, the electronic device may be provided with an image acquisition unit that acquires image information regarding the liquid stored in the container, and the image acquisition unit may be attached to the electronic device so that a lens faces the surface of the liquid stored in the container when the electronic device is placed in the container.

[0011] In this form, the image acquisition unit may acquire image information regarding the surface of the liquid stored in the container, and the electronic device may determine the content to be output from among the multiple contents based on a transition of at least one of the color, pattern, and bubbles appearing on the surface of the liquid. In this embodiment, the content may be content unrelated to both the drink and the extraction of the drink, and may be information relating to at least one of an anime character, an animal, a person, a plant, or a natural feature. In this embodiment, the electronic device may output at least one of audio information and image information as the content.

[0012] One aspect of the present invention is an annular electronic device that, in use, is sandwiched between a dripper that drips liquid poured into a substance for extracting a beverage and a container that collects the liquid dripping from the dripper, is supported by an opening of the container, and supports the dripper, obtains information about the liquid collected in the container, and uses the information to output content from an output unit. Also provided is a processing method for controlling the electronic device, and a program for causing a computer to execute the method.

[0013] One aspect of the present invention is an information processing device that acquires information about each of the generated drinks stored in a plurality of containers at any one of the following times: when a drink generated from a substance and a liquid for generating the drink is stored in the container, when the generated drink is poured into the container, and after the generated drink is poured into the container, and provides content determined from a plurality of contents based on an analysis of each of the acquired information to an electronic device of each person associated with the plurality of containers. Also provided is a determination method for making the determination, a content providing method for providing the determined content, and a program for causing a computer to execute each of these methods.

[0014] In addition, in this form, the information processing device may calculate judgment information for each drink based on information regarding a natural phenomenon occurring in relation to the drink, and use the judgment information to determine which content to provide to the electronic device from among the multiple contents.

[0015] For example, when a beverage generated from a substance and a liquid for generating a beverage is stored in a container, at least one of the timings of generating the beverage, putting the generated beverage into the container, and after putting the generated beverage into the container, an electronic device connected to (or in close proximity to) each of the multiple containers acquires information about the beverage stored in the container (information about a natural phenomenon occurring in relation to the beverage). Next, each electronic device uses a communication function to transmit the acquired information (information about a natural phenomenon occurring in relation to the beverage) to an information processing device. The information processing device acquires each piece of information (information about a natural phenomenon occurring in relation to the beverage) transmitted in this manner. Next, the information processing device obtains comparable judgment information for each beverage based on an analysis of each piece of acquired information. Next, the information processing device uses the judgment information to determine a content to be provided to each electronic device (an electronic device of each person associated with the multiple containers from which the information was obtained) from among the multiple contents. Next, the information processing device provides the determined content to the electronic device of each person associated with the multiple containers from which the information was obtained.

[0016] One aspect of the present invention is an extractor having a storage unit that stores a substance for extracting a beverage and has a hole through which a liquid poured onto the stored substance falls, a container that stores the liquid that falls from the storage unit, an acquisition unit that acquires information about the liquid stored in the container, and a control unit that controls output of content based on the information acquired by the acquisition unit. Also provided is a processing method for performing the control and a program for causing a computer to execute the method.

[0017] The acquisition unit may also be an image acquisition unit that acquires image information relating to the liquid stored in the container via a lens, and the acquisition unit may be attached so that the optical axis direction of the lens faces the surface of the liquid.

[0018] In addition, the container may be a transparent container, the acquisition unit may be an image acquisition unit that acquires image information regarding the liquid stored in the container via a lens, and the acquisition unit may be attached at a position where the image information regarding the liquid stored in the container can be acquired from outside the container.

[0019] Moreover, an electronic device including the acquisition unit and the control unit may be made removable from the extractor.

[0020] The control unit may also perform control to transmit information for outputting the content to another device capable of outputting the content.

[0021] The content may also be content unrelated to both the drink and the brewing of the drink, and may be information about at least one of cartoon characters, animals, people, plants, and geographical features.

[0022] The control unit may select at least one content to be output from among a plurality of contents based on the information acquired by the acquisition unit.

[0023] Also, one form of the present invention is an electronic device that is used by being sandwiched between a storage section that contains a substance for extracting a beverage and has a hole through which liquid poured onto the contained substance can fall, and a container that collects the liquid that falls from the storage section, and is characterized by having a sensor that acquires information about the liquid collected in the container.

[0024] In addition, the electronic device may have an annular portion having a shape corresponding to the shape of a flange portion provided in the storage portion and the shape of an opening of the container, and the electronic device may be installed so that the lower surface of the flange portion contacts the upper surface of the annular portion, the rim of the opening contacts the lower surface of the annular portion, and the sensor is positioned within the opening.

[0025] The annular portion may be provided with a grip portion. The annular portion may have a shape in which a part of the annular portion is missing, a U-shape, a Y-shape, a polygonal shape, a trapezoidal shape, an elliptical shape, or the like.

[0026] Another aspect of the present invention is an information processing device that determines content based on information acquired about a generated drink when a drink generated from a substance and a liquid for generating the drink is stored in a container, when the generated drink is poured into a container, or after the generated drink is poured into the container, and also a determination method for making the determination, a content providing method for providing the determined content, and a program for causing a computer to execute each of the methods.

[0027] Another aspect of the present invention is an information processing device that acquires information about each of the drinks stored in a plurality of containers when a drink generated from a substance and a liquid for generating the drink is stored in the container, when the generated drink is poured into the container, or after the generated drink is poured into the container, and determines content to be provided to each person associated with the plurality of containers based on each of the acquired information. Also provided is a determination method for making the determination, a content provision method for providing the determined content, and a program for causing a computer to execute each of these methods. Effect of the Invention

[0028] The present invention has the effect of providing enjoyment of food and drink. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a diagram showing an example of an external configuration of a coffee extractor 1 to which an electronic device 20 according to a first embodiment is attached. [Diagram 2] 1 is a diagram showing an example of an external configuration of a dripper 10, an electronic device 20, and a container 30 according to a first embodiment. [Diagram 3] 1 is a diagram showing an example of an external configuration of a dripper 10 and an electronic device 20 according to a first embodiment. [Figure 4] 2 is a block diagram showing an example of a functional configuration of an electronic device 20 according to a first embodiment. FIG. [Diagram 5] FIG. 2 is a diagram showing an example of the transition of an image (an image of the surface of coffee accumulated in a container 30) that can be acquired by the electronic device 20 in the first embodiment. [Figure 6] FIG. 4 is a diagram showing an example of various pieces of information used when determining information to be output by the electronic device 20 in the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of various pieces of information used when determining information to be output by the electronic device 20 in the first embodiment. [Figure 8] FIG. 7 is a diagram showing the relationship between an image (an image of the surface of coffee accumulated in a container 30) that can be acquired by the electronic device 20 in the first embodiment and areas A to I shown in FIG. 6(1). [Figure 9] FIG. 4 is a diagram showing a determination example in which information to be output is determined by the electronic device 20 in the first embodiment. [Figure 10] FIG. 2 is a simplified diagram showing an example in which voice information is output from electronic device 20 in the first embodiment. [Figure 11] 5 is a flowchart showing an example of a sound output process by the electronic device 20 in the first embodiment. [Figure 12] FIG. 4 is a diagram showing an example of various pieces of information used when determining information to be output by the electronic device 20 in the first embodiment. [Figure 13]FIG. 11 is a perspective view showing an example of the external configuration of a coffee extractor 200 to which an electronic device 220 according to a second embodiment is attached. [Figure 14] FIG. 13 is a block diagram showing an example of a functional configuration of an electronic device 220 according to a second embodiment. [Figure 15] FIG. 11 is a diagram showing an example of information to be output by electronic device 220 in the second embodiment. [Figure 16] FIG. 11 is a diagram showing an example of how information to be output is determined by the electronic device 220 according to the second embodiment. [Figure 17] FIG. 13 is a simplified diagram showing an example of a case where audio information and image information are output from electronic device 220 in the second embodiment. [Figure 18] 13 is a flowchart showing an example of information output processing by the electronic device 220 according to the second embodiment. [Figure 19] FIG. 13 is a block diagram showing an example of a functional configuration of an electronic device 320 according to a third embodiment. [Figure 20] FIG. 13 is a perspective view showing an example of the external configuration of external devices (electronic devices 350, 360) in the third embodiment. [Figure 21] FIG. 13 is a block diagram showing an example of a functional configuration of an electronic device 350 according to a third embodiment. [Figure 22] 13 is a flowchart showing an example of communication processing by the electronic device 320 and the electronic device 350 according to the third embodiment. [Diagram 23] FIG. 13 is a simplified diagram showing an example of a case where audio information and image information are output from devices (electronic devices 350, 360) other than the electronic device 220 in the third embodiment. [Figure 24] 13 is a flowchart showing an example of communication processing by the electronic device 320 and the electronic device 350 according to the fourth embodiment. [Diagram 25] FIG. 13 is a diagram illustrating an example of a system functional configuration of a communication system 400 according to a fifth embodiment. [Figure 26] FIG. 13 is a block diagram showing an example of a functional configuration of an electronic device 431 according to a fifth embodiment. [Figure 27] FIG. 13 is a block diagram showing an example of a functional configuration of a server 410 in the fifth embodiment. [Figure 28] FIG. 13 is a diagram showing an example of management information stored in a storage unit 413 in the fifth embodiment. [Figure 29] 13A and 13B are diagrams showing examples of stored information relating to electronic devices 431, 441, and 451 in the fifth embodiment, and examples of contents to be provided to electronic devices 431, 441, and 451. [Diagram 30] 13 is a flowchart showing an example of communication processing by an electronic device 431 and a server 410 in the fifth embodiment. [Diagram 31] 13 is a flowchart showing an example of communication processing by an electronic device 431 and a server 410 in the fifth embodiment. [Diagram 32] FIG. 13 is a diagram showing an example in which a server 410 in the fifth embodiment provides content to electronic devices 431, 442, 443, and 452. [Diagram 33] 13 is a flowchart showing an example of a transmission process by the electronic device 431 in the fifth embodiment. [Diagram 34] 13 is a flowchart showing an example of a content output process by the electronic device 431 in the fifth embodiment. [Diagram 35] 13 is a flowchart showing an example of a content providing process by a server 410 in the fifth embodiment. [Diagram 36] 13 is a flowchart showing an example of communication processing by a server 410 and an electronic device 431 in the fifth embodiment. [Figure 37] 1A to 1C are diagrams illustrating examples of mounting a sensor to a coffee extractor or electronic device in the first to fifth embodiments. [Figure 38] 1A to 1C are diagrams showing examples of sensors attached to electronic devices 700, 710, 720, and 725 according to the first to fifth embodiments. [Figure 39] FIG. 13 is a diagram showing the appearance of a modified example (coffee extractor 730) of the coffee extractor 1 in the first embodiment. [Diagram 40]FIG. 13 is a diagram showing the appearance of a modified example (coffee extractor 730) of the coffee extractor 1 in the first embodiment. [Diagram 41] FIG. 23 is a simplified diagram showing an example of a case where information acquired by electronic device 700 in the sixth embodiment is output. [Diagram 42] FIG. 23 is a diagram showing an example of the transition of an image (an image of the surface of coffee when milk is poured into coffee in a coffee cup) that can be acquired by the electronic device in the seventh embodiment. [Diagram 43] FIG. 23 is a diagram showing an example of various pieces of information used when determining information to be output by the electronic device in the seventh embodiment. [Diagram 44] FIG. 13 is a side view showing an example of the external configuration of installation device 830 for installing electronic device 810 in the seventh embodiment. [Diagram 45] FIG. 23 is a simplified diagram showing an example of a case where content is output from an electronic device 840 in the eighth embodiment. [Figure 46] FIG. 23 is a diagram illustrating a simplified configuration example of a communication system 40 according to a ninth embodiment. [Figure 47] FIG. 13 is a diagram showing an example of a correspondence relationship between provided information OI determined by electronic devices 900 and 901 to 904 using determination information in the ninth embodiment and provided information 921. [Figure 48] FIG. 13 is a diagram showing an example of an operation for additionally displaying a tool 75 on a character A displayed on a display unit 911 of a display device 910 using an electronic device 900 in a ninth embodiment. [Figure 49] FIG. 23 is a simplified diagram showing an example of a story game displayed on a display unit 950 of an electronic device 900 in a ninth embodiment. [Figure 50] FIG. 23 is a diagram showing an example of a correspondence relationship between provided information OI determined by the electronic device 900 using determination information in the ninth embodiment and provided information 941, 942, and 943. [Figure 51] FIG. 23 is a diagram showing an example of a display in which provided information is displayed on a display unit 950 of an electronic device 900 in a ninth embodiment. [Figure 52]FIG. 13 is a simplified diagram showing an example of transition of a raising game displayed on a display unit 950 of an electronic device 900 in a ninth embodiment. [Diagram 53] 13 is a diagram showing an example of a correspondence relationship between provided information OI determined by the electronic device 900 using determination information in the ninth embodiment and provided information 961, 962, and 963. FIG. [Figure 54] FIG. 23 is a simplified diagram showing an example of transition of a roaming game displayed on a display unit 950 of an electronic device 900 in a ninth embodiment. [Figure 55] 13 is a diagram showing an example of a correspondence relationship between provided information OI determined by the electronic device 900 using determination information in the ninth embodiment and provided information 971, 972, and 973. FIG. [Figure 56] 23 is a flowchart showing an example of a provision information determination process by the electronic device 900 according to the ninth embodiment. [Figure 57] FIG. 13 is a diagram showing an example of a transition of images (images of food and drink arranged on a table 1010) that can be acquired by the electronic device in the tenth embodiment. [Figure 58] FIG. 23 is a diagram showing an example of various types of information used when determining information to be output by the electronic device in the tenth embodiment. [Figure 59] FIG. 18 is a diagram showing the relationship between an image (an image of food and drink arranged on a table 1010) that can be acquired by the electronic device in the tenth embodiment and areas A to P shown in FIG. 58(1). [Figure 60] FIG. 23 is a diagram showing a determination example of information to be output by the electronic device according to the tenth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Next, an embodiment of the present invention will be described in detail with reference to the drawings.

[0031] [First embodiment: Example of providing audio information using colors and patterns formed when brewing coffee] [Examples of the appearance of coffee extractors and electronic devices] Fig. 1 is a diagram showing an example of the external configuration of a coffee extractor 1 to which an electronic device 20 according to a first embodiment is attached. Fig. 1(1) shows a perspective view of the coffee extractor 1 to which an electronic device 20 is attached, and Fig. 1(2) shows a side view of the coffee extractor 1 to which an electronic device 20 is attached.

[0032] 2 is a diagram showing an example of the external configuration of dripper 10, electronic device 20, and container 30 in the first embodiment. Fig. 2(1) shows a perspective view of coffee extractor 1 shown in Fig. 1 with electronic device 20 removed, and Fig. 2(2) shows a side view of dripper 10.

[0033] Fig. 3 is a diagram showing an example of the external configuration of dripper 10 and electronic device 20 in the first embodiment. Fig. 3(1) shows a top view of dripper 10 and electronic device 20 in a state where electronic device 20 is attached to the lower part of flange portion 12, and Fig. 3(2) shows a bottom view of dripper 10 and electronic device 20 in a state where electronic device 20 is attached to the lower part of flange portion 12. Fig. 3(3) shows a top view of electronic device 20, and Fig. 3(4) shows a bottom view of electronic device 20.

[0034] The coffee extractor 1 includes a dripper 10, an electronic device 20, and a container 30. The coffee extractor 1 can also be called a beverage extractor, a beverage straining device, a beverage preparation device, or the like.

[0035] Dripper 10 is a coffee dripper that has a container 11, a flange 12, and legs 13, and is used by being placed at an opening 32 of a container 30. Electronic device 20 is attached to the lower part of flange 12.

[0036] The storage unit 11 is a storage unit that stores ground coffee beans inside, and has a hole 14 (hole 14 that penetrates vertically) at the bottom for allowing hot water poured into the storage unit 11 to flow down. Thus, the storage unit 11 has an upper opening for pouring hot water onto the powder (ground coffee beans) from above, and a hole 14 (lower opening, extraction port) for dropping the hot water poured onto the powder (ground coffee beans) into the container 30. The storage unit 11 is also a device that pours the hot water poured onto the powder (ground coffee beans) stored inside, via the powder, into the container 30 located below the storage unit 11. Although not shown in FIG. 1, various members such as a handle may be provided on the outer surface of the storage unit 11.

[0037] For example, the storage section 11 can be formed in a funnel shape and can be a storage section that holds a paper filter that filters the liquid extracted (strained) from the coffee beans. The storage section 11 can also be a metal filter, or a storage section that holds a flannel filter. The shape of the storage section 11 can be various shapes such as a trapezoidal shape, a cone shape, a basket shape, etc.

[0038] The flange portion 12 is a disk-shaped flange portion that spreads outward in the horizontal direction from the lower portion of the storage portion 11. In addition, the flange portion 12 is provided at the lower portion of the storage portion 11, and stably supports the storage portion 11 when the storage portion 11 is placed in the opening 32 of the container 30.

[0039] Legs 13 are ring-shaped protrusions provided on the underside of flange 12, and support dripper 10 when dripper 10 is placed directly on a portion of container 30 other than opening 32. Legs 13 correspond to the bottom, thread base, and thread end of a ceramic ware (e.g., a teacup or a bowl). Although Figs. 2 and 3 show an example in which legs 13 are ring-shaped protrusions, legs 13 may be multiple protrusions with three or more points (e.g., multiple protrusions arranged on a circumference) so as to be able to support dripper 10.

[0040] The housing portion 11, the flange portion 12, and the leg portion 13 may be integrally formed from heat-resistant glass, ceramic, resin, etc., or the housing portion 11, the flange portion 12, and the leg portion 13 may be formed separately and then joined together. The leg portion 13 may be omitted.

[0041] The container 30 is a container (coffee server) that collects the coffee bean extract flowing down from the dripper 10. The container 30 has an opening 32 for receiving the coffee bean extract flowing down from the dripper 10, and supports the dripper 10 placed at the opening 32. The container 30 can be made of heat-resistant glass. The container 30 may be made of a material with high heat retention (for example, a stainless steel server), or may be made of ceramic. The shape of the container 30 can be various shapes other than the shape shown in FIG. 1. Although not shown in FIG. 1, various members such as a handle may be provided on the outer surface of the container 30.

[0042] Electronic device 20 is an electronic device or information processing device that is attached so as to be sandwiched between flange portion 12 of dripper 10 and opening 32 of container 30. That is, electronic device 20 is an electronic device or information processing device that is used by being sandwiched between flange portion 12 of dripper 10 and opening 32 of container 30. Also, for example, electronic device 20 is realized by incorporating each device (each part shown in FIG. 4) in a housing (case) made of heat-resistant glass, pottery, metal, ceramic, resin, wood, or the like.

[0043] Electronic device 20 is composed of a rectangular (flat) portion (grip portion 27) and a ring-shaped (annular) portion (annular portion 28). Annular portion 28 is ring-shaped (annular) when viewed from above, and has an upper surface (upper portion) 71 on which dripper 10 is placed, a lower surface (lower portion) 72 placed on opening 32 of container 30, an inner surface (inner wall) 73 facing leg portion 13 on the lower surface of flange portion 12, and an outer surface (outer wall, peripheral wall) 74. Thus, annular portion 28 is doughnut-shaped with a hollow portion (hole 29), and is disposed so as to be sandwiched between flange portion 12 of dripper 10 and opening 32 of container 30 such that upper surface (upper portion) 71 contacts dripper 10 and lower surface (lower portion) 72 contacts the rim of opening 32 of container 30. Electronic device 20 has a shape in which a square portion (grip 27) is joined to a ring-shaped (annular) portion (annular portion 28), and has a shape resembling, for example, a magnifying glass or a goldfish scoop. Annular portion 28 has an inner diameter (diameter of hole 29) larger than the diameter (outer diameter) of leg portion 13 on the lower surface of flange portion 12 of dripper 10. Annular portion 28 has an outer diameter smaller than the diameter of the lower surface of flange portion 12 of dripper 10.

[0044] 3(2), electronic device 20 is attached so as to be sandwiched between flange portion 12 of dripper 10 and opening 32 of container 30 such that leg portion 13 of flange portion 12 enters hole 29 of annular portion 28. That is, electronic device 20 is supported by opening 32 of container 30, and dripper 10 is supported by annular portion 28 of electronic device 20. For example, as shown in FIG. 1(2) and FIG. 3(2), by setting the shape of annular portion 28 of electronic device 20 to match the shape of opening 32 (rim) of container 30 and to match the shape of flange portion 12 of dripper 10, electronic device 20 and dripper 10 are stably supported by container 30 even when electronic device 20 is attached so as to be sandwiched between flange portion 12 of dripper 10 and opening 32 of container 30.

[0045] In this way, the annular portion 28 has a shape corresponding to the annular shape of the flange portion 12 and the annular shape of the opening 32 of the container 30. Furthermore, the electronic device 20 is installed such that the lower surface of the flange portion 12 contacts the upper surface 71 of the annular portion 28, the rim of the opening 32 of the container 30 contacts the lower surface 72 of the annular portion 28, and the lens 21 is disposed within the opening 32.

[0046] Furthermore, when performing installation or removal of electronic device 20, the user can perform each operation while holding grip portion 27 in his / her hand. Therefore, grip portion 27 functions as a grip portion of electronic device 20. Furthermore, grip portion 27 includes operation portion 26, and therefore grip portion 27 also functions as an operation portion of electronic device 20. Furthermore, grip portion 27 includes audio output portion 25, and therefore grip portion 27 also functions as an output portion of electronic device 20.

[0047] In this way, by making the electronic device 20 shaped like a magnifying glass or a goldfish scoop, it is easy to attach and detach the electronic device 20. Furthermore, the electronic device 20 can be stably installed in the coffee extractor 1.

[0048] The shapes of gripping portion 27 and annular portion 28 are not limited to these. For example, the outer diameter of annular portion 28 may be larger than the diameter of the lower surface of flange portion 12 of dripper 10. Also, as long as annular portion 28 can be stably placed in opening 32 of container 30 and can stably support dripper 10, annular portion 28 may have another shape. For example, an annular portion may have a shape in which a part of the ring-shaped portion is missing, or may have a U-shape or Y-shape in top view, or may have a polygonal, trapezoidal, elliptical, or other shape (having a portion equivalent to a hole having a size larger than at least the diameter (outer diameter) of leg portion 13). A modified example of annular portion 28 is shown in FIG. 38.

[0049] Moreover, the electronic device 20 has a lens 21, an audio output unit 25, and an operation unit 26. The functional configurations of the lens 21, the audio output unit 25, and the operation unit 26 will be described in detail with reference to FIG.

[0050] The lens 21 of the electronic device 20 is disposed on the lower part (the lower surface (lower surface 72)) of the annular portion 28. The lens 21 of the electronic device 20 is disposed at a position such that the optical axis direction of the lens 21 faces the surface of the coffee extract accumulated in the container 30 when the dripper 10 and the electronic device 20 are set in the container 30.

[0051] In addition, when hot water is poured into the dripper 10 or when the electronic device 20 is removed, water or the like may adhere to the entire electronic device 20 or each part (lens 21, audio output unit 25, operation unit 26). Therefore, it is preferable to apply a waterproofing treatment to the entire electronic device 20 or each part (lens 21, audio output unit 25, operation unit 26).

[0052] In addition, when hot water is poured into the dripper 10, it is assumed that the dripper 10 and the container 30 will heat up. Therefore, it is preferable that each device built into the electronic device 20 is enclosed in a heat insulating material or is subjected to a heat insulating treatment so as not to be affected by the heat from the dripper 10 and the container 30. For example, a foam-based, felt-based, or other heat insulating material may be used, or two or more of these materials may be combined. In addition, for example, each device (each part shown in FIG. 4) other than the lens 21 and the image acquisition unit 22 may be placed inside the gripping unit 27 so as not to be affected by the heat from the dripper 10 and the container 30.

[0053] In addition, when hot water is poured into dripper 10, steam is generated from the coffee (liquid) stored in container 30, and it is expected that lens 21 will fog up. Therefore, it is preferable to apply an anti-fogging treatment to the surface of lens 21. Alternatively, lens 21 may be entirely covered with a transparent member that has been anti-fogging treated. In addition, to prevent the effects of heat, it is preferable to apply a heat insulating treatment to lens 21 itself or the transparent member that has been anti-fogging treated.

[0054] [Example of electronic device functional configuration] FIG. 4 is a block diagram showing an example of a functional configuration of the electronic device 20 according to the first embodiment.

[0055] The electronic device 20 has a lens 21, an image acquisition unit 22, a control unit 23, a storage unit 24, an audio output unit 25, and an operation unit 26. The electronic device 20 may have a built-in or attached battery and use the battery as a power source, or may not have a built-in battery and use an external power source.

[0056] The lens 21 is composed of a plurality of lenses that collect light from a subject. The light from the subject collected by the lens 21 is incident on an imaging element (included in the image acquisition unit 22).

[0057] The image acquisition unit 22 captures an image of a subject and generates an image (image data) based on the control of the control unit 23, and outputs the generated image to the control unit 23. The image acquisition unit 22 is composed of, for example, an imaging element (image sensor) into which light from the subject collected by the lens 21 is incident, and an image processing unit that performs predetermined image processing on the image data generated by the imaging element. For example, a CCD (Charge Coupled Device) type or CMOS (Complementary Metal Oxide Semiconductor) type imaging element can be used as the imaging element. Note that the lens 21 may have each part (for example, a zoom lens, a focus lens, and an aperture) controlled based on the control of the control unit 23.

[0058] The control unit 23 controls each part of the electronic device 20 based on a control program stored in the storage unit 24. The control unit 23 is realized by, for example, a CPU (Central Processing Unit). The CPU is a central processing unit (electronic device) consisting of a control device (control circuit) and an arithmetic device (arithmetic circuit).

[0059] The storage unit 24 is a memory that stores various information. For example, the storage unit 24 stores various information required for the electronic device 20 to perform various processes (for example, a control program, information for determining information to be output (for example, each piece of information shown in Figs. 6(1)(2) and Fig. 7)). The storage unit 24 also stores information determined based on an image (image data) acquired by the image acquisition unit 22 (for example, information shown in Fig. 9(1)).

[0060] The audio output unit 25 is an audio output unit (for example, a speaker) that outputs various sounds based on the control of the control unit 23.

[0061] The operation unit 26 receives an operation input for operating the electronic device 20. In the first embodiment, for ease of explanation, the operation unit 26 is a switch for turning on the power supply of the electronic device 20. Although only a simplified operation unit 26 is shown in Figs. 1 to 3, other operation members may be provided.

[0062] [Example of the transition of the surface of coffee accumulating in a container] Fig. 5 is a diagram showing an example of the transition of images (images of the surface of coffee accumulated in container 30) that can be acquired by electronic device 20 in the first embodiment. Note that in Fig. 5, for ease of explanation, the liquid coffee and bubbles generated on the surface of the coffee are simplified and shown as circles, etc.

[0063] FIG. 5(1) shows a state where the extracted coffee (liquid) 41 has fallen to the bottom 31 of the container 30. First, the user sets the dripper 10 on the top of the container 30, attaches a paper filter inside the housing portion 11 of the dripper 10, and puts ground coffee beans into the paper filter. Next, the user pours a small amount of hot water that has been boiled and then cooled to about 90 degrees onto the ground coffee beans set in the dripper 10, and steams for about 20 to 30 seconds. In this steaming state, only a small amount of hot water is poured onto the ground coffee beans, and the ground coffee beans absorb the poured hot water, so that the extracted coffee (liquid) 41 often does not fall to the bottom 31 of the container 30.

[0064] After the steaming period is over, the user pours hot water into the steamed ground coffee beans (ground coffee beans set in dripper 10) in the desired amount at the user's desired timing. When hot water is poured onto the ground coffee beans in this way, the ground coffee beans are unable to absorb all the hot water, and coffee (liquid) 41 falls to the bottom 31 of container 30, as shown in FIG. 5(1). In this case, the position (position at bottom 31), size (size at bottom 31), and shape (planar shape at bottom 31) of coffee (liquid) 41 that falls to the bottom 31 of container 30 change depending on the amount of hot water poured onto the ground coffee beans and the position at which it is poured (the position of the ground coffee beans set in dripper 10). For example, as shown in FIGS. 5(1) and 5(2), coffee (liquid) 41 first falls near the center of the bottom 31 of the container 30, and then the coffee (liquid) 41 spreads and accumulates in either direction.

[0065] Furthermore, as shown in Fig. 5(3), when a certain amount of extracted coffee (liquid) 41 accumulates at the bottom 31 of the container 30, bubbles 42 often appear on the surface of the coffee (liquid) 41 accumulated in the container 30. Note that in Fig. 5, the bubbles 42 are simplified and shown in white. Furthermore, when the user continues to pour hot water onto the coffee bean powder until the desired amount is reached, the surface of the coffee (liquid) 41 becomes smaller in size to fit the shape of the container 30, as shown in Figs. 5(3) to (8).

[0066] In this way, when hot water is poured onto the ground coffee beans set in the dripper 10, various colors and patterns, such as the color of the coffee (a dark color) and the color of the foam (a whitish color), are formed on the bottom 31 of the container 30 (including the surface of the coffee (liquid) 41 accumulated in the container 30), as shown in Figs. 5(1) to (8). The colors and patterns thus formed often differ depending on the person brewing the coffee and the environment in which the coffee is brewed. It is also assumed that the colors and patterns thus formed will rarely be the same or approximately the same even if the same person brews coffee in the same environment. Therefore, in the first embodiment, an example is shown in which various information is provided using the colors and patterns formed when brewing coffee.

[0067] [Example of judgment information for determining output information using the colors and patterns formed when brewing coffee] 6 and 7 are diagrams showing an example of various pieces of information used when the electronic device 20 in the first embodiment determines information to be output.

[0068] FIG. 6(1) shows an example of regions A to I used when acquiring information about coffee accumulated in the container 30 (for example, the surface of coffee (liquid) 41 accumulated in the container 30). The circular region shown in FIG. 6(1) is a region corresponding to the circular shape of the bottom 31 of the container 30 in an image acquired by the image acquisition unit 22 of the electronic device 20 (for example, a rectangular image acquired by an imaging element). The circular region shown in FIG. 6(1) is divided into nine regions to be regions A to I. For example, the regions A to I can be set so that each area is approximately the same. Note that the division of the circular region shown in FIG. 6(1) is one example, and the circular region may be divided into a plurality of regions by other division methods. For example, the circular region may be divided into four regions by two straight lines (same lines as the diameter) passing through the center of the circle, the circular region may be divided into six regions by three straight lines (same lines as the diameter) passing through the center of the circle, or the circular region may be divided into eight regions by four straight lines (same lines as the diameter) passing through the center of the circle. Alternatively, multiple regions (closed regions) of irregular shapes (e.g., polygons (e.g., triangles, squares, pentagons), circles, ellipses) can be placed so as not to overlap with a circular region, and multiple regions can be set using each of these regions (closed regions, regions outside the closed regions).

[0069] FIG. 6(2) shows an example of determination information (table) used when determining information to be output based on information acquired using areas A to I shown in FIG. 6(1). The determination information shown in FIG. 6(2) is information showing the correspondence between the transition of information about coffee accumulated in the container 30 (for example, the surface of coffee (liquid) 41 accumulated in the container 30) and information OI to be output. For example, the determination information shown in FIG. 6(2) defines all combinations (patterns) of A to I. For example, a pattern "A, A, ..., A, A" where all are A is defined for information OI1 to be output. Furthermore, a pattern "A, B, A, ..., A, A" where all are A except for one (time t2 is "B") is defined for information OI2 to be output. Furthermore, a pattern "A, B, B, A, ..., A, A" where all are A except for two (times t2 and t3 are "B") is defined for information OI2 to be output. The rest are defined in the same way, and the pattern of I, “I, I, ..., I, I”, is defined for all the information OIM to be output.

[0070] In FIG. 6(2), times t1, ..., tn (where n is a positive integer) are values ​​indicating the elapsed time when coffee is extracted using the coffee extractor 1. For example, if the interval for acquiring information about coffee accumulated in the container 30 is 20 seconds, the intervals are 20 seconds, such as time t1=20 seconds, time t2=40 seconds, time t3=60 seconds, .... Then, the control unit 23 (shown in FIG. 4) sequentially determines the area with the highest proportion of black among the areas A to I shown in FIG. 6(1) at each of times t1, ..., tn, and determines information OI (OI1 to OIM (where M is a positive integer)) to be output based on the determination result. For example, if the area with the highest proportion of black among the areas A to I shown in FIG. 6(1) at each of times t1, ..., tn is all A (area A shown in FIG. 6(1)), the control unit 23 determines OI1 as information OI to be output. Also, for example, if, at each of times t1, ..., tn, among areas A to I shown in Figure 6 (1), the area with the highest proportion of black is all I (area I shown in Figure 6 (1)), the control unit 23 determines OIM as the information OI to be output.

[0071] The interval between the times t1, ..., tn and the value of n may be set in advance, or may be set by the user's operation to a value of the user's preference. The value of M is determined based on the number of regions into which the circle corresponding to the surface of the coffee (liquid) 41 accumulated in the container 30 is divided, and the value of n. For example, if the circle is divided into 4 regions and the value of n is 4, the value of M is determined to be 256 (=4 x 4 x 4 x 4). However, depending on the number of regions and the value of n, it is assumed that the value of M becomes too large and it is not possible to prepare information to be output (for example, the voice information 61 shown in FIG. 7). For this reason, if the value of M is greater than the number of pieces of information to be output (for example, the voice information 61 shown in FIG. 7), the same voice information 61 may be assigned to different pieces of information (information OI to be output).

[0072] FIG. 7 shows an example of the correspondence between the information OI to be output, determined using the determination information shown in FIG. 6(2), and the information to be output (audio information 61). For example, when OI1 is determined as the information OI to be output, information about Santa is output as the audio information 61. The information about Santa is information for outputting the audio, for example, "Today, let's dream of Santa Claus of your dreams." Note that, for ease of explanation, FIG. 7 simply shows only the characters to be output as audio. Note that these output examples will be explained with reference to FIG. 10.

[0073] [An example of determining the information to be output using the colors and patterns formed when brewing coffee] Fig. 8 is a diagram showing the relationship between an image that can be acquired by the electronic device 20 in the first embodiment (an image of the surface of the coffee accumulated in the container 30) and the areas A to I shown in Fig. 6(1). Note that the images of the surface of the coffee accumulated in the container 30 shown in Figs. 8(1) to (8) are the same as the images shown in Figs. 5(1) to (8).

[0074] Fig. 9 is a diagram showing an example of determining information to be output by the electronic device 20 in the first embodiment. Figs. 8 and 9 show an example in which times t1 to t8 are used as the elapsed time when coffee is extracted using the coffee extractor 1. In this case, the value of M is 43046721, and it is assumed that this number of pieces of voice information 61 cannot be prepared (and cannot be stored in the storage unit 24). In such a case, it is possible to associate some of the voice information 61 with the same output target information OI in a redundant manner.

[0075] The images of the surface of the coffee accumulated in the container 30 shown in Figs. 8(1) to (8) are taken at times t1 to t8.

[0076] FIG. 9(1) shows an example of storing the area with the highest black ratio among areas A to I shown in FIG. 6(1) at times t1 to t8 in association with times t1 to t8.

[0077] First, the power supply of the electronic device 20 is turned on. This power supply on / off can be performed by a user manually operating a switch (operation unit 26) for turning the power supply on / off provided in the electronic device 20. Alternatively, the electronic device 20 may be provided with some kind of sensor (e.g., a sound sensor) so that the power supply is automatically turned on / off by the sensor. Alternatively, the electronic device 20 may be provided with a wireless communication unit so that the power supply is turned on / off based on the operation of an external device (e.g., a smartphone).

[0078] In this way, after the power of the electronic device 20 is turned on, the image acquisition unit 22 (shown in FIG. 4) of the electronic device 20 sequentially acquires images (including image data) of the coffee accumulating in the container 30 based on the control unit 23 (shown in FIG. 4) of the electronic device 20.

[0079] In addition, the control unit 23 (shown in FIG. 4) of the electronic device 20 sequentially acquires images (images (including image data) of the coffee accumulating in the container 30) acquired by the image acquisition unit 22 (shown in FIG. 4) of the electronic device 20.

[0080] Here, the information to be initially determined after the power supply of the electronic device 20 is turned on (the image of the surface of the coffee accumulated in the container 30) may be determined based on the passage of time, based on a change in the image, or based on an external operation. For example, the time t1 may be the timing when a predetermined time (e.g., 1 to 20 seconds, the steaming time) has elapsed since a change occurred in the image of the surface of the coffee accumulated in the container 30 after the power supply of the electronic device 20 is turned on. Note that the change in the image of the surface of the coffee accumulated in the container 30 is, for example, a change caused by the coffee liquid first dropping to the bottom 31 of the container 30 (in other words, a change caused by the appearance of the color (black) of the coffee liquid).

[0081] In this way, the control unit 23 sets the image acquired at time t1 (an image of the surface of the coffee accumulated in the container 30) as the first determination target. Then, the control unit 23 determines a circular area in the image acquired at time t1 (a circular area corresponding to the surface of the coffee (liquid) 41 accumulated in the container 30). Note that the circular area in the image acquired at time t1 is often the same (or approximately the same) as the size of the bottom 31 of the container 30, since there is often very little coffee accumulated in the container 30.

[0082] Here, the circular region in the image acquired by control unit 23 becomes smaller as time passes. That is, as the amount of coffee (liquid) 41 accumulating in container 30 increases, the surface of coffee (liquid) 41 becomes higher, and so the area (circular area) of the surface of coffee (liquid) 41 becomes smaller in accordance with the shape of container 30. For this reason, as shown in Figs. 8(1) to (8), control unit 23 changes the size of areas A to I in accordance with the determined circular area (the circular area corresponding to the surface of coffee (liquid) 41 accumulated in container 30). Note that the sizes of areas A to I may be left unchanged and the determination process may be performed with them fixed.

[0083] Next, the control unit 23 acquires a feature amount for an image included in a circular region (a circular region corresponding to the surface of the coffee (liquid) 41 accumulated in the container 30) in the image acquired at the timing of time t1. Specifically, the control unit 23 acquires a feature amount for each image included in the circular region in the image acquired at the timing of time t1 (each image included in the regions A to I shown in FIG. 6(1)). For example, the control unit 23 obtains information on the color of coffee as a feature amount for each image included in the regions A to I. For example, the control unit 23 determines each pixel in each image included in the image acquired by the image acquisition unit 22 to be either a black region or a white region. For example, this can be determined by a binarization process (for example, a process in which a threshold value is set in advance, and if the pixel value is greater than the threshold value, it is determined to be white, and if the pixel value is smaller than the threshold value, it is determined to be black). Note that it is preferable to set the threshold value used in the binarization process to a value that can distinguish between coffee and foam. Then, the control unit 23 obtains the ratio of the black region and the white region in each image included in the regions A to I as a feature amount.

[0084] Next, control unit 23 determines the area with the highest proportion of black areas from among the images contained in areas A to I. In the example shown in Fig. 8(1), the area with the highest proportion of black areas from among the images contained in areas A to I is area A. Therefore, control unit 23 determines area A as the area with the highest proportion of black areas from the images acquired at time t1.

[0085] Thereafter, control unit 23 similarly determines the area with the highest proportion of black areas for each image (images of the surface of the coffee accumulated in container 30) acquired at times t2 to t8. Control unit 23 then sequentially stores the feature amounts (areas with the highest proportion of black areas) determined for each image acquired at times t1 to t8 in memory unit 24 in association with times t1 to t8. An example of this storage is shown in FIG. 9(1). That is, the example of storage shown in FIG. 9(1) is an example of storage for the examples shown in FIGS. 8(1) to (8).

[0086] Fig. 9(2) shows an example of extraction when extracting a feature amount that matches the feature amount shown in Fig. 9(1) (information associating times t1 to t8 with areas A to I) from the determination information shown in Fig. 6(2). Note that the determination information shown in Fig. 9(2) is the same as the determination information shown in Fig. 6(2), and a part of the determination information shown in Fig. 6(2) is omitted.

[0087] Next, the control unit 23 determines information to be output from the audio output unit 25 based on the acquired feature amount. Specifically, the control unit 23 extracts a feature amount that matches the acquired feature amount (information that associates times t1 to t8 with areas A to I shown in FIG. 9(1)) from the determination information shown in FIG. 6(2). For example, as shown in FIG. 9(2), the feature amount shown in FIG. 9(1) matches the feature amount shown in the dashed rectangle 51. Therefore, the control unit 23 extracts the feature amount shown in the dashed rectangle 51 from the determination information shown in FIG. 9(2) as the feature amount that matches the feature amount shown in FIG. 9(1). In addition, the control unit 23 determines OIX1 as information OI (information to be output) corresponding to the extracted feature amount.

[0088] Next, the control unit 23 causes the audio output unit 25 to output audio based on the audio information 61 stored in the storage unit 24 in association with OIX1 determined as the information OI to be output. For example, as shown in Fig. 9(3), based on the audio information 61 (shown in the dashed rectangle 52) associated with OIX1, audio "Today, let's wait patiently like a snowman" is output from the audio output unit 25. An example of this output is shown in Fig. 10.

[0089] [Example of outputting voice information from an electronic device when brewing coffee] Fig. 10 is a simplified diagram showing an example of a case where audio information is output from electronic device 20 in the first embodiment. Note that the example shown in Fig. 10 shows an example of outputting audio information determined by the examples shown in Figs. 8 and 9.

[0090] As described above, the control unit 23 outputs the voice "Today, let's wait patiently like a snowman" from the voice output unit 25 based on the voice information (within the dashed rectangle 52 shown in FIG. 9(3)) associated with the OIX1. This allows the user P1, who is brewing coffee using the coffee extractor 1, to hear the voice based on the feature amount acquired by the extraction after brewing his / her ideal coffee. For example, whether the user P1 is a coffee brewing expert or an amateur, it is considered that the color and pattern appearing on the surface of the coffee accumulated in the container 30 will often be different each time. Therefore, when coffee is extracted using such coffee characteristics, the user can enjoy drinking the extracted coffee as well as voice information (information unrelated to the coffee or its extraction) based on the feature amount acquired during the extraction.

[0091] [Example of outputting audio information] Fig. 11 is a flowchart showing an example of a sound output process by the electronic device 20 in the first embodiment. Fig. 11 shows an example of an operation after the power of the electronic device 20 is turned on by operating the operation unit 26 and then a change occurs in the image of the surface of the coffee accumulated in the container 30.

[0092] First, the image acquisition unit 22 (shown in FIG. 4) of the electronic device 20 sequentially acquires images (including image data) of the coffee accumulating in the container 30 based on the control unit 23 (shown in FIG. 4) of the electronic device 20 (step S101). The control unit 23 also sequentially acquires images (images (including image data) of the coffee accumulating in the container 30) acquired by the image acquisition unit 22 (step S101).

[0093] Next, the control unit 23 analyzes the acquired image information to acquire features (step S102). For example, the control unit 23 determines a circular region (a circular region corresponding to the surface of the coffee (liquid) 41 accumulated in the container 30) in the acquired image. Next, the control unit 23 acquires features for the images included in the determined circular region. Specifically, the control unit 23 acquires features (proportions of black and white regions) for each image (each image included in regions A to I shown in FIG. 6(1)) included in the circular region in the acquired image. Next, the control unit 23 determines the region with the highest proportion of black regions from among the images included in regions A to I.

[0094] Next, the control unit 23 stores the acquired feature amount in the storage unit 24 in association with the elapsed time (step S103). For example, as shown in Fig. 9(1), the control unit 23 stores the feature amount (area with the highest proportion of black areas) determined for each image acquired at the timing of time t1 to t8 in the storage unit 24 in association with the time t1 to t8.

[0095] Next, the control unit 23 judges whether or not the end condition is satisfied (step S104). If the end condition is not satisfied, the process returns to step S101, and if the end condition is satisfied, the process proceeds to step S105.

[0096] Here, as a criterion for judging whether the end condition is satisfied, for example, whether a predetermined time has passed or whether no change (or almost no change) occurs in the acquired image information can be used. The predetermined time can be judged based on, for example, the time that has passed since the power supply of the electronic device 20 was turned on or the time that has passed since the image of the surface of the coffee accumulated in the container 30 changed after the power supply of the electronic device 20 was turned on. The predetermined time may be set in advance or may be set by a user operation according to the user's preference. For example, as in the example shown in FIG. 8 and FIG. 9, when image information is acquired at the timing of times t1 to t8, the predetermined time is time t8 (or after time t8 has passed). The fact that no change occurs in the acquired image information can be judged based on, for example, the fact that the difference between consecutive images in a time series is equal to or less than a predetermined value continues for a predetermined period of time by analyzing the acquired image information and comparing the images. In other words, the fact that no change occurs in the acquired image information can be judged based on whether the time during which no change occurs in the image of the surface of the coffee accumulated in the container 30 has passed for a predetermined time. It is also possible that no change occurs due to the lens 21 becoming cloudy.

[0097] When the termination condition is satisfied (step S104), the control unit 23 determines information to be output from the audio output unit 25 based on the acquired feature amount (step S105). For example, the control unit 23 extracts a feature amount that matches the acquired feature amount (for example, information associating times t1 to t8 with areas A to I shown in FIG. 9(1)) from the determination information shown in FIG. 6(2). Next, the control unit 23 determines information (information to be output) OI corresponding to the extracted feature amount, and determines audio information 61 (shown in FIG. 7) stored in the storage unit 24 in association with the determined information (information to be output) OI.

[0098] Next, the control unit 23 causes the audio output unit 25 to output audio based on the determined audio information (step S106). For example, audio is output from the audio output unit 25 as shown in FIG.

[0099] [Example of using other judgment information] Although an example of determining the audio information to be output using the determination information shown in Fig. 6 has been described above, the audio information to be output may be determined using other determination information. Therefore, an example of using other determination information will be described here.

[0100] FIG. 12 is a diagram showing an example of various pieces of information used when the electronic device 20 in the first embodiment determines information to be output.

[0101] FIG. 12(1) shows an example of an image pattern relating to the surface of coffee accumulating in the container 30. As shown in FIG. 12(1), an image pattern that is expected to appear on the surface of coffee accumulating in the container 30 is stored in the memory unit 24 as determination information. Note that in FIG. 12(1), the image patterns are shown in a simplified form for ease of explanation. Also, the number N of image patterns (where N is a positive integer) may be increased or decreased according to the storage capacity of the memory unit 24.

[0102] Fig. 12(2) shows an example of determination information (table) used when determining information to be output based on information acquired using the determination information (image patterns PA1-PAN) shown in Fig. 12(1). The determination information shown in Fig. 12(2) is information showing the correspondence between the transition of information about coffee accumulated in the container 30 (for example, the surface of coffee (liquid) 41 accumulated in the container 30) and information OI to be output. Note that the determination information shown in Fig. 12(2) corresponds to the determination information shown in Fig. 6(2), and since the information common to Fig. 6(2) is similar to Fig. 6(2), a description thereof will be omitted here.

[0103] For example, the control unit 23 (shown in FIG. 4) determines a circular region (a circular region corresponding to the surface of the coffee (liquid) 41 accumulated in the container 30) in the image acquired from the image acquisition unit 22 (shown in FIG. 4). Next, the control unit 23 compares the image included in the determined circular region with each of the image patterns PA1 to PAN in the determination information shown in FIG. 12(1), and extracts the image pattern with the highest degree of similarity as a feature.

[0104] For example, each pixel of each of the image patterns PA1-PAN represents two values ​​(black, white). The control unit 23 also determines each pixel in the image included in the determined circular area as either a black area or a white area (for example, binarization process). Next, the control unit 23 compares corresponding pixels (pixels at the same position in the circular area) between each pixel (black, white) in the image included in the determined circular area and each pixel (black, white) in each of the image patterns PA1-PAN, and calculates the number of matching pixels for each of the image patterns PA1-PAN. Next, the control unit 23 extracts the image pattern PA1-PAN with the largest number of matching pixels as the image pattern with the highest degree of similarity.

[0105] Next, the control unit 23 stores the extracted image patterns PA1-PAN in association with the elapsed time in the storage unit 24. For example, the control unit 23 stores the feature amounts (image patterns PA1-PAN) determined for the images acquired at the timings of times t1-t8 in association with the times t1-t8 in the storage unit 24.

[0106] Next, the control unit 23 determines information to be output from the audio output unit 25 based on the acquired feature amount. For example, the control unit 23 extracts a feature amount that matches the acquired feature amount (for example, information associating times t1 to t8 with image patterns PA1 to PAN) from the determination information (table) shown in Fig. 12(2). Next, the control unit 23 determines information (information to be output) OI corresponding to the extracted feature amount, and determines audio information stored in the storage unit 24 in association with the determined information (information OI to be output).

[0107] [Example of using drip speed as judgment information] For example, the transition in size of a circular area corresponding to the surface of coffee (liquid) 41 accumulated in the container 30 can be used as the determination information. In other words, the drip speed can be used as the determination information.

[0108] For example, a plurality of sizes of a circular area corresponding to the surface of coffee (liquid) 41 accumulated in container 30 are stored in storage unit 24 as judgment information. Then, control unit 23 (shown in FIG. 4) compares the size of the circular area (the circular area corresponding to the surface of coffee (liquid) 41 accumulated in container 30) in the image acquired from image acquisition unit 22 (shown in FIG. 4) with the plurality of sizes stored in storage unit 24, and extracts the size with the highest degree of similarity as a feature. Next, control unit 23 stores the extracted size in storage unit 24 in association with the elapsed time. Next, control unit 23 determines information to be output from audio output unit 25 based on the acquired feature. For example, control unit 23 extracts a feature that matches the acquired feature (for example, information associating times t1 to t8 with size) from judgment information corresponding to FIG. 6(2) and FIG. 12(2). Next, the control unit 23 determines information (information to be output) OI corresponding to the extracted features, and determines the audio information stored in the memory unit 24 in association with the determined information (information OI to be output).

[0109] Also, the height of coffee (liquid) 41 accumulated in container 30 may be used as the determination information. In this case, a sensor for acquiring the height of coffee (liquid) 41 accumulated in container 30 (for example, an ultrasonic distance sensor, an image sensor for acquiring an image of the side of transparent container 30) is provided in electronic device 20 or coffee extractor 1, and the height of coffee (liquid) 41 accumulated in container 30 is acquired using this sensor.

[0110] [Example of using ripples and the direction of water droplets as information for judgment] For example, various information such as ripples of water droplets (coffee) falling on the surface of coffee (liquid) 41 accumulated in container 30, the direction (direction moving on the surface) of the water droplets (coffee) that have fallen on the surface of coffee (liquid) 41 accumulated in container 30, the length of their movement, and the amount of bubbles generated by the water droplets (coffee) that have fallen on the surface of coffee (liquid) 41 accumulated in container 30 can be used as the determination information. In other words, natural phenomena that occur during dripping can be used as the determination information.

[0111] For example, when detecting ripples of water droplets (coffee) falling on the surface of coffee (liquid) 41 stored in container 30, one or more ripple image patterns for identifying ripples are stored in storage unit 24. Then, control unit 23 (shown in FIG. 4) detects the occurrence of ripples when an image matching the ripple image pattern is detected from an image of a circular region (a circular region corresponding to the surface of coffee (liquid) 41 stored in container 30) in an image acquired from image acquisition unit 22 (shown in FIG. 4). In this case, control unit 23 can store the type of ripple (identifiable by the ripple image pattern), the position where the ripple occurred (a position in the circular region corresponding to the surface of coffee (liquid) 41 stored in container 30), the timing (elapsed time) when the ripple occurred, and the like, in storage unit 24 in association with the elapsed time. Next, control unit 23 determines information to be output from audio output unit 25 based on the acquired feature amount. For example, various information such as the occurrence of ripples, the number of times ripples occurred, the type of ripples, the time when the ripples occurred, and the position of the ripples can be stored in association with the information OI to be output, and the information OI to be output can be determined based on the various information, and the audio information stored in the memory unit 24 can be determined in association with the determined information (information OI to be output).

[0112] Also, for example, when detecting the direction and length of movement of a water droplet (coffee) that has fallen on the surface of coffee (liquid) 41 stored in container 30, determination information for detecting the water droplet (for example, information for detecting an object moving linearly in each of the images that are successive in time series) is stored in storage unit 24. Then, control unit 23 (shown in FIG. 4) detects the occurrence of a water droplet moving on the surface from an image of a circular region (a circular region corresponding to the surface of coffee (liquid) 41 stored in container 30) in an image acquired from image acquisition unit 22 (shown in FIG. 4). In this case, control unit 23 can store in storage unit 24 the position at which the water droplet moving on the surface occurred (the position in the circular region corresponding to the surface of coffee (liquid) 41 stored in container 30), the length of the water droplet moving on the surface, the timing (elapsed time) at which the water droplet moving on the surface occurred, and the like, in association with the elapsed time. Next, control unit 23 determines information to be output from audio output unit 25 based on the acquired feature amount. For example, various information such as the presence or absence of water droplets moving along the surface, the number of times that water droplets moving along the surface have occurred, the total length of the water droplets moving along the surface, the time that the water droplets moving along the surface have occurred, and the position of the water droplets moving along the surface can be stored in association with the information OI to be output, and the information OI to be output can be determined based on the various information, and the audio information stored in the memory unit 24 can be determined in association with the determined information (information OI to be output).

[0113] Also, for example, when detecting the amount of foam generated by water droplets (coffee) falling on the surface of coffee (liquid) 41 stored in the container 30, instead of the image pattern shown in Fig. 12(1), a plurality of image patterns for detecting the amount of foam are stored in the storage unit 24. Then, the audio information can be determined in the same manner as the determination method using the judgment information shown in Fig. 12(1).

[0114] The above-mentioned pieces of judgment information are merely examples, and other pieces of judgment information may be used. For example, the positions (positions on the surface) of the water droplets (coffee) falling on the surface of the coffee (liquid) 41 stored in the container 30, the degree of spreading of the water droplets on the surface, the color depth (for example, the degree of coffee color) of the surface of the coffee (liquid) 41 stored in the container 30, the distribution of bubbles generated by the water droplets (coffee) falling on the surface of the coffee (liquid) 41 stored in the container 30, the proportion of the surface of the bubbles, and natural phenomena occurring during dripping, such as each transition of these, may be used as judgment information. Also, the above-mentioned pieces of judgment information may be combined to make a judgment. For example, the fallen ripples and the direction in which the water droplets move can be combined to make a judgment.

[0115] In the first embodiment, the electronic device 20 is configured as an integrated unit. However, the electronic device 20 may be configured as a plurality of components. For example, the lens 21 and the image acquisition unit 22 may be configured as separate components from other components, and the image acquisition unit 22 may transmit image information to the control unit 23 by wire or wirelessly. The position at which the electronic device 20 is installed is not limited to the positions shown in Figs. 1 and 3. For example, the electronic device 20 may be installed in another position as long as an image of the surface of the coffee (liquid) 41 stored in the container 30 can be acquired through the lens 21. For example, the electronic device 20 may be installed on the outside of the transparent container 30 so that the lens 21 faces the surface of the coffee (liquid) 41 stored in the container 30. However, in order to prevent the influence of heat generated in the container 30, it is preferable to perform a heat insulating process using a heat insulating member or the like.

[0116] [Second embodiment: Example of providing audio and image information using colors and patterns formed when brewing coffee] In the first embodiment, an example is shown in which audio information is provided using the colors and patterns formed when coffee is brewed, but other information can also be provided. Therefore, in the second embodiment, an example is shown in which audio information and image information are provided using the colors and patterns formed when coffee is brewed. Note that the second embodiment is a modified example of the first embodiment, and some of the explanations of the parts common to the first embodiment will be omitted.

[0117] [Example of the appearance of a coffee extractor] Fig. 13 is a perspective view showing an example of the external configuration of coffee extractor 200 to which electronic device 220 in the second embodiment is attached. Fig. 13(1) shows a side view of coffee extractor 200 (side view seen from the direction of arrow A shown in Fig. 13(1)).

[0118] Coffee extractor 200 has dripper 210, electronic device 220 (shown in FIG. 14), container 230, and stand 240. Note that coffee extractor 200 is configured such that electronic device 20 is provided on stand 240 instead of on dripper 10 in coffee extractor 1 shown in FIG. 1. Container 230 is similar to container 30 shown in FIG. 1. Dripper 210 is substantially similar to dripper 10 shown in FIGS. 1 and 3. Note that differences from coffee extractor 1 in the first embodiment will be mainly described here.

[0119] Stand 240 is a support base for supporting dripper 210, and can be formed of a flat plate-like member in a U-shape (as shown in FIG. 13(1)). For example, an upper portion of stand 240 (a portion where flange portion 212 shown in FIG. 13(1) is installed) is provided with a hole having a shape (e.g., a circular shape) that is larger than the diameter of leg portion 213 at the lower portion of dripper 210 and smaller than the diameter of flange portion 212 of dripper 210. When dripper 210 is installed on stand 240, it can be installed so that leg portion 213 at the lower portion of dripper 210 fits into the hole of stand 240.

[0120] Lens 221 is disposed outside the hole in the upper part of stand 240 (the part where flange portion 212 shown in FIG. 13(1) is disposed). It is preferable to place lens 221 in a position where it will not be splashed with liquid (coffee) dropping from the hole (inside leg portion 213) of dripper 210 and will not be significantly affected by the heat of the liquid.

[0121] A user interface 229 is disposed in the middle (rear) of the stand 240. The user interface 229 can be configured, for example, by a touch panel that displays various information and accepts user operations. Note that the stand 240 can be made of, for example, heat-resistant glass, pottery, metal, ceramic, resin, wood, etc.

[0122] [Example of electronic device functional configuration] FIG. 14 is a block diagram showing an example of a functional configuration of the electronic device 220 according to the second embodiment.

[0123] The electronic device 220 has a lens 221, an image acquisition unit 222, a control unit 223, a storage unit 224, an audio output unit 225, and a user interface 229. The electronic device 220 is the electronic device 20 shown in FIG. 4 to which a user interface 229 has been added, and the other parts are common to the electronic device 20 shown in FIG. 4. Therefore, here, the differences from the electronic device 20 shown in FIG. 4 will be mainly described. As shown in FIG. 13, the lens 221 and the image acquisition unit 222 are installed at the top of the stand 240, and the user interface 229 is installed at the middle (back) of the stand 240. The other parts may be installed integrally with the user interface 229 at the middle (back) of the stand 240, or may be installed at the bottom or top of the stand 240.

[0124] The user interface 229 is composed of an input unit (operation receiving unit) 226 and a display unit 227 .

[0125] The input unit 226 is an operation receiving unit that receives an operation input performed by a user, and outputs operation information corresponding to the received operation input to the control unit 223. The input unit 226 is realized by, for example, a touch panel, a keyboard, a mouse, or a sensor (for example, a touch interface).

[0126] The display unit 227 is a display unit that outputs various information based on the control of the control unit 223. For example, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel can be used as the display unit 227. In this manner, the input unit 226 and the display unit 227 can be integrally configured using a touch panel that allows a user to perform an operation input by touching or approaching the display surface with his / her finger. However, the input unit 226 and the display unit 227 may be configured separately, and the input unit 226 may receive an operation by an operation member. Also, an operation member separate from the user interface 229 such as a touch panel may be provided, and the operation may be received by this operation member.

[0127] [Example of output information] Fig. 15 is a diagram showing an example of information to be output by the electronic device 220 in the second embodiment. Note that this example shows an example in which the determination information shown in Figs. 6(1) and (2) is used.

[0128] FIG. 15 shows an example of output target information (information to be output) determined using the determination information shown in FIG. 6(2). For example, when OI1 is determined as the information OI to be output, information about Santa (audio information 61, image information 62) is output. Information about Santa is, for example, information for outputting the sound "Today, let's dream of Santa of your dreams" (audio information 61) and information for displaying a photo or a graphic image of Santa (image information 62). Note that, in FIG. 15, for ease of explanation, characters to be output as audio and images to be displayed are simply shown. Note that these output examples will be explained with reference to FIG. 17.

[0129] [An example of determining the information to be output using the colors and patterns formed when brewing coffee] Fig. 16 is a diagram showing an example of determining information to be output by the electronic device 220 in the second embodiment. Note that the example shown in Fig. 16 shows an example in which times t1 to t8 are used as the elapsed time when coffee is extracted using the coffee extractor 200, similar to the example shown in Fig. 9. Also, Figs. 16(1) and (2) are the same as Figs. 9(1) and (2).

[0130] As described in the first embodiment, the control unit 223 (shown in Fig. 14) extracts the feature amount shown in the dashed rectangle 51 from the determination information shown in Fig. 16(2) as the feature amount that coincides with the feature amount shown in Fig. 16(1). Furthermore, the control unit 223 determines OIX1 as information (information to be output) OI corresponding to the extracted feature amount.

[0131] Next, the control unit 223 causes the audio output unit 225 to output audio based on the audio information 61 stored in the storage unit 224 in association with the OIX1 determined as the information OI to be output. The control unit 223 also causes the display unit 227 to display an image based on the image information 62 stored in the storage unit 224 in association with the OIX1 determined as the information OI to be output.

[0132] For example, as shown in Fig. 16(3), based on audio information 61 (shown in dashed rectangle 54) associated with OIX1, the audio "Today, let's wait patiently like a snowman" is output from audio output unit 225. Also, for example, as shown in Fig. 16(3), based on image information 62 (shown in dashed rectangle 54) associated with OIX1, an image of Santa is displayed on display unit 227. An example of this output is shown in Fig. 17.

[0133] [Example of outputting audio and image information from an electronic device when brewing coffee] Fig. 17 is a simplified diagram showing an example of a case where audio information and image information are output from electronic device 220 in the second embodiment. Note that the example shown in Fig. 17 shows an example of outputting audio information determined by the examples shown in Figs. 8 and 16.

[0134] As described above, the control unit 223 outputs the sound "Today, let's wait patiently like a snowman" from the sound output unit 25 based on the sound information 61 (in the dashed rectangle 54 shown in FIG. 16(3)) associated with the OIX1. The control unit 223 also displays an image of Santa on the display unit 227 based on the image information 62 (in the dashed rectangle 54 shown in FIG. 16(3)) associated with the OIX1. This allows the user P1, who is brewing coffee using the coffee extractor 200, to enjoy the sound and image based on the feature amount acquired by the extraction after brewing his or her ideal coffee. In this way, by displaying the image together with the sound, the user P1 can visually enjoy the information based on the feature amount acquired by the extraction. Note that FIG. 17 shows an example in which the sound and the image are output simultaneously, but the sound and the image may be output at different times, or only the image may be output.

[0135] [Example of outputting audio and image information] Fig. 18 is a flowchart showing an example of information output processing by the electronic device 220 in the second embodiment. Note that steps S112 to S115 shown in Fig. 18 correspond to steps S101 to S104 shown in Fig. 11. Therefore, the following description will focus on the points that are different from the example shown in Fig. 11.

[0136] First, the control unit 223 of the electronic device 220 judges whether or not a start operation has been performed (step S111). For example, it is judged whether or not a touch operation of the start button (shown in FIG. 13(2)) has been performed on the user interface 229. If a start operation has been performed (step S111), the process proceeds to step S112. If a start operation has not been performed (step S111), the monitoring continues.

[0137] As described above, steps S112 to S115 correspond to steps S101 to S104 shown in FIG.

[0138] Here, as a criterion for judging whether the end condition of step S115 is satisfied, other than the criterion shown in the first embodiment, for example, whether or not a user has performed an end operation may be judged. For example, after a start button (shown in FIG. 13(2)) is touched in the user interface 229, an end button may be displayed, and if the end button is touched, it may be judged that the end condition is satisfied.

[0139] When the termination condition is satisfied (step S115), the control unit 223 determines information to be output based on the acquired feature amount (step S116). That is, the control unit 223 determines the audio information and image information stored in the storage unit 224 in association with the determined information (information OI to be output).

[0140] Next, the control unit 223 causes the audio output unit 225 to output audio based on the determined audio information, and also causes the user interface 229 (display unit 227) to display an image based on the determined image information (step S117). For example, as shown in Fig. 17, audio is output from the audio output unit 225, and an image is displayed on the user interface 229 (display unit 227).

[0141] [Third embodiment: Example of outputting audio information and image information in an external device] In the first and second embodiments, an example was shown in which audio information and image information were output by an electronic device attached to a coffee extractor. However, audio information and image information may be output from a device other than the coffee extractor. This eliminates the need to provide an audio output unit and a display unit in the electronic device attached to the coffee extractor, thereby reducing the manufacturing cost of the electronic device. Note that the third embodiment is a modified example of the first and second embodiments, and some of the explanations of the parts common to the first and second embodiments will be omitted.

[0142] [Example of electronic device functional configuration] FIG. 19 is a block diagram showing an example of a functional configuration of an electronic device 320 according to the third embodiment.

[0143] The electronic device 320 has a lens 321, an image acquisition unit 322, a control unit 323, a storage unit 324, and a communication unit 325. The electronic device 320 is similar to the electronic device 20 shown in FIG. 4 except that the communication unit 325 is added and the audio output unit 25 is omitted. The other parts are common to the electronic device 20 shown in FIG. 4 and the electronic device 220 shown in FIG. 14. Therefore, the following description will focus on the differences from the electronic device 20 shown in FIG. 4 and the electronic device 220 shown in FIG. 14. Although not shown in FIG. 19, an audio output unit and a user interface may be provided.

[0144] The communication unit 325 exchanges various information with other devices via a wired or wireless line based on the control of the control unit 323. For example, when exchanging various information using wireless communication, the communication unit 325 transmits and receives radio waves via an antenna (not shown). For example, the communication unit 325 can perform wireless communication using a wireless LAN (Local Area Network) (e.g., Wi-Fi (Wireless Fidelity)), NFC (Near Field Communication), Bluetooth (registered trademark), infrared, mobile phone radio waves, etc. The communication unit 325 is realized, for example, by a communication module having an antenna and a wireless communication function, or a communication module having a wired communication function.

[0145] [Example of external device configuration] FIG. 20 is a perspective view showing an example of the external configuration of external devices (electronic devices 350, 360) in the third embodiment. FIG. 20(1) shows a portable electronic device 350. The electronic device 350 is a portable information processing device, such as a smartphone, a tablet terminal, or a personal computer. FIG. 20(2) shows a stationary electronic device 360. The electronic device 360 ​​is a stationary speaker device, such as an AI speaker equipped with artificial intelligence (AI), a speaker with wireless communication capabilities, or the like.

[0146] [Example of electronic device functional configuration] Fig. 21 is a block diagram showing an example of a functional configuration of an electronic device 350 according to the third embodiment. Note that the functional configuration of an electronic device 360 ​​shown in Fig. 20 is substantially similar to that of the electronic device 350, and therefore a description thereof will be omitted here.

[0147] The electronic device 350 has a communication unit 351, a control unit 352, a storage unit 353, an audio output unit 354, a user interface 357 (input unit 355, display unit 356), and a location information acquisition unit 358. Each unit in the electronic device 350 corresponds to each unit of the same name in the electronic device 20 shown in FIG. 4 and the electronic device 220 shown in FIG. 14. Therefore, the following description will focus on the differences from the electronic device 20 shown in FIG. 4 and the electronic device 220 shown in FIG. 14. The communication unit 351 in the electronic device 350 can perform wireless communication with electronic devices present within a predetermined range, and can also perform wireless communication by connecting to a predetermined network (e.g., the Internet) via a base station installed by a communication carrier such as a mobile phone company.

[0148] The location information acquisition unit 358 acquires location information on the location where the electronic device 350 exists, and outputs the acquired location information to the control unit 352. The location information acquisition unit 358 can be realized, for example, by a GPS receiver that receives a GPS (Global Positioning System) signal and calculates location information based on the GPS signal. The calculated location information includes data on the position such as latitude, longitude, and altitude at the time of receiving the GPS signal. A location information acquisition device that acquires location information by other location information acquisition methods may be used. For example, a location information acquisition device that derives location information using access point information by a wireless LAN (Local Area Network) present in the vicinity and acquires this location information may be used.

[0149] [Communication when outputting audio and image information from external devices other than electronic devices] Fig. 22 is a flowchart showing an example of communication processing by the electronic device 320 and the electronic device 350 in the third embodiment. Steps S122 to S126 shown in Fig. 22 correspond to steps S112 to S116 shown in Fig. 18. Therefore, the following description will focus on the points that are different from the example shown in Fig. 18. It is also assumed that applications (programs) for realizing the processes shown in Fig. 22 are preset in the electronic devices 320 and 350.

[0150] First, the control unit 323 of the electronic device 320 detects a start instruction (step S121). In the case where the electronic device 320 is provided with a start operation member (not shown), this start instruction is detected by the user operating the start operation member. It is also possible that the user performs a start operation using the electronic device 350, and start instruction information is transmitted from the electronic device 350 to the electronic device 320 in response to this start operation. In this case, the start instruction is detected by receiving the start instruction information from the electronic device 350. Also, a sensor (for example, a human detection sensor, a temperature sensor, a water drop sensor, a vibration sensor) for detecting the start instruction may be provided in the electronic device 320, and the start instruction may be automatically detected by this sensor.

[0151] As described above, steps S122 to S126 correspond to steps S112 to S116 shown in FIG.

[0152] The control unit 323 determines the information to be output (audio information and image information) (step S126). Next, the communication unit 325 transmits the determined audio information and image information to the electronic device 350 under the control of the control unit 323 (step S127).

[0153] The communication unit 351 of the electronic device 350 receives the information (audio information and image information) transmitted from the electronic device 320 (step S128). Next, the control unit 352 of the electronic device 350 stores the received information (audio information and image information) in the storage unit 353 (step S129). Then, the control unit 352 of the electronic device 350 outputs the received information (audio information and image information) (step S129). That is, the control unit 352 of the electronic device 350 causes the audio output unit 354 to output audio based on the received audio information, and causes the user interface 357 (display unit 356) to display an image based on the received image information (step S129). For example, as shown in FIG. 23(1), audio is output from the audio output unit 354 and an image is displayed on the user interface 357 (display unit 356).

[0154] [Example of outputting audio and image information from a device other than an electronic device attached to a coffee brewer] Fig. 23 is a simplified diagram showing an example of a case where audio information and image information are output from devices (electronic devices 350, 360) other than the electronic device 220 in the third embodiment. Fig. 23(1) shows an example of a case where audio information and image information are output from the electronic device 350. Fig. 23(2) shows an example of a case where audio information and image information are output from the electronic device 360. Note that the example shown in Fig. 23 shows an example of outputting audio information determined by the examples shown in Figs. 8 and 16.

[0155] It is also assumed that applications (programs) for implementing the processes shown in FIG. 22 are preset in the electronic devices 350 and 360.

[0156] As described in the second embodiment, the control unit 323 of the electronic device 320 determines the audio information 61 (within the dashed rectangle 54 shown in FIG. 16(3)) and image information 62 (within the dashed rectangle 54 shown in FIG. 16(3)) associated with OIX1 as information to be output. Next, the control unit 323 of the electronic device 320 performs control to wirelessly transmit the audio information 61 (within the dashed rectangle 54 shown in FIG. 16(3)) and image information 62 (within the dashed rectangle 54 shown in FIG. 16(3)) determined as information to be output from the communication unit 325 to an external device.

[0157] For example, when audio information and image information are output from the electronic device 350, the control unit 323 of the electronic device 320 performs control to wirelessly transmit the audio information 61 (in the dashed rectangle 54 shown in FIG. 16(3)) and the image information 62 (in the dashed rectangle 54 shown in FIG. 16(3)) determined as the information to be output from the communication unit 325 to the electronic device 350. The control unit 352 of the electronic device 350 that has received each of these pieces of information outputs the audio of "Today, let's wait patiently like a snowman" from the audio output unit 354 based on the received audio information. In addition, the control unit 352 displays an image of Santa on the user interface 357 (display unit 356) based on the received image information. In this way, the user P1 who is brewing coffee using the coffee extractor 300 can enjoy the audio and images based on the feature amount acquired by the extraction while tasting the coffee after brewing the coffee of his / her ideal taste, even in a place far from the coffee extractor 300. Furthermore, for example, the user P1 can store the output sound and the displayed image of Santa in the storage unit 353 of the electronic device 350 and enjoy them later.

[0158] Furthermore, for example, when audio information is output from the electronic device 360, the control unit 323 of the electronic device 320 performs control to wirelessly transmit the audio information 61 (within the dashed rectangle 54 shown in FIG. 16(3)) determined as the information to be output from the communication unit 325 to the electronic device 360. The control unit of the electronic device 360 ​​that receives this audio information outputs the audio "Today, let's wait patiently like a snowman" from the audio output unit based on the received audio information. This allows the user P1, who is brewing coffee using the coffee extractor 300, to enjoy the audio based on the feature amount acquired by the extraction while tasting the coffee after brewing his or her ideal coffee, even in a place away from the coffee extractor 300.

[0159] In the above, an example has been shown in which audio and images are output from the electronic devices 350 and 360 that can directly communicate wirelessly with the electronic device 320. However, audio and images may be output from devices that cannot directly communicate wirelessly with the electronic device 320. For example, in the electronic device 320, each piece of information (audio information, image information) determined as information to be output is wirelessly transmitted from the communication unit 325 to the electronic devices 350 and 360, and the electronic devices 350 and 360 that have received the information (audio information, image information) transfer the information (audio information, image information) received from the electronic device 320 to a predetermined device (for example, a device designated as a destination by the electronic device 320 or a device that has been preset as a destination). For example, the electronic devices 350 and 360 can connect to a predetermined network (for example, the Internet) using wireless communication and transmit the information (audio information, image information) received from the electronic device 320 to a predetermined destination. As a result, for example, user P1 who is brewing coffee using coffee extractor 300 can, after finishing brewing his or her ideal coffee, provide a person who is far away from coffee extractor 300 with audio and images based on the features acquired by the extraction (for example, from an electronic device carried by the person).

[0160] [Fourth embodiment: an example in which part of the processing performed by an electronic device attached to a coffee extractor is performed by another electronic device] In the first to third embodiments, an example was shown in which the information to be output is determined by an electronic device attached to the coffee extractor. Here, it is possible to have the electronic device attached to the coffee extractor perform only relatively simple processing, and have other processing performed by another device with high processing power. Therefore, in the fourth embodiment, an example is shown in which part of the processing performed by the electronic device attached to the coffee extractor is performed by another electronic device.

[0161] The fourth embodiment will be described with reference to electronic device 320 (shown in FIG. 19) and electronic device 350 (shown in FIG. 21) described in the third embodiment.

[0162] For example, electronic device 320 (shown in FIG. 19) attached to coffee extractor 300 (shown in FIG. 23) performs a process of acquiring an image of the surface of coffee (liquid) accumulated in container 330 (shown in FIG. 23). Then, electronic device 320 sequentially transmits the acquired image information (image of the surface of coffee (liquid) accumulated in container 330) from communication unit 325 (shown in FIG. 19) to electronic device 350 periodically or irregularly.

[0163] The storage unit 353 (shown in FIG. 21) of the electronic device 350 stores the judgment information (e.g., each judgment information shown in FIG. 6(1)(2) and each judgment information shown in FIG. 12(1)(2)) and the output target information (e.g., the output target information shown in FIG. 7 and the output target information shown in FIG. 15) shown in the first to third embodiments. The control unit 352 (shown in FIG. 21) of the electronic device 350 sequentially stores the image information (image of the surface of the coffee (liquid) accumulated in the container 330) received from the electronic device 320 via the communication unit 351 in the storage unit 353. Next, the control unit 352 (shown in FIG. 21) of the electronic device 350 performs a process of determining the output target information for the image information (image of the surface of the coffee (liquid) accumulated in the container 330) received from the electronic device 320 using the judgment information stored in the storage unit 353. Next, the control unit 352 of the electronic device 350 outputs each piece of information determined by the determination process.

[0164] 23(1), the control unit 352 of the electronic device 350 causes the audio output unit 354 to output the audio "Today, let's wait patiently like a snowman" based on the audio information determined by the determination process. Also, the control unit 352 causes the user interface 357 (display unit 356) to display an image of Santa based on the image information determined by the determination process.

[0165] [Example of communication when performing a series of processes among multiple electronic devices] Fig. 24 is a flowchart showing an example of communication processing by the electronic device 320 and the electronic device 350 in the fourth embodiment. Note that steps S131, S133, S135, and S143 to S146 shown in Fig. 24 correspond to steps S121 to S126 shown in Fig. 22. Therefore, the following description will focus on the points that are different from the example shown in Fig. 22.

[0166] When the control unit 323 of the electronic device 320 detects a start instruction (step S131), the control unit 323 transmits start information for notifying the start of the operation to the electronic device 350 from the communication unit 325 (step S132).

[0167] When the control unit 352 of the electronic device 350 receives the start information via the communication unit 351, it grasps the start of the operation of the electronic device 320 (step S141).

[0168] The control unit 323 of the electronic device 320 acquires the image (an image (including image data) of the coffee accumulating in the container 330) acquired by the image acquisition unit 322 (step S133), and transmits the acquired image information to the electronic device 350 (step S134). In this case, the transmission of the image information from the electronic device 320 to the electronic device 350 may be performed periodically or irregularly.

[0169] Step S135 shown in FIG. 24 corresponds to step S125 shown in FIG.

[0170] When the control unit 352 of the electronic device 350 receives the image information from the electronic device 320 (step S142), it analyzes the received image information and acquires a feature amount (step S143). Note that steps S143 to S146 shown in FIG. 24 correspond to steps S123 to S126 shown in FIG.

[0171] Here, as a criterion for judging whether or not the termination condition of step S145 is satisfied, other than the criteria shown in the first to third embodiments, for example, a criterion may be that image information has not been received from the electronic device 320 for a predetermined time or more. For example, when image information has not been received from the electronic device 320 for a predetermined time or more, the control unit 352 of the electronic device 350 can judge that the termination condition is satisfied (step S145).

[0172] The control unit 352 of the electronic device 350 determines information to be output (audio information and image information) (step S146). Next, the control unit 352 of the electronic device 350 outputs the determined audio information and image information (step S147). That is, the control unit 352 of the electronic device 350 causes the audio output unit 354 to output audio based on the determined audio information, and causes the user interface 357 (display unit 356) to display an image based on the determined image information (step S147). For example, as shown in FIG. 23, audio is output from the audio output unit 354 and an image is displayed on the user interface 357 (display unit 356).

[0173] In the above, an example has been shown in which the determination information shown in Fig. 6 and Fig. 12 and the output target information shown in Fig. 7 and Fig. 15 are stored in advance in the storage units 24, 224, 324 of the electronic devices 20, 220, 320. However, each of these pieces of information may be acquired from the outside and used. For example, the electronic device may be provided with a communication unit (e.g., the communication unit 325 shown in Fig. 19) capable of communicating with other devices via a wired line or a wireless line, and the electronic device may be connected to an external device (or a predetermined network (e.g., the Internet)) via this communication unit, and all or part of the pieces of information may be acquired from the external device. In this case, each piece of information may be acquired and used every time the coffee extractor (electronic device) is used, or each piece of information may be acquired and used periodically or irregularly.

[0174] In the above, an example has been shown in which the determination information shown in Fig. 6 and Fig. 12 and the output target information shown in Fig. 7 and Fig. 15 are stored in advance in the storage units 24, 224, 324 of the electronic devices 20, 220, 320 (or other nearby devices), and the electronic devices 20, 220, 320 (or other nearby devices) perform the processes for outputting the content. However, this information may be stored in an external device (e.g., a server) located at a remote location, and the external device may perform the processes for outputting the content, while the electronic devices 20, 220, 320 (or other nearby devices) only acquire information related to coffee brewing and output the content.

[0175] Also, for example, a specific supplier (for example, a company that provides specific content or discount coupons) may be partnered with to output the output target image. For example, a specific image transition (for example, an image pattern (shown in FIG. 12(1))) may be provided from a specific supplier, and when the image matches the specific image transition, information provided by the specific supplier (for example, a discount coupon, specific content) may be output from the electronic device. An example of this display is shown in FIG. 45.

[0176] Also, the user may register output target information, and the information to be output may be determined from the registered output target information by a determination process. For example, the output target information (e.g., Santa) on the day coffee was brewed using the coffee extractor and the behavior history of that day (e.g., went to an amusement park, took an English test, etc.) are associated and registered (stored). Then, when the information determined as the output target when coffee is brewed using the coffee extractor at a later date is the same as the registered output target information (e.g., Santa), the output target information (e.g., Santa) and information on the behavior history of the registered day (e.g., went to an amusement park, took an English test, etc.) are output. This allows the user to enjoy the taste of coffee while remembering what he or she did on the day coffee with the same pattern was brewed in the past. Also, the user can enjoy coffee while feeling the same as on the day coffee with the same pattern was brewed in the past, and thinking about various things.

[0177] Furthermore, content to be output may be determined using artificial intelligence (AI). For example, machine learning may be used to generate determination information on the pattern when coffee is brewed using a coffee brewer (e.g., the pattern of the surface of the coffee) and the mood and physical condition of the day (e.g., determined by searches and movement history of a smartphone, etc.), and the generated determination information may be used to determine content to be output. For example, content showing a smile may be output when the user is in a good mood, and content that cheers the user up may be output when the user is in a bad mood. In this way, for example, content may be output using some feature amount related to the surface of coffee brewed when the user is in a good mood and the surface of coffee brewed when the user is in a bad mood, and the mood at that time may be estimated.

[0178] [Fifth embodiment: Example of aggregating information from multiple coffee extractors] In the first to fourth embodiments, an example was shown in which information (information about coffee extraction) acquired by one coffee extractor is used to output audio information and image information. In recent years, coffee extractors have become widespread in homes around the world. Each home brews its own preferred coffee. If information about the coffee brewed in each home could be collected and some information could be provided to each home based on this information, it would be possible to enhance the enjoyment of brewing coffee. Similarly, if information about coffee brewed at workplaces, schools, and other places other than the home could be collected and some information could be provided to each person based on this information, it would be possible to enhance the enjoyment of brewing coffee. In the fifth embodiment, an example is shown in which information from multiple coffee extractors is collected and used.

[0179] [Example of communication system configuration] FIG. 25 is a diagram illustrating an example of a system functional configuration of a communication system 400 according to the fifth embodiment.

[0180] The communication system 400 includes a server 410, a network 420, and electronic devices 431, 441, 442, 451, and 452. Note that, although base stations and access points for wireless communication are also assumed as devices constituting the communication system 400, illustration and description thereof will be omitted here.

[0181] The server 410 is a server (information processing device) that collects and manages information transmitted from each of the electronic devices 431, 441, and 451, and provides various types of information to each of the electronic devices 431, 441, 442, 451, and 452 using the managed information.

[0182] The network 420 is a network such as a public line network or the Internet.

[0183] Electronic device 431 is attached to coffee extractor 430, electronic device 441 is attached to coffee extractor 440, and electronic device 451 is attached to coffee extractor 450. Electronic device 442 performs wireless communication with electronic device 441, and electronic device 452 performs wireless communication with electronic device 451.

[0184] Moreover, the electronic device 431 has a communication unit (corresponding to the communication unit 325 shown in FIG. 19) and can connect to the network 420 using wireless communication to exchange various information. Moreover, each of the electronic devices 441 and 451 has a communication unit (corresponding to the communication unit 325 shown in FIG. 19) and can exchange information with other devices (electronic devices 442 and 452) using wireless communication. Each of the electronic devices 441 and 451 can connect to the network 420 via the other devices (electronic devices 442 and 452) using wireless communication to exchange various information.

[0185] [Example of electronic device functional configuration] Fig. 26 is a block diagram showing an example of a functional configuration of electronic device 431 in the fifth embodiment. Note that the functional configurations of electronic devices 441 and 451 shown in Fig. 25 are substantially similar to that of electronic device 431 except that they do not have a user interface, and therefore descriptions thereof will be omitted here. In addition, the functional configurations of electronic devices 442 and 452 shown in Fig. 25 are also substantially similar to that of electronic device 431, and therefore descriptions thereof will be omitted here.

[0186] The electronic device 431 has a lens 481, an image acquisition unit 482, a control unit 483, a storage unit 484, an audio output unit 485, a user interface 488 (an input unit 486, a display unit 487), and a communication unit 489. Each unit in the electronic device 431 corresponds to each unit of the same name in the electronic device 20 shown in FIG. 4, the electronic device 220 shown in FIG. 14, and the electronic device 320 shown in FIG. 19. Therefore, the following description will focus on the differences from the electronic device 20 shown in FIG. 4, the electronic device 220 shown in FIG. 14, and the electronic device 320 shown in FIG. 19. The communication unit 489 in the electronic device 431 can perform wireless communication with electronic devices present within a predetermined range, and can also connect to a predetermined network (for example, the Internet) via a base station installed by a communication carrier such as a mobile phone company to perform communication.

[0187] The location information setting unit 490 sets location information relating to the location where the electronic device 431 is located, and outputs the set location information to the control unit 483. The location information setting unit 490 stores, for example, location information (for example, latitude and longitude, and area name (country, administrative district)) relating to the location where the coffee extractor 430 to which the electronic device 431 is attached is installed. For example, the location information relating to the location where the coffee extractor 430 is installed is acquired by a user operation or a location information acquisition unit (for example, a built-in GPS receiver).

[0188] [Server functional configuration example] FIG. 27 is a block diagram showing an example of a functional configuration of the server 410 in the fifth embodiment.

[0189] The server 410 includes a communication unit 411 , a control unit 412 , and a storage unit 413 .

[0190] The communication unit 411 exchanges various information with other devices via a wired line or a wireless line based on the control of the control unit 412. For example, the communication unit 411 exchanges various information with the electronic devices 431, 441, 442, 451, and 452 via the network 420.

[0191] The control unit 412 controls each unit of the server 410 based on a control program stored in the storage unit 413. The control unit 412 is realized by, for example, a CPU (Central Processing Unit).

[0192] The storage unit 413 is a memory that stores various information. For example, the storage unit 413 stores various information (e.g., control programs) required for the server 410 to perform various processes. The storage unit 413 also stores management information for managing each piece of information (information transmitted from the electronic devices 431, 441, 451) acquired by the communication unit 411 via the network 420. The storage unit 413 also stores content to be provided to each electronic device (electronic devices 431, 441, 451) connected via the network 420.

[0193] [Example of management information configuration] Fig. 28 is a diagram showing an example of management information stored in storage unit 413 in the fifth embodiment. Fig. 28(1) shows an example of management information to be managed by server 410. Fig. 28(2) shows an example of content provided from server 410 to each electronic device.

[0194] 28(1) is composed of terminal identification information 461, reception time 462, reception information 463, determination result information 464, and provided content information 465. These pieces of information (terminal identification information 461, reception time 462, reception information 463) are transmitted from each electronic device connected to the server 410, and are stored in sequence in the storage unit 413 by the control unit 412.

[0195] The terminal identification information 461 is identification information for identifying each electronic device (e.g., a device-specific ID (identification), a MAC (Media Access Control) address). For example, the terminal identification information 461 is identification information for identifying an electronic device to which content is to be transmitted. Note that in FIG. 28, for ease of explanation, the terminal identification information is expressed as a simple combination of letters and numbers.

[0196] The reception time 462 is information about the time when information was received from each electronic device connected to the server 410. In the management information shown in Fig. 28(1), for example, each piece of information received in a specific time period (for example, one day based on a certain point on the earth) is stored in order.

[0197] The received information 463 is information transmitted from each electronic device connected to the server 410. This received information is, for example, information transmitted from each electronic device based on a predetermined rule. For example, when performing a determination process using the determination information shown in FIG. 6 as the predetermined rule, a part of each piece of information shown in the dashed rectangle 51 in FIG. 9(2) (for example, information OI (OIX1) to be output) is transmitted. In this case, the information OI (OIX1) to be output is stored as the received information 463. Note that other information (for example, each piece of information shown in FIG. 9(1) (the letters A, A, D, E, F, E, I, and F and their order)) may also be stored in the received information 463 as a transmission target.

[0198] The determination result information 464 stores the result determined by the control unit 412 based on the information stored in the received information 463. For example, the control unit 412 (shown in FIG. 27) of the server 410 determines the identification information (e.g., serial number) to be assigned to each electronic device based on the information stored in the received information 463. For example, the control unit 412 converts the information stored in the received information 463 (received information of each electronic device) into comparable numerical values ​​(e.g., numerical values ​​(1, 2, 3, ...) of the order of the information OI to be output (OI1, OI2, OI3, ...) shown in FIG. 6(2)) based on a predetermined rule, and can determine the identification information (e.g., serial number) to be assigned to each electronic device based on a comparison of the numerical values.

[0199] Also, the identification information (e.g., serial number) to be assigned to each electronic device may be determined based on other rules. For example, the control unit 412 can determine the identification information (e.g., serial number) to be assigned to each electronic device based on the reception time 462 and the information stored in the reception information 463. For example, the control unit 412 can determine a value (e.g., 1, 2, 3, ... in order of the earliest reception time) to be multiplied by a numerical value specified by the reception information 463 based on the reception time 462, and determine the identification information (e.g., serial number) to be assigned to each electronic device based on the calculation results. Also, for example, the control unit 412 performs an arithmetic process on the information stored at the reception time 462 based on a predetermined rule (for example, in the case of 20171204 07:56, the numerical values ​​of each digit are simply added to obtain "35"), and can determine the identification information (for example, a serial number) to be assigned to each electronic device based on the result of an arithmetic process (for example, multiplication, addition, subtraction, division) between the numerical value of this calculation result and the numerical value specified by the reception information 463. In this way, a rule for assigning the identification information (for example, a serial number) to each electronic device is set, and the identification information (for example, a serial number) to be assigned to each electronic device is determined based on this rule. In this way, the identification information (for example, a serial number) assigned to each electronic device does not indicate the superiority of the electronic device (or the user, or the coffee brewed by the user), but means the identification information for assigning the content.

[0200] The provided content information 465 is information for identifying the content determined based on the information stored in the determination result information 464. When the number of electronic devices to be managed is the same as the number of contents that can be provided to the electronic devices to be managed, or when the number of contents that can be provided to the electronic devices to be managed is greater than the number of electronic devices to be managed, the value of the determination result information 464 and the information of the provided content information 465 correspond one-to-one. However, when the number of contents that can be provided to the electronic devices to be managed is less than the number of electronic devices to be managed, one piece of information in the provided content information 465 corresponds to multiple values ​​in the determination result information 464. That is, the same content may be provided to multiple electronic devices. The provided content information 465 corresponds to the provided content information 471 shown in FIG. 28(2).

[0201] The content management information shown in FIG. 28(2) is made up of provided content information 471 and content 472.

[0202] Content 472 is content provided to the electronic device that has transmitted the information to be stored in received information 463. The content can be, for example, content consisting of only audio, content consisting of audio and images, content consisting of audio and video, content relating to smell, taste, touch, etc. Furthermore, the content can be each of characters A, B, C, ..., XYZ of an anime having a plurality of types of characters, as shown in FIG. 28(2). In this case, each character may not be provided in duplicate so that the same character is not provided to different electronic devices, and some characters may be provided in duplicate so that the same character is provided to some electronic devices.

[0203] Also, information on at least one movie among a plurality of types of movie information (e.g., movie promotional videos, pamphlets) may be provided as content. Information on at least one music among a plurality of music (e.g., singer videos, concert videos) may be provided as content. Information on at least one travel destination (e.g., Kusatsu Onsen, Ikaho Onsen) among a plurality of travel destination information (places, famous places, transportation, discount coupons, etc.) may be provided as content. Also, at least one of a plurality of discount coupons, complimentary coupons, and discount coupons that can be used on the Internet may be provided as content. Also, discount coupons, complimentary coupons, and discount coupons related to at least one store among a plurality of stores may be provided as content. Also, information on point allocation of at least one store among a plurality of stores may be provided as content. Also, information on fortune telling (e.g., today's coffee fortune telling) may be provided as content. In this way, the content may be information about one or more of anime characters, animals (e.g., animals appearing in a movie, amusement parks at the travel destination), people (e.g., characters in a movie, celebrities at the travel destination), plants (e.g., local specialties at the travel destination, flowers of interest), local features (e.g., maps at the travel destination, buildings at the travel destination, tourist spots at the travel destination), music, etc. Also, for example, the content may be provided by applying AR (Augmented Reality). For example, each content may be displayed superimposed on an image (photograph) generated by a camera of an electronic device (e.g., a smartphone camera). Note that each of these contents may be output targets in the first to fourth embodiments.

[0204] [Examples of information stored on each electronic device and examples of content provided to each electronic device] Fig. 29 is a diagram showing an example of storage of information related to electronic devices 431, 441, and 451 in the fifth embodiment, and an example of content provided to electronic devices 431, 441, and 451. Fig. 29(1) corresponds to Fig. 28(1), and Fig. 29(2) corresponds to Fig. 28(2).

[0205] In the storage example shown in Fig. 29(1), the terminal identification information 461 "EX0431" indicates the terminal identification information of the electronic device 431, the terminal identification information 461 "EX0441" indicates the terminal identification information of the electronic device 441, and the terminal identification information 461 "EX0451" indicates the terminal identification information of the electronic device 451. Each piece of information shown in Fig. 29 will be described with reference to Figs. 30 to 32.

[0206] [Example of communication when the server 410 acquires information from the electronic device 431] Fig. 30 is a flowchart showing an example of communication processing by the electronic device 431 and the server 410 in the fifth embodiment. Steps S151 to S155 shown in Fig. 30 correspond to steps S121 to S125 shown in Fig. 22. Therefore, here, differences from the example shown in Fig. 22 will be mainly described. Also, it is assumed that a program for accessing the server 410 and performing various communications with the server 410 is set in advance in the electronic device 431. That is, it is assumed that an application (program) for realizing each process shown in Fig. 30 is set in advance in the electronic device 431.

[0207] As described above, steps S151 to S155 correspond to steps S121 to S125 shown in FIG.

[0208] The control unit 483 (shown in FIG. 26) of the electronic device 431 determines information to be output (for example, information OI (OIX1) to be output shown in FIG. 9(2)) (step S156). Next, the communication unit 489 (shown in FIG. 26) transmits the determined information to the server 410 under the control of the control unit 483 (step S157).

[0209] The communication unit 411 of the server 410 receives the information transmitted from the electronic device 431 (step S158). Next, the control unit 412 of the server 410 stores the received information in the storage unit 413 (step S159). For example, as shown in FIG. 29(1), the received information (OIX1 (shown as "X1" in FIG. 29(1))) is stored as received information 463 in association with the terminal identification information 461 "EX0431". Also, as shown in FIG. 29(1), information regarding the time when the information (OIX1) was received from the electronic device 431 is stored as received time 462 in association with the terminal identification information 461 "EX0431".

[0210] 25, information on the electronic device 441 is transmitted from the electronic device 441 to the server 410 via the electronic device 442. In this case, the electronic device 442 transmits the terminal identification information on the electronic device 441 to the server 410 together with the information (Y3) transmitted by the electronic device 441. Then, as shown in FIG. 29(1), the information (Y3) transmitted by the electronic device 441 is stored as received information 463 in association with the terminal identification information 461 "EX0441", and information on the time when the information (Y3) was received from the electronic device 441 is stored as received time 462.

[0211] 25, information on the electronic device 451 is transmitted from the electronic device 451 to the server 410 via the electronic device 452. In this case, the electronic device 452 transmits the terminal identification information on the electronic device 451 to the server 410 together with the information (Z7) transmitted by the electronic device 451. Then, as shown in FIG. 29(1), the information (Z7) transmitted by the electronic device 451 is stored as received information 463 in association with the terminal identification information 461 “EX0451”, and information on the time when the information (Z7) was received from the electronic device 451 is stored as received time 462.

[0212] Since the information on the electronic device 441 is transmitted to the server 410 via the electronic device 442, the terminal identification information 461 may record the terminal identification information on the electronic device 442 instead of "EX0441". Also, instead of "EX0441", the terminal identification information on the electronic devices 441 and 442 (identification information on both the electronic devices 441 and 442, or information that the electronic device 442 has transferred information on the electronic device 441) may be recorded. The same applies to the information on the electronic device 451.

[0213] [Example of communication when the server 410 provides content to the electronic device 431] 31 is a flowchart showing an example of communication processing by the electronic device 431 and the server 410 in the fifth embodiment. The server 410 transmits information to the electronic device 431 based on the terminal identification information 461 (shown in FIG. 29(1)).

[0214] First, the control unit 412 of the server 410 determines whether or not the transmission condition is satisfied (step S161). If the transmission condition is not satisfied, the control unit 412 waits until the transmission condition is satisfied (step S161).

[0215] Here, the transmission condition may be, for example, when the content is provided on a daily basis, that it is 12:00 PM (24:00) at a certain point in the world. For example, when the server 410 is installed in Japan, it may be determined that the transmission condition is met when it is 12:00 PM (24:00) Japan time. Also, when the content is provided only within a limited area, it may be determined that the end of the activity time in this area (for example, 9:00 PM) is reached. However, it is assumed that some users want to see a character related to the coffee immediately after brewing the coffee. Therefore, for example, even when the content is provided on a daily basis, it may be determined that the transmission condition is met at short intervals (for example, 1-minute intervals, 10-minute intervals, or 1-hour intervals), and the content may be provided. In this way, a user who brews coffee in the morning can see the character that morning.

[0216] When the transmission condition is satisfied (step S161), the control unit 412 of the server 410 determines the content to be provided to each electronic device based on each piece of information stored in the management information (step S162). Specifically, the control unit 412 of the server 410 determines the identification information (e.g., a serial number) to be assigned to each electronic device based on the information stored in the received information 463, and stores the result in the determination result information 464. Then, the control unit 412 of the server 410 determines the content to be provided to each electronic device based on the information stored in the determination result information 464, and stores information for identifying the content in the provided content information 465.

[0217] For example, when it is determined that the transmission condition is satisfied at short intervals (e.g., 1-minute intervals, 10-minute intervals, or 1-hour intervals), the identification information (e.g., serial numbers) is ranked for each electronic device that transmitted information during that time. For example, assume that it is determined that the transmission condition is satisfied at 10-minute intervals. In this case, assume that there are three electronic devices that transmitted information in the first 10 minutes, five electronic devices that transmitted information in the next 10 minutes, and four electronic devices that transmitted information in the next 10 minutes. In this case, identification information (e.g., serial numbers) of 1 to 3 is determined for each electronic device that transmitted information in the first 10 minutes, identification information (e.g., serial numbers) of 4 to 8 is determined for each electronic device that transmitted information in the next 10 minutes, and identification information (e.g., serial numbers) of 9 to 12 is determined for each electronic device that transmitted information in the next 10 minutes.

[0218] Next, the communication unit 411 of the server 410 transmits the content corresponding to the information stored in the provided content information 465 to each electronic device under the control of the control unit 412 (step S163). For example, as shown in FIG. 29, the content corresponding to the information "CT67" stored in the provided content information 465 (content related to "character ABC") is transmitted to the electronic device 431. Also, the content corresponding to the information "CT217" stored in the provided content information 465 (content related to "character EFG") is transmitted to the electronic device 442 (electronic device 441). Also, the content corresponding to the information "CT347" stored in the provided content information 465 (content related to "character HIJK") is transmitted to the electronic device 452 (electronic device 451).

[0219] The communication unit 489 of the electronic device 431 receives the content transmitted from the server 410 (step S164). Next, the control unit 483 of the electronic device 431 stores the received content in the storage unit 484 (step S165). Next, the control unit 483 of the electronic device 431 outputs the received content (step S166). An example of this output is shown in FIG.

[0220] [An example in which a server 410 provides content to electronic devices 431, 442, 443, and 452] FIG. 32 is a diagram showing an example in which a server 410 in the fifth embodiment provides content to electronic devices 431, 442, 443, and 452. In FIG.

[0221] The server 410 transmits contents to each electronic device based on the management information shown in FIG. 29(1). For example, as shown in FIG. 29(1), the server 410 transmits contents related to "characters ABC" to the electronic device 431 based on the provided content information 465 "CT67" associated with the terminal identification information 461 "EX0431". Also, the server 410 transmits contents related to "characters EFG" to the electronic device 441 (or electronic device 442) based on the provided content information 465 "CT217" associated with the terminal identification information 461 "EX0441". Also, the server 410 transmits contents related to "characters HIJK" to the electronic device 451 (or electronic device 452) based on the provided content information 465 "CT347" associated with the terminal identification information 461 "EX0441".

[0222] The electronic device that has received the content performs output related to the received content. For example, as shown in FIG. 32, the electronic device 431 that has received the content related to "character ABC" displays an image related to "character ABC" on the user interface 488 (display unit 487) (shown in FIG. 26). The electronic device 431 also outputs a sound related to "character ABC" from the sound output unit 485 (shown in FIG. 26). The electronic device 442 that has received the content related to "character EFG" transmits the content to the electronic device 443 (having a display unit) and displays an image related to "character EFG" on the display unit of the electronic device 443. The electronic device 442 also outputs a sound related to "character EFG" from the sound output unit. The electronic device 452 that has received the content related to "character HIJK" displays an image related to "character HIJK" on the display unit. The electronic device 452 also outputs a sound related to "character HIJK" from the sound output unit.

[0223] [Example of sending acquired information to a server] Fig. 33 is a flowchart showing an example of a transmission process by electronic device 431 in the fifth embodiment. Steps S171 to S177 shown in Fig. 33 correspond to steps S151 to S157 shown in Fig. 30. Therefore, the following description will focus on the points that are different from the example shown in Fig. 30.

[0224] First, the control unit 483 of the electronic device 431 judges whether or not a start instruction is detected (step S171). If a start instruction is detected (step S171), the process proceeds to step S172. If a start instruction is not detected (step S171), the process continues to monitor.

[0225] [Example of outputting content provided by the server] Fig. 34 is a flowchart showing an example of content output processing by electronic device 431 in the fifth embodiment. Steps S181 to S183 shown in Fig. 34 correspond to steps S164 to S166 shown in Fig. 31. Therefore, the following description will focus on the points that are different from the example shown in Fig. 31.

[0226] First, the control unit 483 of the electronic device 431 judges whether or not content has been received from the server 410 (step S181). If content has been received from the server 410 (step S181), the process proceeds to step S182. If content has not been received from the server 410 (step S181), the process continues to monitor.

[0227] [Example of providing content to each electronic device] Fig. 35 is a flowchart showing an example of a content providing process by server 410 in the fifth embodiment. Steps S193 to S197 shown in Fig. 35 correspond to steps S158, S159, and S161 to S163 shown in Fig. 30 and Fig. 31. Therefore, the following description will focus on the points that differ from the example shown in Fig. 30 and Fig. 31.

[0228] First, control unit 412 of server 410 determines whether or not a start condition is satisfied (step S191). If the start condition is satisfied (step S191), the process proceeds to step S192. If the start condition is not satisfied (step S191), the process proceeds to step S193.

[0229] Here, the start condition may be, for example, when the content is provided on a daily basis, that it is midnight at a certain point in the world. For example, when the server 410 is installed in Japan, it may be determined that the start condition is met when it is midnight Japan time. Also, when the content is provided only within a limited area, the start time of the activity period in that area (for example, 4 a.m.) may be the condition.

[0230] When the start condition is satisfied (step S191), the control unit 412 of the server 410 erases the contents of the management information in the storage unit 413 (step S192). This makes it possible to provide new content (characters) to each electronic device based on new information.

[0231] Next, the control unit 412 of the server 410 determines whether or not information (e.g., information to be output that has been determined by the determination process (e.g., information OI (OIX1) to be output shown in FIG. 9(2))) has been received from the electronic device (step S193). If information has been received from the electronic device (step S193), the process proceeds to step S194. If information has not been received from the electronic device (step S193), the process continues to monitor.

[0232] When information is received from an electronic device (step S193), control unit 412 of server 410 stores the received information in storage unit 413 in association with the electronic device that transmitted the information (step S194).

[0233] Next, control unit 412 of server 410 determines whether or not the transmission condition is satisfied (step S195). If the transmission condition is not satisfied, the process returns to step S193.

[0234] If the transmission conditions are met (step S195), control unit 412 of server 410 determines the content to be provided to each electronic device based on each piece of information stored in the management information (step S196).

[0235] Next, the communication unit 411 of the server 410 transmits the content corresponding to the information stored in the provided content information 465 to each electronic device under the control of the control unit 412 (step S197).

[0236] [Example of operation when the server performs each process] In the above, an example has been shown in which an electronic device attached to a coffee extractor performs a determination process for determining information to be output, and transmits the information determined by this determination process to server 410. Here, as shown in the fourth embodiment, it is also possible to have the electronic device attached to the coffee extractor perform only relatively simple processing, and have the server 410, which has high processing power, perform other processing. Thus, Fig. 36 shows an example in which the electronic device attached to the coffee extractor only acquires and transmits image information of the surface of the coffee collected in the container, and other processing is performed by server 410.

[0237] [Example of communication when performing a series of processes among multiple electronic devices] Fig. 36 is a flowchart showing an example of communication processing by server 410 and electronic device 431 in the fifth embodiment. Steps S501 to S504 shown in Fig. 36 correspond to steps S131, S133 to S135 shown in Fig. 24. Steps S515 and S516 shown in Fig. 36 correspond to steps S161 and S162 shown in Fig. 31. Therefore, here, the points different from the examples shown in Figs. 24 and 31 will be mainly described.

[0238] When the control unit 483 of the electronic device 431 detects a start instruction (step S501), it acquires an image (an image (including image data) of coffee accumulating in the container) acquired by the image acquisition unit 482 (step S502), and transmits the acquired image information to the server 410 (step S503). In this case, the transmission of the image information from the electronic device 431 to the server 410 may be performed periodically or irregularly.

[0239] Step S504 shown in FIG. 36 corresponds to step S135 shown in FIG.

[0240] When the control unit 412 of the server 410 receives image information from the electronic device 431 (step S511), it analyzes the received image information and acquires a feature amount (step S512). Next, the control unit 412 of the server 410 associates the acquired feature amount with the electronic device 431 and stores it in the storage unit 413 (step S513). In this manner, until the end condition is satisfied, the analysis process is performed on the received image information, and the feature amounts acquired by this analysis process (for example, each piece of information shown in FIG. 9(1) and each piece of information shown in FIG. 16(1)) are associated with the electronic device 431 and sequentially stored in the storage unit 413 (steps S511 to S514). For example, each of the processes shown in the first to fourth embodiments (for example, each of the processes of steps S102 to S105 shown in FIG. 11, each of the processes of steps S112 to S115 shown in FIG. 18, each of the processes of steps S123 to S126 shown in FIG. 22, steps S143 to S146 shown in FIG. 24, steps S153 to S156 shown in FIG. 30, and steps S173 to S176 shown in FIG. 33) is performed, and the processing results (feature amounts) are associated with the elapsed time and stored in the memory unit 413 in association with the electronic device 431 (for example, each of the information shown in FIG. 9(1) and each of the information shown in FIG. 16(1)).

[0241] Furthermore, if the transmission condition is satisfied (step S515), control unit 412 of server 410 determines the content to be provided to each electronic device based on each piece of information stored in the management information (step S516).

[0242] Next, the communication unit 411 of the server 410 transmits a notification to the electronic device 431 that the content is available under the control of the control unit 412 (step S517). For example, a notification means such as an email, a social networking service (SNS), or a notification on the Internet can be used. In addition, in step S517, all or a part of the content may be transmitted (for example, by attaching it to an email, transmitting it on an SNS, or transmitting it on the Internet).

[0243] Furthermore, when the communication unit 489 of the electronic device 431 receives the notification from the server 410, it notifies the user of the notification (step S505). For example, the notification can be made by audio output of a message to that effect, display of a message to that effect, vibration, lighting of a lamp, blinking, or the like. The user who has seen this notification can operate the electronic device 431 (or another electronic device) to access the server 410, connect the electronic device 431 (or another electronic device) and the server 410, and receive the content provided from the server 410 (steps S518, S506). As a result, the control unit 483 of the electronic device 431 can output audio from the audio output unit 485 based on the provided content, and can display an image on the user interface 488 (display unit 487) based on the provided content (step S506). For example, as shown in FIG. 32, audio is output from the audio output unit and an image is displayed on the user interface.

[0244] 36, it is assumed that some cause (e.g., fogging of the lens, interruption of wireless communication) may prevent all image information from being received from electronic device 431. It is also assumed that information with different extraction times may be transmitted from a plurality of electronic devices. It is also assumed that image information transmitted from a plurality of electronic devices may not be transmitted based on a unified rule.

[0245] For example, it is assumed that image information on five images is transmitted from the electronic device 431, image information on three images is transmitted from the electronic device 441, and image information on one image is transmitted from the electronic device 451. In such a case, the control unit 412 of the server 410 analyzes each piece of image information received from the electronic devices 431, 441, and 451, and makes the feature amount based on each piece of image information comparable information. For example, when the determination information shown in FIG. 6(1) and (2) is used, the control unit 412 of the server 410 analyzes each piece of image information received from the electronic devices 431, 441, and 451, and obtains the feature amount (A to I) based on each piece of image information. Then, the control unit 412 of the server 410 calculates a comparable numerical value based on the obtained feature amount (A to I) and the number of images received. For example, assume that the feature amounts based on five images received from electronic device 431 are A, A, B, C, and A in the order of reception, the feature amounts based on three images received from electronic device 441 are A, C, and D in the order of reception, and the feature amount based on one image received from electronic device 451 is G. In this case, a multiplier (5 to 1) is determined based on the maximum number of receptions (5), and each feature amount is multiplied by this to calculate a comparable numerical value. Specifically, the feature amounts A to I are quantified, and the feature amounts A, A, B, C, and A related to electronic device 431 are converted to 1, 1, 2, 3, and 1, the feature amounts A, C, and D related to electronic device 441 are converted to 1, 3, and 4, and the feature amount G related to electronic device 451 is converted to 7. Then, the feature amounts are multiplied by the multipliers (5 to 1) in the order of reception and added to calculate a comparable numerical value. Specifically, for the electronic device 431, 22 (=1×5+1×4+2×3+3×2+1×1) is calculated, for the electronic device 441, 29 (=1×5+3×4+4×3) is calculated, and for the electronic device 451, 35 (=7×5) is calculated. In this case, since 22 (electronic device 431)<29 (electronic device 441)<35 (electronic device 451), the identification numbers (serial numbers) are determined in this order. That is, 1 is determined for the electronic device 431, 2 is determined for the electronic device 441, and 3 is determined for the electronic device 451. It is also assumed that the values ​​calculated as comparable numerical values ​​are the same for multiple electronic devices. In this case, for example, the identification numbers (serial numbers) can be determined based on the reception time.For example, numbers can be assigned in order from the earliest reception time to the electronic device.

[0246] In this way, even if image information transmitted from a plurality of electronic devices is not transmitted based on a unified rule, it is possible to determine identification information (e.g., serial number) to be assigned to each electronic device, and to provide content to each electronic device. This makes it possible to provide content to a user who, for example, photographs the surface of coffee with an electronic device (e.g., a smartphone) in a coffee shop and transmits the image to the server 410. That is, each user, such as a user who drinks coffee brewed with a coffee extractor at home or at work, a user who drinks coffee brewed with instant coffee, and a user who drinks coffee brewed by someone else in a coffee shop, can participate in a service in which at least one content is provided from among a plurality of contents, and enjoy the provided content. In this way, in the fifth embodiment, content can be determined and provided by comparing relative information.

[0247] It is also assumed that image information transmitted from a plurality of electronic devices will be the same (or the same pattern) or similar (approximate) (or similar (approximate) patterns). In such a case, some kind of privilege or preferential treatment can be given to each electronic device that transmitted image information of the same (or the same pattern) or similar (approximate) (or similar (approximate) pattern) in addition to the content determined by each process described above. For example, when providing content on a daily basis, the control unit 412 of the server 410 can determine whether or not there is an electronic device that transmitted image information of the same (or the same pattern) or similar (or similar pattern) at 12:00 p.m. (24:00) at the reference point, based on each piece of information stored in the management information. Then, when there is an electronic device that transmitted image information of the same (or the same pattern) or similar (or similar pattern), the control unit 412 of the server 410 can provide other content (for example, a special character, a character of the main character of an anime, a specific service) in addition to the determined content to each electronic device. This allows a user who has received regular content to receive other content (some kind of special offer or privilege) in addition to the regular content, thereby increasing the enjoyment of the content.

[0248] 36 shows an example in which the server 410 performs a series of processes based on image information from the electronic device 431. However, a part of the processes performed by the server 410 may be performed by another device (one or more devices). In this case, an information processing system made up of multiple devices performs each process shown in FIG. 36 (each process performed by the server 410).

[0249] In this way, server 410 (or multiple devices that constitute it) is an example of an information processing device, information processing system, or communication system that acquires information about the generated coffee (information acquired from the coffee) for each piece of coffee placed in multiple containers when the coffee is generated from substances for making coffee (e.g., coffee beans, ground coffee powder, instant coffee) and hot water and is stored in a container, when the generated coffee is being made, when the generated coffee is being placed in the container, or after the generated coffee is placed in the container, and determines content (e.g., information unrelated to the coffee) to be provided to each person associated with the multiple containers based on each piece of acquired information.

[0250] Furthermore, server 410 (or multiple devices that constitute it) is an example of an information processing device, information processing system, or communication system that acquires information about the extracted coffee when the extracted coffee is poured into a container or after the extracted coffee is poured into a container, for each coffee poured into multiple containers, and determines content to be provided to each person associated with the multiple containers based on each piece of acquired information.

[0251] Furthermore, the server 410 (or a plurality of devices constituting the server 410) can use, as information about the extracted coffee, for example, feature quantities related to changes in the surface of the extracted coffee in the container (for example, color, pattern, generation of foam) when the extracted coffee is poured into the container or after the extracted coffee is poured into the container. Furthermore, the server 410 (or a plurality of devices constituting the server 410) can determine content to be assigned to each person associated with the plurality of containers, for example, by comparing each piece of information acquired about each coffee poured into the plurality of containers.

[0252] [Example using other sensors] In the above, an example has been shown in which an image sensor (image sensor, image pickup element) is mainly used to obtain various information (e.g., surface transition) when coffee is made or after coffee is made, but other sensors may be used to obtain various information when coffee is made or after coffee is made. Fig. 37 shows an example in which various sensors are attached to a coffee extractor (electronic device).

[0253] Fig. 37 is a diagram showing an example of mounting a sensor attached to a coffee extractor or electronic device in the first to fifth embodiments. Note that the coffee extractors 600, 610, 620, 630, 640 shown in Fig. 37(1) to (5) have stands 602, 612, 622, 632, 642, substantially similar to the coffee extractor 200 shown in Fig. 13. Also, the coffee extractor 650 shown in Fig. 37(6) does not have a stand, substantially similar to the coffee extractor 1 shown in Fig. 1. Also, the coffee extractors 600, 610, 620, 630, 640, 650 have drippers 601, 611, 621, 631, 641, 651 and containers 603, 613, 623, 633, 643, 652. The sensors 604, 614, 624, 634, 644, and 661 will be explained in the following explanation of each sensor. Sensor 634, which will be immersed in the liquid (coffee), and sensor 624, which may be exposed to steam, are preferably waterproofed. Sensors that are affected by the temperature of hot water are preferably insulated and heat-resistant.

[0254] For example, a temperature sensor capable of measuring temperature can be used. For example, a temperature sensor is attached to a position where the temperature of the liquid (coffee) accumulated in a container of the coffee extractor can be measured, and the temperature transition from the start to the end of coffee extraction can be obtained by this temperature sensor. For example, temperature sensors can be attached to the positions of sensors 604, 614, 624, 634, and 644 shown in Fig. 37(1) to (5). Then, the content to be output can be determined based on the temperature transition.

[0255] Also, for example, an odor sensor (odor sensor) capable of measuring odors can be used. This odor sensor may be, for example, a sensor capable of detecting a specific odor, or a sensor capable of outputting some information according to the odor. For example, an odor sensor is attached to a position in a coffee extractor where the odor of the liquid (coffee) stored in a container can be measured, and the odor transition from the start to the end of coffee extraction can be obtained by this odor sensor. The odor sensor can be attached, for example, inside the container or below the dripper (so as to enter the inside of the container). For example, the odor sensor can be attached to the positions of the sensors 624 and 634 shown in Fig. 37(3) and (4). Then, the content to be output can be determined based on the odor transition. For example, information on each odor acquired in chronological order is quantified and calculated (for example, added or multiplied) to calculate a comparable numerical value, and the numerical value is compared to determine the content to be output.

[0256] Also, for example, a sound sensor (for example, a microphone) capable of measuring sound can be used. For example, a sound sensor is attached to a position where sound generated from liquid (coffee) stored in a container in a coffee extractor can be measured, and the transition of sound generated from the start to the end of coffee extraction can be acquired by this sound sensor. The position where the sound sensor is attached can be, for example, inside the container, the lower part of the dripper (so as to enter the inside of the container), or outside the container. For example, the sound sensor can be attached to the positions of each sensor 604, 614, 624, 634, and 644 shown in Fig. 37(1) to (5). Then, based on the transition of the sound, the content to be output can be determined. For example, information on each sound acquired in time series is quantified and calculated (for example, added or multiplied) to calculate a comparable numerical value, and the numerical value is compared to determine the content to be output.

[0257] Also, for example, a weight sensor capable of measuring weight can be used. For example, a weight sensor can be attached to a position (for example, the bottom of the container) in the coffee extractor where the weight of the liquid (coffee) accumulated in the container can be measured, and the weight sensor can acquire the transition of weight from the start to the end of coffee extraction. The position where the weight sensor is attached can be, for example, the bottom of the container. For example, the weight sensor can be attached to the position of sensor 644 shown in FIG. 37(5). Then, the content to be output can be determined based on the transition of weight. For example, the weight values ​​acquired in chronological order can be calculated (for example, added or multiplied) to calculate a comparable value, and the content to be output can be determined by comparing the values.

[0258] Also, for example, a taste sensor capable of measuring taste can be used. For example, a taste sensor can be attached to a position where the taste of the liquid (coffee) stored in a container in a coffee extractor can be measured, and the taste sensor can acquire the transition of taste from the start to the end of coffee extraction. The position where the taste sensor is attached can be, for example, inside the container or the lower part of the dripper (so as to enter the inside of the container). For example, the taste sensor can be attached to the position of sensor 634 (for example, a sensor attached to the opening of container 633 in the form of a clip) shown in FIG. 37(4). Then, the content to be output can be determined based on the transition of taste. For example, information on each taste acquired in chronological order is quantified and calculated (for example, added or multiplied) to calculate a comparable numerical value, and the content to be output can be determined by comparing the numerical values.

[0259] The above-mentioned sensors are merely examples, and sensors that obtain other information may be used. The sensors used may have a performance capable of obtaining some information (for example, whether the temperature is 75 degrees or higher, whether there is a specific smell, whether there is a specific sound, whether there is a specific taste). In other words, a sensor capable of obtaining information that can be compared for each extraction may be used. As shown in FIG. 37(6), a sensor 661 provided in an electronic device 660 installed outside the coffee extractor 650 may be used. For example, the electronic device 660 (for example, a smartphone) may be installed outside the container 652 of the coffee extractor 650 so as to obtain image information related to the liquid (coffee) that accumulates in the container 652 of the coffee extractor 650. The sensor 661 (for example, a camera) of the electronic device 660 may be used to obtain image information related to the liquid (coffee) that accumulates in the container 652 of the coffee extractor 650, and this image information may be used. In this case, the electronic device 660 functions as the electronic device (electronic device attached to the coffee extractor) shown in the first to fifth embodiments. Also, a mounting section (e.g., a smartphone mounting section) for mounting the electronic device 660 may be provided on a stand (mounting stand 653) on which the container 652 is mounted, and the electronic device 660 may be mounted on this mounting section. Also, using other sensors (e.g., a sensor for detecting vibration, a microphone) of the electronic device 660 (e.g., a smartphone), various pieces of information (e.g., vibrations generated by the falling liquid, sounds generated by the falling liquid) related to the liquid (coffee) stored in the container 652 of the coffee extractor 650 may be acquired, and this information may be used. This allows anyone to easily install the electronic device 660 and enjoy the output of the contents shown in the first to fifth embodiments.

[0260] [Example of using multiple sensors] In the above, an example has been shown in which various sensors are used to obtain various pieces of information when coffee is made or after coffee is made, but multiple sensors out of the various sensors may be used to obtain various pieces of information when coffee is made or after coffee is made. Fig. 38 shows an example in which multiple sensors are attached to an electronic device.

[0261] FIG. 38 is a diagram showing an example of attaching sensors to electronic devices 700, 710, 720, and 725 in the first to fifth embodiments. The electronic devices 700, 710, 720, and 725 shown in FIG. 38(1) to (4) are modified versions of the electronic device 20 shown in FIG. 1 to FIG. 3. The electronic device 700 shown in FIG. 38(1) is an electronic device in which the shape of the gripping part 27 of the electronic device 20 is modified. The electronic devices 710 and 720 shown in FIG. 38(2) and (3) are an electronic device in which the shape of the annular part 28 of the electronic device 20 is modified. The electronic device 725 shown in FIG. 38(4) is an electronic device in which the gripping part 27 of the electronic device 20 is omitted and the shape of the annular part 28 is modified. The bottom views of the electronic devices 700, 710, 720, and 725 are shown in FIG. 38(1) to (4) as in FIG. 3(4).

[0262] The electronic device 700 shown in Fig. 38(1) is provided with four sensors 701-704 instead of the lens 21 of the electronic device 20 shown in Fig. 3(4). Also, a recess 705 (dent) is provided in the gripping portion so that the user can easily hold the electronic device 700.

[0263] An electronic device 710 shown in FIG. 38(2) is configured such that the shape of the annular portion 28 of the electronic device 20 shown in FIG. 3(4) is U-shaped, and three sensors 711 to 713 are provided.

[0264] An electronic device 720 shown in FIG. 38(3) is an electronic device 20 shown in FIG. 3(4) in which the annular portion 28 is rectangular in shape, and four sensors 721 to 724 are provided.

[0265] An electronic device 725 shown in Fig. 38(4) is an electronic device in which the shape of the annular portion 28 of the electronic device 20 shown in Fig. 3(4) is elliptical, and four sensors 726 to 729 are provided. Also, the grip portion 27 of the electronic device 20 is omitted. Note that each part (e.g., the operation unit 26, the audio output unit 25) provided on the grip portion 27 of the electronic device 20 can be provided on other parts (e.g., the side surface, the part protruding from the flange portion (e.g., the part indicated by reference numeral 706)).

[0266] In addition, the sensors 701-704 shown in FIG. 38(1) may be provided with a plurality of sensors of the same type, or all may be different sensors. For example, the sensors 701-704 may be at least one of the above-mentioned sensors. Also, a plurality of sensors of the same type (for example, image sensors) may be provided to increase the accuracy of the sensors. The same applies to the sensors 711-713 shown in FIG. 38(2), the sensors 721-724 shown in FIG. 38(3), and the sensors 726-729 shown in FIG. 38(4). Also, the position where the sensor is provided is not limited to the example shown in FIG. 38, and the sensor may be provided at a position where the sensor can exhibit its performance. Also, the electronic device attached to the coffee extractor may omit components such as a control unit other than the sensor. In this case, information from the sensor can be transmitted to another electronic device via a wired line or the like and used.

[0267] [Modification of coffee extractor] 39 and 40 are diagrams showing the appearance of a modified example (coffee extractor 730) of coffee extractor 1 in the first embodiment. FIG. 39 shows a perspective view of coffee extractor 1. FIG. 40(1) shows a top view of dripper 740 with electronic device 750 attached to flange portion 742, and FIG. 40(2) shows a bottom view of dripper 740 with electronic device 750 attached to flange portion 742. Note that, in FIG. 40(1), accommodating portion 741 is omitted for ease of explanation. FIG. 40(3) shows a bottom view of electronic device 750 removed from flange portion 742, and FIG. 40(4) shows a bottom view of electronic device 750 removed from flange portion 742.

[0268] Coffee extractor 730 has dripper 740, electronic device 750, and container 760. Note that dripper 740, electronic device 750, and container 760 correspond to dripper 10, electronic device 20, and container 30 shown in Figures 1 to 3, respectively. Therefore, the following description will focus on the differences from dripper 10, electronic device 20, and container 30.

[0269] Dripper 740 is a coffee dripper that has a container portion 741 and a flange portion 742, and is used by being placed at the opening of a container 760. Electronic device 750 is attached to flange portion 742.

[0270] Electronic device 750 is an electronic device that is removably attached to flange portion 742, and includes audio output portion 755, lens 751, and attachment portion 757. Electronic device 750 is attached by sliding into a recess (a recess corresponding to the shape of the protrusion of attachment portion 757) of attachment portion 745 of flange portion 742. Electronic device 750 is realized by incorporating each device (for example, each portion shown in FIG. 4) in a housing (case) made of heat-resistant glass, pottery, metal, ceramic, resin, wood, or the like.

[0271] The mounting portion 757 is a portion for mounting to a notch portion (mounting portion 745) provided in a part of the flange portion 742. The mounting portion 757 is a convex portion corresponding to the shape of the concave portion of the mounting portion 745, and is formed so as to fit into the concave portion of the mounting portion 745. Note that, contrary to the example shown in FIG. 40, the mounting portion 757 may be formed to have a concave shape, the mounting portion 745 of the flange portion 742 may be formed to have a convex shape (a convex portion corresponding to the shape of the concave portion of the mounting portion 757), and the mounting portion 757 may be formed to fit into the convex portion of the mounting portion 745. Also, in this example, an example is shown in which a part of the flange portion 742 is made into a notch portion (mounting portion 745) and electronic device 750 is attached to the mounting portion 745, but other mounting methods may be used.

[0272] For example, when attaching the electronic device 750 to the flange portion 742, the convex portion of the attachment portion 757 is aligned with the concave portion of the attachment portion 745 of the flange portion 742, and the electronic device 750 is pushed and slid in the direction of the arrow 2 (FIG. 40(3)). When removing the electronic device 750 from the flange portion 742, the electronic device 750 is slid so as to be pulled toward the outside of the flange portion 742 (the direction of the arrow 3 (FIG. 40(3))). Note that a locking member for locking the electronic device 750 to the flange portion 742 may be provided on at least one of the attachment portion 745 of the flange portion 742 and the attachment portion 757 of the electronic device 750.

[0273] In this way, by attaching the electronic device 750 in a sliding manner, the electronic device 750 can be easily attached and detached.

[0274] Furthermore, lens 751 of electronic device 750 is disposed on the lower surface of flange portion 742. Furthermore, lens 751 of electronic device 750 is disposed such that the optical axis direction of lens 751 faces the surface of the coffee extract accumulated in container 760 when dripper 740 is set in container 760.

[0275] Moreover, it is preferable to perform heat insulation and waterproofing treatment on each device built into electronic device 750. Moreover, it is preferable to perform anti-fogging treatment on lens 751.

[0276] In this way, various methods can be used to attach the electronic device to the coffee brewer.

[0277] [Example of obtaining and using other information about the liquid (coffee)] In the above, an example of acquiring and using information about the liquid (coffee) stored in the container of the coffee extractor has been shown. However, other information about the liquid (coffee) may be acquired and used. For example, when liquid (hot water) is poured into a dripper that contains coffee powder, the coffee powder and liquid (hot water) in the container of the dripper change depending on how the powder is poured. For example, when the coffee beans are finely crushed, the transition may be different (for example, a depression may appear in the middle or a mountain may form). Also, depending on the speed at which the liquid (hot water) is poured, the transition may be different (for example, the liquid may be filled with hot water, or hot water may exist only in the depression in the powder). For example, depending on how the powder is poured and how crushed the powder is, the coffee powder in the container of the dripper may be filled with the liquid (hot water), and then bubbles may occur. For example, depending on how the powder is crushed, the coffee powder in the container of the dripper may swell or become flat. Therefore, when or after coffee is made, various sensors may be used to obtain various information related to the coffee powder and liquid (hot water) in the container of the dripper that contains the coffee powder. For example, as shown in Fig. 37(3), an image sensor (image sensor, image sensor) may be installed above the container of the dripper to obtain and use image information related to the coffee powder and liquid (hot water) in the container of the dripper.

[0278] As other information related to the liquid (coffee), for example, information related to grinding coffee beans using a coffee mill (a device for grinding coffee beans) may be acquired and used. For example, information related to the sound generated when grinding coffee beans using a coffee mill may be acquired and used. Also, the information acquired about the coffee mill and the information acquired from the coffee extractor may be used in combination.

[0279] [Example of using time and location information along with information about the liquid (coffee)] In the above, an example has been shown in which various information related to the liquid (coffee) is acquired and the content to be provided is determined using the acquired information. Also, an example has been shown in which the content to be provided is determined based on image information and time information related to the liquid (coffee). However, if the electronic device can use location information (for example, if the electronic device has the location information acquisition unit 358 shown in FIG. 21 or the location information acquisition unit 490 shown in FIG. 26), the location information can be used. For example, an electronic device (an electronic device attached to a coffee extractor, an electronic device that transfers information from an electronic device attached to a coffee extractor) can transmit location information related to the device itself (for example, latitude and longitude, information for identifying a region) to the server 410 together with information related to the liquid (coffee). In this case, the control unit 412 of the server 410 can determine the content to be provided based on information related to the liquid (coffee) (for example, image information, sound information, etc.) and the location information. For example, a plurality of contents are classified into a plurality of groups (for example, the number of countries providing the contents), and the groups assigned to each country are changed in sequence every day. For example, each electronic device present in a certain country A is provided with content selected from the content belonging to the group assigned to country A on that day.

[0280] Furthermore, the control unit 412 of the server 410 may determine the content to be provided based on the information on the liquid (coffee), as well as the location information and time information. For example, the control unit 412 of the server 410 may convert the information on the liquid (coffee) into a numerical value, convert both the location information and the time information into a numerical value, and perform an operation (e.g., four arithmetic operations) of these numerical values ​​to calculate a comparable numerical value and determine the content to be provided. The location information may be assigned a numerical value on a country basis, such as 1 for country A, 2 for country B, and so on, or a numerical value on a region (e.g., prefecture) basis within a country. Furthermore, a numerical value may be calculated by performing an operation (e.g., four arithmetic operations (e.g., simple addition of each numerical value)) on the latitude and longitude.

[0281] [Example of obtaining and using information about drinks other than coffee] In the above, an example of outputting contents using information acquired about the extracted coffee when the extracted coffee is put into a container or after the extracted coffee is put into a container has been shown. That is, an example of outputting contents using information acquired when the ground coffee is soaked in liquid until it becomes delicious has been shown. However, the first to fifth embodiments can also be applied to drinks other than coffee. For example, contents can be output using information acquired when a substance other than coffee beans is soaked in liquid until it becomes delicious. For example, the first to fifth embodiments can also be applied to various drinks such as black tea, Japanese tea (e.g., green tea, roasted green tea, brown rice tea, sencha, bancha, and gyokuro), Chinese tea (e.g., eucommia tea and oolong tea), herbal tea, medicinal tea, cocoa, and hot chocolate. For example, when tea is brewed in a tea utensil (e.g., a teapot or teapot), when the hot water poured onto the tea leaves is poured into the tea utensil, or after the hot water poured onto the tea leaves has entered the tea utensil, the content can be output using information acquired about the hot water (tea) poured onto the tea leaves. In this case, at least one of the above-mentioned sensors can be attached to the tea utensil or in its vicinity to acquire the above-mentioned information. The server 410 can also hold management information for each type (category) of drink, perform the above-mentioned processes for each type (category) of drink, and perform content provision processing for each type (category) of drink. In this case, for example, the same content may be provided to an electronic device that has transmitted information about coffee and an electronic device that has transmitted information about black tea. The server 410 can also perform the above-mentioned processes for the electronic device that has transmitted information about the drink, regardless of the type (category) of drink, and perform content provision processing. In this case, for example, the same judgment process, decision process, etc. can be performed for each electronic device that has transmitted information about different drinks, and different content can be provided.

[0282] [Sixth embodiment: Example of obtaining and outputting information about drinks] In the above, an example has been shown in which content is output using information acquired about the extracted drink when the extracted drink is placed in a container or after the extracted drink is placed in a container. However, it is also possible to output the information acquired about the extracted drink itself (or information that has been subjected to some processing (e.g., audio amplification processing)) when the extracted drink is placed in a container or after the extracted drink is placed in a container. Therefore, in the sixth embodiment, an example is shown in which the information acquired about the extracted drink itself is output when the extracted drink is placed in a container or after the extracted drink is placed in a container.

[0283] As described above, in the conventional technology, various devices can be used to brew beverages according to personal preferences. In addition, brewing beverages using various devices takes a certain amount of time. Therefore, when brewing (extracting) a beverage using various devices, if various information can be obtained at the time of brewing the beverage and this information can be provided to the person who brewed the beverage, it is considered that the enjoyment of brewing a beverage will increase. Therefore, the sixth embodiment aims to provide the enjoyment of brewing a beverage.

[0284] Fig. 41 is a simplified diagram showing an example of outputting information acquired by electronic device 700 in the sixth embodiment. Note that electronic device 700 corresponds to electronic device 700 shown in Fig. 38(1). However, electronic device 700 includes a communication unit (corresponding to communication unit 325 shown in Fig. 19).

[0285] Fig. 41(1) shows an example of a case where information acquired by electronic device 700 is output from electronic device 700. As shown in Fig. 38(1), electronic device 700 is provided with four sensors 701-704. The four sensors 701-704 are, for example, a sound sensor, an image sensor, a temperature sensor, and an odor sensor.

[0286] For example, the control unit of the electronic device 700 can output information acquired by each of the sensors 701 to 704 from the electronic device 700. For example, when the electronic device 700 includes an audio output unit, as shown in FIG. 41(1), the sound generated during coffee extraction acquired by the sound sensor can be amplified and output. This allows the user P1 brewing coffee using the coffee extractor 760 to brew coffee while listening to the sound generated during the brewing (for example, the amplified sound of water droplets of coffee falling into a container). After brewing the coffee, the user P1 can enjoy the content based on the feature amount acquired by the brewing while tasting the coffee.

[0287] Furthermore, for example, if electronic device 700 has a display unit, an image (the surface of the coffee in the container) acquired by the image sensor during coffee extraction can be displayed on the display unit. This allows user P1 brewing coffee using coffee extractor 760 to brew coffee while watching the transition during brewing (transition of the surface of the coffee in the container). After brewing the coffee, user P1 can enjoy the content based on the feature amount acquired by the extraction while tasting the coffee.

[0288] Furthermore, for example, when electronic device 700 is equipped with an audio output unit and a display unit, the sound generated during coffee extraction acquired by the sound sensor can be amplified and output, and an image during coffee extraction (the surface of the coffee in the container) acquired by the image sensor can be displayed on the display unit. The temperature value during coffee extraction acquired by the temperature sensor can be displayed and output as audio, and the smell during coffee extraction acquired by the smell sensor can be displayed and output as audio. This allows user P1 brewing coffee using coffee extractor 760 to further enjoy the coffee extraction. Similar outputs can also be performed when other sensors are used.

[0289] Fig. 41(2) shows an example of a case where information acquired by the electronic device 700 is output from another electronic device (electronic device 350). The example shown in Fig. 41(2) is similar to the example shown in Fig. 41(1) except that information acquired by the electronic device 700 is output from another electronic device (electronic device 350).

[0290] Fig. 41(3) shows an example of a case where information acquired by the electronic device 700 is output from another electronic device (electronic device 360). The example shown in Fig. 41(3) is similar to the example shown in Fig. 41(1) except that information acquired by the electronic device 700 is output from another electronic device (electronic device 360).

[0291] In the above, an example has been shown in which information relating to the extraction of coffee acquired by one or more sensors is subjected to a predetermined process (e.g., sound amplification, generation of a display image (video)) and then output. However, information relating to the extraction of coffee acquired by multiple sensors may be subjected to a predetermined process (e.g., sound amplification, generation of a display image (video)) and then output as a single piece of information. For example, the information acquired by each sensor and a musical scale such as do-re-mi-fa-so-la-ti-do may be converted into a numerical value (e.g., 0, 1, 2, ...), and the information acquired by each sensor may be subjected to a predetermined calculation process, and the values ​​calculated based on the calculation results may be sequentially assigned to the numerical values ​​of the musical scale to be output as sound. For example, for image information acquired by an image sensor, the proportion of black in a circular image is quantified (e.g., 0, 1, 2, ...), for audio information acquired by a sound sensor, the loudness of the sound is quantified (e.g., 0, 1, 2, ...), for temperature information acquired by a temperature sensor, the temperature is converted into a temperature value (or other value) (e.g., 0, 1, 2, ...), and for odor information acquired by an odor sensor, the strength of the odor is quantified (e.g., 0, 1, 2, ...). Next, a predetermined calculation (e.g., four arithmetic operations) is performed on each of the values ​​calculated for each of these sensors to calculate a value for conversion to a musical scale. Next, sound is output based on the calculated numerical value (musical scale). In addition, image output can also be performed based on a value calculated based on the calculation result by performing a predetermined calculation process on the information acquired by each sensor. Note that, as in the first to fifth embodiments, these audio and image outputs can also be understood as examples of outputting content based on information acquired by each sensor. In addition, these processes may be performed by other devices and output from other devices. For example, as shown in Figures 41(2) and (3), information regarding the extraction of coffee obtained by multiple sensors may be sequentially transmitted to electronic devices 350, 360, and each process may be performed by the electronic devices 350, 360, and each piece of information may be output by the electronic devices 350, 360.Furthermore, as shown in the fifth embodiment, information relating to the extraction of coffee obtained by a plurality of sensors may be sequentially transmitted to server 410 via network 420, each process may be performed by server 410, and output information based on the processes may be sequentially received from server 410 and output on the device itself (or another device).

[0292] [Seventh embodiment: Example of acquiring and using information about drinks containing other foods] In the above, an example of outputting content using information acquired about a drink has been shown. Here, other foods are often added to the generated drink. For example, milk, sugar, etc. are often added to coffee or tea depending on personal preference. Also, when other foods are added to a drink in this way, information about the drink (for example, its surface, smell) often changes. This example is shown in FIG. 42. Note that the seventh embodiment is an example in which the first to sixth embodiments are partially modified, and some of the explanations of the parts common to the first to sixth embodiments will be omitted.

[0293] [Example of the transition of the surface of coffee after milk is poured] Fig. 42 is a diagram showing an example of the transition of an image (an image of the surface of coffee when milk is poured into coffee in a coffee cup) that can be acquired by the electronic device in the seventh embodiment. Note that Fig. 42 shows an example of the surface of coffee in the coffee cup viewed from above. Also, in Fig. 42, for ease of explanation, the surface of the coffee in the coffee cup is simplified by a circle, and the surface of the coffee and milk, etc. are also simplified.

[0294] 42(1) shows a case where milk 801 is poured into coffee 800 in a coffee cup. For example, the example shows a case where a user pours milk 801 into a part of coffee 800 after stirring (rotating) coffee 800 in a coffee cup with a spoon or the like.

[0295] In this way, when milk 801 is poured while coffee 800 is rotating inside the coffee cup, as shown in Fig. 42(2), milk 801 flows along with the flow of coffee 800 inside the coffee cup. Then, as shown in Figs. 42(3) to (6), milk 801 spreads on the surface of coffee 800. Furthermore, as milk 801 spreads in coffee 800, coffee 800 and milk 801 mix together, and the color of the surface of coffee 800 changes from black to whitish.

[0296] In this way, when milk 801 is poured into coffee 800 in a coffee cup, various colors and patterns, such as the color of coffee (a dark color) and the color of milk (a whitish color), are formed on the surface of coffee 800 in the coffee cup, as shown in Fig. 42 (1) to (6). The colors and patterns thus formed often differ depending on the person pouring the milk and the environment in which the milk is poured. It is also assumed that the colors and patterns thus formed are unlikely to be the same or approximately the same even if the same person pours milk in the same environment. Thus, in the seventh embodiment, an example is shown in which various information is provided using the colors and patterns formed when milk 801 is poured into coffee 800 in a coffee cup.

[0297] [Example of judgment information] FIG. 43 is a diagram showing an example of various information used when determining information to be output by the electronic device in the seventh embodiment.

[0298] FIG. 43(1) shows an example of an image pattern relating to the surface of coffee when milk is poured into coffee in a coffee cup. As shown in FIG. 43(1), an image pattern that is assumed to appear on the surface of coffee when milk is poured into coffee in a coffee cup is stored as determination information in a memory unit of an electronic device. Note that in FIG. 43(1), the image patterns are shown in a simplified form for ease of explanation. Also, the number N of image patterns (where N is a positive integer) may be increased or decreased according to the storage capacity of the memory unit of the electronic device. Note that the image patterns shown in FIG. 43(1) correspond to the image patterns shown in FIG. 12(1).

[0299] Fig. 43(2) shows an example of judgment information (table) used when determining information to be output based on information acquired using the judgment information (image patterns PPA1 to PPAN) shown in Fig. 43(1). The judgment information shown in Fig. 43(2) is information showing the correspondence between the transition of information about coffee (for example, the surface of coffee when milk is poured into coffee in a coffee cup) and information OI to be output. Note that the judgment information shown in Fig. 43(2) corresponds to the judgment information shown in Fig. 12(2).

[0300] [Example of external configuration of installed equipment] 44 is a side view showing an example of an external configuration of an installation device 830 for installing an electronic device 810 according to the seventh embodiment. The electronic device 810 is an electronic device or information processing device such as a smartphone or a tablet terminal. The image acquisition unit 811 is a camera (e.g., a camera capable of acquiring moving images and still images) provided in the electronic device 810. The display unit 812 is a user interface (e.g., a touch panel capable of displaying and inputting operations) provided in the electronic device 810.

[0301] Installation device 830 is a device for installing coffee cup 820 at the bottom and electronic device 810 on the top of installation device 830 to acquire information about the surface of the coffee in coffee cup 820. That is, installation device 830 is a support stand for supporting electronic device 810, and can be formed of a flat member having a U-shape in side view. Also, for example, an upper portion of installation device 830 (a portion where electronic device 810 is installed) is provided with a hole having a shape (for example, a circular shape) larger than the diameter of the lens of image acquisition unit 811 of electronic device 810 and smaller than the housing of electronic device 810. This hole may be provided with a transparent member (for example, glass) for waterproofing or the like. Then, the user installs electronic device 810 so that the lens of image acquisition unit 811 of electronic device 810 faces the surface of the coffee in coffee cup 820 through the hole. The user can install electronic device 810 while looking at display unit 812 of electronic device 810, for example.

[0302] Further, the installed device 830 may be provided with a milk refilling machine 831. The milk refilling machine 831 includes a milk storage section 832, an operating member 833, and a milk supplying section 834. The milk storage section 832 stores milk to be poured into coffee in a coffee cup 820 installed in the installed device 830. The operating member 833 is an adjustment valve for adjusting the amount of milk supplied from the milk storage section 832 to the milk supplying section 834 through a tube. The milk supplying section 834 is a spout that supplies milk supplied from the milk storage section 832 through a tube to coffee in the coffee cup 820 installed in the installed device 830. It is preferable that the milk supplying section 834 is movable with respect to the coffee cup 820 installed in the installed device 830. Then, the user can pour the desired milk into the coffee in the coffee cup 820 by installing the coffee cup 820 containing coffee in the installed device 830 and operating the operating member 833. In this case, the milk that spreads on the surface of the coffee in the coffee cup 820 differs depending on the position of the coffee cup 820 installed in the installed device 830 (or the position of the spout of the milk supplying unit 834) and the amount of milk poured from the milk refilling machine 831 (determined by the user's operation of the operating member 833). In other words, the user can enjoy the transition of the milk spreading on the surface of the coffee in the coffee cup 820 by adjusting the position of the coffee cup 820 installed in the installed device 830 (or the position of the spout of the milk supplying unit 834) and the amount of milk poured from the milk refilling machine 831 (determined by the user's operation of the operating member 833). Note that, for example, heat-resistant glass, pottery, metal, ceramic, resin, wood, etc. can be used as the material of the installed device 830.

[0303] For example, the control unit of electronic device 810 determines a circular area (a circular area corresponding to the surface of coffee (liquid) in coffee cup 820) in the image acquired from image acquisition unit 811. Next, the control unit of electronic device 810 compares the image included in the determined circular area with each of image patterns PPA1 to PPAN in the determination information shown in Fig. 43(1), and extracts the image pattern with the highest degree of similarity as a feature.

[0304] For example, each pixel of each image pattern PPA1-PPAN represents two values ​​(black, white). The control unit of electronic device 810 determines each pixel in the image included in the determined circular region as either a black region or a white region (for example, binarization process). Next, the control unit of electronic device 810 compares corresponding pixels (pixels at the same position in the circular region) between each pixel (black, white) in the image included in the determined circular region and each pixel (black, white) in each image pattern PPA1-PPAN, and calculates the number of matching pixels for each image pattern PPA1-PPAN. Next, the control unit of electronic device 810 extracts image pattern PPA1-PPAN with the largest number of matching pixels as the image pattern with the highest degree of similarity.

[0305] Next, the control unit of the electronic device 810 stores the extracted image patterns PPA1-PPAN in association with the elapsed time in the storage unit of the electronic device 810. For example, the control unit of the electronic device 810 stores the feature amounts (image patterns PPA1-PPAN) determined for the images acquired at the timings of times t1-t8 in association with the times t1-t8 in the storage unit of the electronic device 810.

[0306] Next, the control unit of the electronic device 810 determines information to be output from the audio output unit based on the acquired feature amount. For example, the control unit of the electronic device 810 extracts a feature amount that matches the acquired feature amount (for example, information associating times t1 to t8 with image patterns PPA1 to PPAN) from the determination information (table) shown in Fig. 43(2). Next, the control unit of the electronic device 810 determines information (information to be output) OI corresponding to the extracted feature amount, and determines audio information stored in the storage unit of the electronic device 810 in association with the determined information (information to be output OI).

[0307] In this way, when acquiring information about the surface of coffee when milk is poured into coffee in a coffee cup, for example, electronic device 810 (for example, a device such as a smartphone or tablet terminal) equipped with image acquisition unit 811 can be used. For example, the user can use electronic device 810 to acquire information about the surface of the coffee onto which milk has been poured at the timing of pouring milk (before pouring, the moment of pouring, or after pouring). Note that while FIG. 44 shows an example in which image information is acquired with electronic device 810 fixed using installed device 830, image information may also be acquired with the user holding electronic device 810 in his or her hand.

[0308] In the above, an example using the determination information shown in FIG. 43 has been shown, but other determination information (for example, the determination information shown in the first to sixth embodiments) may be used. In the above, an example in which the electronic device 810 that has acquired image information outputs content has been shown, but as shown in the third and fourth embodiments, content may be output from other devices. In addition, as shown in the fifth embodiment, information from a plurality of electronic devices may be collected and content may be output from each device based on the collected result. In addition, as shown in the sixth embodiment, information itself acquired about the drink may be output. In the above, an example in which image information about a drink containing other food (for example, milk) is acquired and used has been shown, but information (for example, sound, temperature, smell) about the drink containing other food may be acquired and used by the above-mentioned other sensors (for example, sound sensor, temperature sensor, smell sensor). In addition, two or more of these sensors, or two or more types of sensors, may be used.

[0309] [Eighth embodiment: Example of outputting specific content or discount coupons from electronic device] As described above, it is possible to cooperate with a specific supplier (e.g., a company that provides specific content or discount coupons) and output the output target image. For example, a specific image transition (e.g., an image pattern (shown in FIG. 12(1) and FIG. 43(1))) is provided from the specific supplier, and when it matches, information provided by the specific supplier (e.g., a discount coupon, specific content) can be output from the electronic device. FIG. 45 shows an example of such output. Note that the eighth embodiment is an example in which the first to seventh embodiments are partially modified, and some of the explanations of the parts common to the first to seventh embodiments will be omitted.

[0310] 45 is a simplified diagram showing an example of outputting content from an electronic device 840 in the eighth embodiment. The electronic device 840 is an electronic device or information processing device such as a smartphone or a tablet terminal. The electronic device 840 may acquire information about the drink (e.g., image information) using a sensor provided in the electronic device, or may use information acquired by another device.

[0311] When entering a restaurant, a user often decides whether to enter or not, taking into consideration his / her own preferences, budget, the restaurant's menu, etc. Similarly, after entering a restaurant, a user often decides what to order from the menu, taking into consideration his / her own preferences, budget, the restaurant's menu, etc. Therefore, to suit the preferences of each user, content may be provided in multiple stages, as shown in FIG.

[0312] FIG. 45(1) shows an example in which the electronic device 840 outputs a discount coupon (contents (display screen and audio information)) related to store A.

[0313] 45(1), the control unit of electronic device 840 causes audio output unit 841 to output the audio "Welcome. Today's special offer is ready," based on audio information (not shown) determined based on information about the drink. Also, the control unit of electronic device 840 causes display unit 850 to display the text "Welcome. Today's special offer is ready," a "Special offer output" button 851, and a "Back" button 852, based on image information (not shown) determined based on information about the drink.

[0314] As shown in Fig. 45(1), the user can be notified that a special offer (content) can be output by audio information and image information. When the user understands this notification and presses the "Special Offer Output" button 851, the display screen shown in Fig. 45(2) is displayed. When the "Back" button 852 is pressed, the original screen is returned to. For example, the user can return to a standby screen or a screen for performing the special offer determination process again.

[0315] FIG. 45(2) shows an example of the contents (display screen and audio information) that are output after the "preferential treatment output" button 851 is pressed on the display screen shown in FIG. 45(1).

[0316] 45(2), the control unit of electronic device 850 causes audio output unit 841 to output the audio "Welcome. Whitebait rice bowls are half price today," based on audio information determined based on information related to drinks. Also, the control unit of electronic device 850 causes display unit 850 to display the text "Welcome. Whitebait rice bowls are half price today," an image related to Store A, a menu for Store A (an image showing whitebait), a "More Special Offers" button 853, and an "Exit" button 854, based on image information determined based on information related to drinks.

[0317] After the content shown in Fig. 45(2) is output, when the user presses the "More Special Treatment" button 853, the display screen shown in Fig. 45(3) is displayed. When the "Exit" button 854 is pressed, the content output process ends and the display returns to the normal screen.

[0318] FIG. 45(3) shows an example of the contents (display screen and audio information) that are output after the "More Special Treatment" button 853 is pressed on the display screen shown in FIG. 45(2).

[0319] As shown in FIG. 45(3), the control unit of the electronic device 850 causes the voice output unit 841 to output the voice "Here is a discount coupon for the specialty of the A store" based on the voice information determined based on the information about the drink. The control unit of the electronic device 840 also causes the display unit 850 to display the text "Here is a discount coupon for the specialty of the A store," a "30% discount off pickled celery" button 855, a "30% discount off shrimp curry" button 856, a "save" button 857, and an "exit" button 858 based on the image information determined based on the information about the drink. The "30% discount off pickled celery" button 855 is a button for starting an operation to receive a special offer (30% discount off) on the food and drink "pickled celery" provided by the A store. The "30% discount off shrimp curry" button 856 is a button for starting an operation to receive a special offer (30% discount off) on the food and drink "shrimp curry" provided by the A store.

[0320] For example, when at least one of the "30% off pickled celery" button 855 and the "30% off shrimp curry" button 856 is pressed, the pressed button is selected (for example, the color of the button changes). Then, when the "save" button 857 is pressed in the selected state, a process for receiving a special offer for the food or drink corresponding to the selected button is started. For example, an image showing a 30% off pickled celery or an image showing a 30% off shrimp curry is saved. Then, the user can receive each special offer by going to the store A and showing the saved image. Note that the buttons displayed after pressing the "more special offer" button 853 (for example, the "30% off pickled celery" button 855, the "30% off shrimp curry" button 856) may be related to the content displayed together with the "more special offer" button 853 (for example, content related to whitebait rice bowl) (for example, food to eat with whitebait rice bowl), or may be unrelated to the content (for example, today's recommended item). Information for displaying buttons that are displayed after pressing the "More Special Offers" button 853 (for example, "30% off pickled celery" button 855, "30% off shrimp curry" button 856) is also stored as provided information.

[0321] In this way, when the user enters Store A, the user can receive content related to Store A. In this case, the information provided may change depending on the coffee the user makes, so the user can enjoy the output of unexpected content.

[0322] Although Fig. 45 shows an example of outputting content in three stages, content may be output in one or two stages, or content may be output in four or more stages. Also, Fig. 45(3) shows an example in which there are two buttons to be selected, but the number of buttons to be selected is not limited to this, and may be one, or three or more.

[0323] [Ninth embodiment: Application to games] By applying the above-mentioned technology, for example, when playing a game, it is conceivable to provide various kinds of information related to the game. Therefore, in the ninth embodiment, an example is shown in which various kinds of information related to the game is provided when playing the game. Note that the ninth embodiment is an example in which the first to eighth embodiments are partially modified, and some of the explanations of the parts common to the first to eighth embodiments will be omitted.

[0324] [Example of providing information related to a character battle game displayed on a display device installed outdoors] Fig. 46 is a simplified diagram showing an example of the configuration of communication system 40 in the ninth embodiment. Fig. 46 shows an example in which wireless communication is performed between electronic devices 900, 901 to 904, and a display device 910. Also, an example is shown in which electronic devices 900, 901 to 904 are carried by users P0 to P4, and display device 910 is fixedly installed.

[0325] The electronic devices 900, 901 to 904 are electronic devices capable of displaying characters on a display unit 911 of a display device 910 and controlling each character based on a user operation. The electronic devices 900, 901 to 904 are, for example, smartphones, tablet terminals, and game devices.

[0326] Also, character A is a character controlled by electronic device 900, character B is a character controlled by electronic device 901, character C is a character controlled by electronic device 902, and character D is a character controlled by electronic device 903. Note that the character controlled by electronic device 904 is not yet displayed on display unit 911 of display device 910.

[0327] The display device 910 is a display device (information processing device, electronic device) including a communication unit (not shown) capable of wireless communication, a control unit (not shown) that controls each unit, and a display unit 911. For example, the control unit (not shown) of the display device 910 controls the display of the display unit 911 based on information transmitted from the electronic devices 900, 901 to 904. Specifically, the control unit (not shown) of the display device 910 causes the display unit 911 to display character A based on information transmitted from the electronic device 900, causes the display unit 911 to display character B based on information transmitted from the electronic device 901, causes the display unit 911 to display character C based on information transmitted from the electronic device 902, and causes the display unit 911 to display character D based on information transmitted from the electronic device 903. In addition, a control unit (not shown) of the display device 910 controls the movement of character A on the display unit 911 based on information transmitted from the electronic device 900, controls the movement of character B on the display unit 911 based on information transmitted from the electronic device 901, controls the movement of character C on the display unit 911 based on information transmitted from the electronic device 902, and controls the movement of character D on the display unit 911 based on information transmitted from the electronic device 903.

[0328] Further, a control unit (not shown) of the display device 910 adds various information to characters A to D on the display unit 911 and displays them based on information transmitted from the electronic devices 900, 901 to 903. For example, the control unit (not shown) of the display device 910 adds a tool 71 to character A on the display unit 911 and displays it based on information transmitted from the electronic device 900, adds a tool 72 to character B on the display unit 911 and displays it based on information transmitted from the electronic device 901, adds a tool 73 to character C on the display unit 911 and displays it based on information transmitted from the electronic device 902, and adds a tool 74 to character D on the display unit 911 and displays it based on information transmitted from the electronic device 903.

[0329] [Example of information provided to characters in fighting games] Fig. 47 is a diagram showing an example of a correspondence relationship between the provided information OI determined by the electronic devices 900, 901 to 904 using the determination information in the ninth embodiment and the provided information 921. Note that the determination information (table) used when determining the provided information may be each of the above-mentioned determination information (Figs. 6, 12, and 43). Each of these pieces of information is stored in the storage unit of each electronic device.

[0330] 47 shows an example in which the tools used by each character are determined as the provided information, but other information (for example, experience points, attributes, etc., given to each character) may be determined. Also, a plurality of pieces of information may be provided.

[0331] For example, when OIXY1 is determined as the provided information OI, the tool 71 (possessed by character A shown in FIG. 46) is the information to be provided. When OIYY2 is determined as the provided information OI, the tool 74 (possessed by character D shown in FIG. 46) is the information to be provided. When OIYZ1 is determined as the provided information OI, the tool 72 (possessed by character B shown in FIG. 46) is the information to be provided. When OIZZ7 is determined as the provided information OI, the tool 73 (possessed by character C shown in FIG. 46) is the information to be provided.

[0332] [Example of operation when a character displayed on a display device acquires a tool] FIG. 48 is a diagram showing an example of an operation for additionally displaying a tool 75 on a character A displayed on a display unit 911 of a display device 910 using the electronic device 900 in the ninth embodiment.

[0333] 48(1) shows an example of output when outputting notification information for notifying the user that it is possible to acquire a new tool. The display unit 950 of the electronic device 900 displays a message that it is possible to acquire a new tool, and the audio output unit 960 outputs an audio message that it is possible to acquire a new tool. In addition, the display unit 950 of the electronic device 900 displays an operation button group 81 and an "Acquire" button 82.

[0334] The operation button group 81 is buttons (indicated by circles and triangles) for operating the character A displayed on the display unit 911 of the display device 910. For example, the user P0 can press a desired button from the operation button group 81 to cause the character A displayed on the display unit 911 of the display device 910 to perform a predetermined action (for example, advance, retreat, attack, or defend).

[0335] The "Acquire" button 82 is a button that is displayed when a new tool can be acquired. When the user P0 presses the "Acquire" button 82, a display screen shown in FIG. 48(2) is displayed on the display unit 950 of the electronic device 900.

[0336] 48(2) shows an example of output when outputting notification information for notifying the user that a new tool has been acquired. The display unit 950 of the electronic device 900 displays a message that a new tool has been acquired and an image showing the newly acquired tool, and the audio output unit 960 outputs an audio message that a new tool has been acquired. In addition, the display unit 950 of the electronic device 900 displays a group of operation buttons 81 and a "use" button 83.

[0337] The "Use" button 83 is a button that is displayed when a new tool is acquired by pressing the "Acquire" button 82. The newly acquired tool may be used immediately or at a desired timing (for example, when actually used in the game).

[0338] When the "use" button 83 is pressed by the user P0, the tool 75 is additionally displayed for the character A on the display unit 911 of the display device 910, as shown in Fig. 48(3). That is, the control unit of the electronic device 900 transmits information about the acquired provided information (tool 75) (information for displaying the tool 75 on the display unit 911 of the display device 910 and making it usable by the character A) to the display device 910 based on the pressing operation of the "use" button 83. Then, the control unit of the display device 910 additionally displays the tool 75 for the character A on the display unit 911 based on the information received from the electronic device 900.

[0339] 48(3), by additionally displaying tool 75 on character A on display unit 911 of display device 910, character A can possess tool 71 and tool 75, which gives an advantage in battles with other characters. Note that, although this example shows an example in which a character is displayed on an external device (display device 910), the same can be applied to a game played by a user of each electronic device by displaying a character on the display unit of each electronic device.

[0340] [Example of providing information related to storytelling games] Fig. 49 is a simplified diagram showing an example of a story game displayed on display unit 950 of electronic device 900 in the ninth embodiment. Fig. 49 shows an example in which a user causes display unit 950 of electronic device 900 to display the story game and plays the story game.

[0341] Here, the story game is a game played using a predetermined sign to achieve a predetermined objective. For example, the story game is a game in which characters (predetermined signs) playing each role progress to a destination while experiencing various experiences (e.g., adventure, exploration, battle). In FIG. 49, a game in which character X (shown in the lower part) progresses to XX Castle (shown in the upper part) as a destination while experiencing various experiences (e.g., battle with other characters, progress along each path) will be described as an example. Note that, in FIG. 49, for ease of explanation, an example is shown in which the entire story game is displayed on the display screen of the display unit 950 of the electronic device 900, but an area larger than the display screen of the display unit 950 may be set as the story game area, and the story game may be performed using multiple screens (e.g., screens with a hierarchical structure). In this way, the story game can be called a goal achievement game, a character progression game, an adventure game, an exploration game, or the like.

[0342] The control unit of the electronic device 900 causes the display unit 950 to display a story game based on a program (e.g., a story game application) stored in the storage unit. The control unit of the electronic device 900 also controls various actions of the character X based on a user operation (e.g., a user operation on an operation unit, a user operation based on a user's voice input, an operation based on a change in the posture of the electronic device 900).

[0343] Furthermore, the control unit of the electronic device 900 adds various information to the character X on the display unit 950 and displays the same based on the information related to the drink. For example, the control unit of the electronic device 900 adds tools 933 and 934 (shown in FIG. 51 ) to the character X on the display unit 950 and displays the same based on multiple information related to the drink.

[0344] [Example of information provided to characters in a narrative game] Fig. 50 is a diagram showing an example of a correspondence relationship between the provided information OI determined by the electronic device 900 in the ninth embodiment using the determination information, and the provided information 941, 942, and 943. Note that the determination information (table) used when determining the provided information may be each of the above-mentioned determination information (Figs. 6, 12, and 43). Each of these pieces of information is stored in the storage unit of the electronic device 900.

[0345] Although FIG. 50 shows an example in which three or more pieces of information are provided as the provided information, it is also possible to provide only one or two pieces of information.

[0346] For example, when OIJK1 is determined as the provided information OI, the provided items are "tool (rectangular tool)", "house 108", and "money 99". When OIGT3 is determined as the provided information OI, the provided items are "experience value 89", "tool (T-shaped tool)", and "money 67". When OIMU9 is determined as the provided information OI, the provided items are "tool (sword-shaped tool)", "money 678", and "experience value 2". When OIPO6 is determined as the provided information OI, the provided items are "tank 7", "money 100", and "tool (rod-shaped tool)".

[0347] For example, if the provided information is a tool, the tool is provided to character X, and character X can use the tool. Also, for example, if the provided information is an experience value, the numerical value of the experience value is sequentially calculated (for example, added), and character X can perform various actions based on the experience value based on the calculation result. Also, for example, if the provided information is money, the numerical value of the money is sequentially calculated (for example, added), and character X can perform various purchases, etc. based on the total amount based on the calculation result. Also, for example, if the provided information is a car, a house, a tank, or an airplane, the numerical value of the item is sequentially calculated (for example, added), and character X can use various vehicles, properties, etc. based on the total value of each item based on the calculation result.

[0348] [Example of information provided] Fig. 51 is a diagram showing a display example of the case where provided information is displayed on the display unit 950 of the electronic device 900 in the ninth embodiment. Fig. 51 shows an example in which provided information acquired during a predetermined period is displayed on a screen of a story game on the display unit 950 of the electronic device 900.

[0349] The control unit of the electronic device 900 can output (e.g., output audio, display images, transmit to another device) the provided information acquired during a predetermined period (e.g., the period from the previous display of provided information) at a predetermined output timing (e.g., when a story game is started, when an instruction is given by a user operation, or when a predetermined time arrives). For example, the control unit of the electronic device 900 can sequentially store the provided information acquired during the predetermined period in the storage unit. Then, when the predetermined output timing arrives, the control unit of the electronic device 900 can display a list of the provided information acquired during the predetermined period in the provided information display area 932. For example, for experience points, money, houses, vehicles, etc., each calculation result (e.g., total value) is displayed along with each item. For tools, an image representing the tool is displayed.

[0350] [Example of providing information related to the game] Fig. 52 is a simplified diagram showing an example of transitions in a fish raising game displayed on the display unit 950 of the electronic device 900 in the ninth embodiment. Fig. 52 shows a time series of transitions in which a fish grows in the fish raising game displayed on the display unit 950 of the electronic device 900.

[0351] Here, the raising game is a game in which animals, plants, etc. are raised. For example, the raising game is a game in which characters (prescribed symbols) such as animals and plants grow while taking on various roles and having various experiences (for example, adventures, exploration, and battles). Figure 52 shows a game in which a fish grows.

[0352] FIG. 52(1) shows an example in which a newborn fish (juvenile fish) is displayed in the game of raising fish. FIG. 52(2) shows an example in which a slightly grown fish is displayed in the game of raising fish. FIG. 52(3) shows an example in which a large grown fish is displayed in the game of raising fish. Note that, for ease of explanation, FIG. 52 shows an example in which the whole game of raising fish is displayed on the display screen of the display unit 950 of the electronic device 900, but the field of the game of raising fish may be a larger area than the display screen of the display unit 950, and the game of raising fish may be performed using multiple screens (for example, screens with a hierarchical structure). FIG. 52 shows an example of a game in which one fish is raised, but multiple fish may be raised, and multiple types of fish, different types of animals, plants, etc. may be raised. Also, the fish may be changed during the raising process (for example, changed from a fish to a cow). In this way, the game of raising fish can be called a raising game, a watching game, a breeding game, a pet game, etc.

[0353] The control unit of the electronic device 900 causes the display unit 950 to display a fish-raising game based on a program (e.g., a fish-raising game application) stored in the storage unit. The control unit of the electronic device 900 also controls various operations for raising fish based on user operations (e.g., user operations on an operation unit, user operations based on user voice input, operations based on changes in the posture of the electronic device 900).

[0354] Furthermore, the control unit of the electronic device 900 adds various information to the fish on the display unit 950 based on the information about the drink and displays it. For example, the control unit of the electronic device 900 adds food or an enemy to the fish on the display unit 950 and displays it based on multiple pieces of information about the drink.

[0355] [Example of information to provide to characters in a game you are raising] Fig. 53 is a diagram showing an example of a correspondence relationship between the provided information OI determined by the electronic device 900 in the ninth embodiment using the determination information, and the provided information 961, 962, and 963. Note that the determination information (table) used when determining the provided information may be each of the above-mentioned determination information (Figs. 6, 12, and 43). Each of these pieces of information is stored in the storage unit of the electronic device 900.

[0356] Although FIG. 53 shows an example in which three or more pieces of information are provided as the provided information, it is also possible to provide only one or two pieces of information.

[0357] For example, when OIHH1 is determined as the provided information OI, the provided information includes "being attacked by a shark", "being caught", and "becoming a yaki". In this case, the provided information 1 or 2 may be selected from the three provided information by a user operation, and the fish may experience all three provided information (for example, in the order of the first to third). For example, when the provided information of "being attacked by a shark" is to be experienced by the fish, the display unit 950 displays a shark together with the fish to be raised, and sequentially displays the fish to be raised escaping from the shark. Also, for example, when the provided information of "being caught" is to be experienced by the fish, the display unit 950 displays the fish to be raised being caught by a person, and sequentially displays the fish to be raised escaping from the fishing rod. Also, for example, when the provided information of "becoming a yaki" is to be experienced by the fish, the display unit 950 displays the fish to be raised becoming a yaki, and sequentially displays the fish to be raised waking up from the yaki dream.

[0358] In this way, in the fish breeding game, images related to the provided information are displayed simultaneously or sequentially. Alternatively, in the fish breeding game, audio information related to the provided information may be output simultaneously or sequentially.

[0359] For example, in the game of raising fish, when a user operation is performed to start the game, eggs or fry of the fish to be raised are displayed on the display unit 950. After that, the fish can be automatically raised until it grows. For example, when a user brews coffee every day, the provided information is sequentially acquired according to the brewing. Then, when the user performs an operation to instruct the display of the game of raising fish, the fish that has grown based on the acquired provided information is displayed on the display unit 950. Alternatively, after the user performs an operation to instruct the display of the game of raising fish, the transition of the growth of the fish based on the acquired provided information may be sequentially displayed on the display unit 950. Also, the fish may be grown based on the user operation.

[0360] The control unit of the electronic device 900 can perform processing (e.g., growing taller, making friends) based on the provided information acquired during a predetermined period (e.g., the period from the previous display of the raising game) at a predetermined timing (e.g., when an operation to instruct display of the raising game is performed, or when a predetermined time arrives). For example, the control unit of the electronic device 900 sequentially stores the provided information acquired during the predetermined period in the storage unit. Then, the control unit of the electronic device 900 performs processing based on the provided information acquired during the predetermined period when a predetermined output timing arrives.

[0361] [Example of providing information related to the Roaming Game] Fig. 54 is a simplified diagram showing an example of transitions in a roaming game displayed on the display unit 950 of the electronic device 900 in the ninth embodiment. Fig. 54 shows a chronological transition in which a sheep, the main character, travels around the world in the roaming game displayed on the display unit 950 of the electronic device 900.

[0362] Here, the wandering game is a game in which the user enjoys growing and moving animals, plants, etc. For example, the wandering game is a game in which characters (prescribed signs) such as animals and plants travel around the world while taking on various roles (e.g., travelers) and having various experiences (e.g., taking commemorative photos at tourist spots, sightseeing at tourist spots). FIG. 54 shows a game in which a sheep travels.

[0363] FIG. 54(1) shows an example in which a sheep, a main character, is displayed in a wandering game. FIG. 54(2) and FIG. 54(3) show an example in which a sheep is displayed after traveling to a predetermined place in a wandering game. In FIG. 54, for ease of explanation, an example is shown in which the entire wandering game is displayed on the display screen of the display unit 950 of the electronic device 900, but the field of the wandering game may be made to be a larger area than the display screen of the display unit 950, and the wandering game may be performed using multiple screens (for example, screens with a hierarchical structure). In FIG. 54, an example of a game in which one sheep is wandered is shown, but multiple sheep may be wandered, and multiple types of animals, different types of animals, plants, etc. may be raised. In addition, the sheep may be replaced (for example, replaced by a cow) during the wandering. In this way, the wandering game can be called an idle game, etc.

[0364] The control unit of the electronic device 900 causes the display unit 950 to display the roaming game based on a program (e.g., a roaming game application) stored in the storage unit. The control unit of the electronic device 900 also controls various operations for causing the sheep to roam based on user operations (e.g., user operations on an operation unit, user operations based on user voice input, operations based on changes in the posture of the electronic device 900).

[0365] Furthermore, based on the information related to the drink, the control unit of the electronic device 900 adds various information to the sheep on the display unit 950 and displays it. For example, based on multiple pieces of information related to the drink, the control unit of the electronic device 900 adds and displays food, travel preparations, food, souvenirs from the wandering place, and photos from the wandering place to the sheep on the display unit 950.

[0366] [Example of information to provide to characters in the Roaming Game] Fig. 55 is a diagram showing an example of a correspondence relationship between the provided information OI determined by the electronic device 900 in the ninth embodiment using the determination information, and the provided information 971, 972, and 973. Note that the determination information (table) used when determining the provided information may be each of the above-mentioned determination information (Figs. 6, 12, and 43). Each of these pieces of information is stored in the storage unit of the electronic device 900.

[0367] Although FIG. 55 shows an example in which three or more pieces of information are provided as the provided information, it is also possible to provide only one or two pieces of information.

[0368] For example, when OIHK9 is determined as the provided information OI, the provided information includes "obtain a camera", "obtain 10 kg of grass", and "have a part of wool cut". In this case, the provided information 1 or 2 may be selected from the three provided information by a user operation, and the sheep may experience all three provided information (for example, in the order of the first to third provided information) based on a predetermined rule (for example, the first provided information → the second provided information → the third provided information). For example, when the sheep experiences the provided information of "obtain a camera", the display unit 950 displays a sheep with the obtained camera around its neck. Then, when the sheep goes on a journey after this, it can be set to obtain a photo taken by the camera and return. Also, for example, when the sheep experiences the provided information of "obtain 10 kg of grass", the display unit 950 displays the sheep eating the obtained grass at home. Then, the user may select to use the remaining grass for a lunch box for the journey. Also, for example, when the sheep is made to experience the provided information of "having part of its wool sheared," the display unit 950 displays a scene of someone shearing the sheep's wool, and thereafter, until a predetermined time has elapsed, images of the sheep without part of its wool are displayed in sequence.

[0369] In this way, in the roaming game, images related to the provided information are displayed simultaneously or sequentially. Alternatively, in the roaming game, audio information related to the provided information may be output simultaneously or sequentially.

[0370] For example, in the wandering game, when a user operation is performed to start the game, a sheep, the main character, is displayed on the display unit 950. After that, the sheep can automatically go on a wandering journey around the world based on various information added to the sheep. For example, when a user brews coffee every day, the provided information is sequentially acquired according to the brewing. Then, when the user performs a display instruction operation of the wandering game, the sheep in the wandering area and the sheep that has returned home are displayed on the display unit 950 based on the acquired provided information. Alternatively, after the user performs a display instruction operation of the wandering game, history information of various places where the sheep has traveled in the past (for example, commemorative photos taken at tourist spots) may be sequentially displayed on the display unit 950 based on the acquired provided information. For example, when a "Travel History" button 981 shown in FIG. 54(2)(3) is pressed, history information of the sheep's past travels (for example, the places where the sheep traveled (for example, tourist spot names, place names, country names), and the date and time of the travel) are displayed on the display unit 950. Furthermore, for example, when a "Travel Photo" button 982 shown in Fig. 54(2)(3) is pressed, photo information of places where the sheep has traveled in the past (for example, commemorative photos taken at places where the sheep has traveled) is displayed on the display unit 950. Furthermore, for example, when a "Souvenir" button 983 shown in Fig. 54(2)(3) is pressed, souvenir information of places where the sheep has traveled in the past (for example, images of local products, famous products, etc. of the places where the sheep has traveled, and descriptions of souvenirs) is displayed on the display unit 950. Furthermore, the sheep may be made to travel based on a user operation.

[0371] The control unit of the electronic device 900 can perform processing (e.g., travel to Gunma, travel to Okinawa) based on the provided information acquired during a predetermined period (e.g., the period from the previous display of the roaming game) at a predetermined timing (e.g., when an operation to instruct display of the roaming game is performed, or when a predetermined time arrives). For example, the control unit of the electronic device 900 sequentially stores the provided information acquired during the predetermined period in the storage unit. Then, the control unit of the electronic device 900 performs processing based on the provided information acquired during the predetermined period when a predetermined output timing arrives.

[0372] [Game operation example] 56 is a flowchart showing an example of the provision information determination process by the electronic device 900 in the ninth embodiment. Note that this operation example shows an example in which information about the drink is stored in the storage unit of the electronic device 900 by each of the above-mentioned processes. For example, the electronic device 900 may acquire and store information about the drink (e.g., image information) using a sensor or the like of the electronic device 900, or the electronic device 900 may acquire and store information acquired by another device (information about the drink (e.g., image information)).

[0373] First, the control unit of the electronic device 900 judges whether or not a user operation has been performed (step S1001). For example, if any of the operation buttons 81 shown in Fig. 48(1) is pressed, it is judged that a user operation has been performed. If a user operation has not been performed, the process proceeds to step S223.

[0374] When a user operation is performed (step S1001), the control unit of the electronic device 900 performs processing based on the user operation (step S1002). For example, when any of the operation buttons 81 shown in Fig. 48(1) is pressed, the control unit of the electronic device 900 transmits information to the display device 910 for causing the character A to perform an action corresponding to the pressed button.

[0375] Next, the control unit of the electronic device 900 judges whether the provision conditions are satisfied (step S1003). If the provision conditions are not satisfied, the operation is terminated, and if the provision conditions are satisfied, the operation proceeds to step S1004.

[0376] Here, as a criterion for judging whether the provision condition is satisfied, for example, it is possible to use whether a predetermined time has elapsed, whether a desired provision information (e.g., a desired tool, a desired experience) can be determined as a provision target, whether a predetermined user operation (e.g., pressing the "Acquire" button 82 shown in FIG. 48(1)) has been performed, and whether new provision information (e.g., a desired tool, a desired experience) can be acquired. Note that the predetermined time can be determined based on, for example, the elapsed time since the power supply of the electronic device 900 was turned on, or the elapsed time since the previous information (provision information) was provided after the power supply of the electronic device 900 was turned on. Also, the predetermined time may be a time set in advance, or may be set by a user operation according to the user's preference.

[0377] When the provision condition is satisfied (step S1003), the control unit of the electronic device 900 determines information to be provided based on information (information about the drink) stored in the storage unit (step S1004).

[0378] Next, the control unit of the electronic device 900 stores the information determined as the information to be provided in the storage unit (step S1005). In this way, the information to be provided stored in the storage unit can be used based on, for example, a user operation. For example, the acquired tool can be used by pressing the "Use" button 83 shown in FIG. 48(2).

[0379] In this way, the control unit of the electronic device 900 can determine at least one piece of content to provide to the user from among a plurality of pieces of content (e.g., a plurality of pieces of information to be imparted to characters in a competitive game, a plurality of pieces of information to be imparted to characters in a story game, a plurality of pieces of information to be imparted to characters in a development game, and a plurality of pieces of information to be imparted to characters in a wandering game) based on information about the drink.

[0380] Furthermore, the control unit of electronic device 900 can select and execute at least one process from a plurality of processes (e.g., a process for progressing a game, a process for giving information to a character in a game) different from the process (e.g., an extraction process) related to the drink, based on the information related to the drink. Note that each of these games is merely an example, and the same can be applied to other games.

[0381] As shown in the fifth embodiment, the server may collect information from a plurality of electronic devices and determine the information to be provided to each device based on the collected results.

[0382] [Tenth embodiment: Application to other foods and beverages] In the above, an example has been shown in which various processes (e.g., content output, game processing) are performed mainly based on information acquired about drinks. However, various processes (e.g., content output, game processing) may be performed based on information acquired about other foods and drinks. Therefore, in the following, an example will be shown in which various processes (e.g., content output, game processing) are performed based on information acquired about foods and drinks. Note that the tenth embodiment is an example in which the first to ninth embodiments are partially modified, and some of the explanations of the parts common to the first to ninth embodiments will be omitted.

[0383] For example, information on the ingredients in a pot during cooking can be acquired and used. For example, image information on the ingredients in the pot can be acquired sequentially, and each process (for example, content output, game process) can be performed based on the image information (or the transition of the image information). For example, an electronic device (for example, a smartphone, a tablet terminal) equipped with an image acquisition unit captures an image of the inside of the pot during cooking. In this case, the image capture range is the ingredients in the pot during cooking, and the image capture period can be determined according to the user's preference. In this case, for example, in the case of a round pot, each of the above-mentioned determination information (FIGS. 6, 12, 43) can be used. Also, each process (for example, content output, game process) can be performed using other determination information (for example, determination using various colors instead of white and black determination, determination using the arrangement of each color).

[0384] Also, for example, information on a plurality of dishes arranged on a table (or dishes on a table being prepared) can be acquired and used. For example, image information on each of the dishes can be acquired sequentially, and each process (for example, content output, game process) can be performed based on the image information (or the transition of the image information). For example, an electronic device (for example, a smartphone, a tablet terminal) equipped with an image acquisition unit captures an image of the plurality of dishes arranged on a table (or dishes on a table being prepared). In this case, the shooting range is a range preferred by the user, and the shooting period can be determined according to the user's preference. In this case, for example, when the shooting range is a circular range, each of the above-mentioned determination information (FIGS. 6, 12, 43) can be used. Also, each process (for example, content output, game process) can be performed using other determination information (for example, determination using various colors instead of white and black determination, determination using the arrangement of each color). Also, for example, when the shooting range is a rectangular range, image information of the rectangular range can be used instead of the above-mentioned determination information (FIGS. 6, 12, 43). This example is shown in FIG. 57 to FIG. 60. It should be noted that the example shown in Figs. 57 to 60 is a modified example of the example shown in Figs. 5 to 12, and a description of common parts will be omitted.

[0385] [Example of food and drink transitions on a table] Fig. 57 is a diagram showing an example of the transition of images (images of food and drink arranged on table 1010) that can be acquired by the electronic device in the tenth embodiment. Note that in Fig. 57, for ease of explanation, table 1010 is shown simplified by a rectangle, and the food and drink arranged on table 1010 are also shown simplified.

[0386] Fig. 57(1) shows a state where there is no food or drink on table 1010 other than condiment container 1016. Fig. 57(2) shows a state where container 1016 has been removed from table 1010, and a dish (fried egg and seaweed) has been arranged on plate 1011 and a dish (bread) has been arranged on plate 1012. Fig. 57(3) shows a state where a dish (salad) has been arranged on plate 1013 and a dish (soup) has been arranged on plate 1014 on table 1010. Fig. 57(4) shows a state where a dish (squid ink pasta) has been arranged on plate 1014 on table 1010.

[0387] Fig. 57(5) shows a state where some of the various dishes laid out on the table 1010 have been eaten. Fig. 57(6) shows a state where most of the various dishes laid out on the table 1010 have been eaten.

[0388] [Example of judgment information for determining the information to be output using image information about food and drink arranged on a table] FIG. 58 is a diagram showing an example of various information used when determining information to be output by the electronic device in the tenth embodiment.

[0389] FIG. 58(1) shows an example of regions A to P used when acquiring information about food and drink arranged on a table (for example, image information about food and drink arranged on a table 1010). The rectangular region shown in FIG. 58(1) is a region corresponding to the rectangle of the table in an image acquired by an image acquisition unit of an electronic device (for example, a rectangular image acquired by an imaging element). The rectangular region shown in FIG. 58(1) is divided into 16 regions to set the regions A to P. For example, the regions A to P can be set so that each region has approximately the same area. Note that the division of the rectangular region shown in FIG. 58(1) is just an example, and the rectangular region may be divided into a plurality of regions by other division methods. For example, the rectangular region may be divided into four regions, the rectangular region may be divided into six regions, or the rectangular region may be divided into eight regions. Alternatively, multiple regions (closed regions) of irregular shapes (e.g., polygons (e.g., triangles, squares, pentagons), circles, ellipses) can be placed in a rectangular region so as not to overlap, and multiple regions can be set using each of these regions (closed regions, regions outside the closed regions).

[0390] FIG. 58(2) shows an example of determination information (table) used when determining information to be output based on information acquired using areas A to P shown in FIG. 58(1). The determination information shown in FIG. 58(2) is information showing the correspondence between the transition of information about food and drink arranged on the table (for example, image information about food and drink arranged on the table 1010) and information OI to be output. For example, the determination information shown in FIG. 58(2) defines all combinations (patterns) of A to P. For example, a pattern "A, A, ..., A, A" in which all are A is defined for information OI1 to be output. Furthermore, a pattern "A, B, A, ..., A, A" in which all are A except for one (time t2 is "B") is defined for information OI2 to be output. Furthermore, a pattern "A, B, B, A, ..., A, A" in which all are A except for two (times t2 and t3 are "B") is defined for information OI2 to be output. The rest are defined in the same manner, and the pattern of P “P, P, . . . , P, P” is defined for all the information OIM to be output.

[0391] In FIG. 58(2), times t1, ..., tn (where n is a positive integer) are values ​​indicating the elapsed time when image information is acquired using the image acquisition unit of the electronic device. For example, if the interval for acquiring image information is 20 seconds, then the intervals are 20 seconds, i.e., time t1 = 20 seconds, time t2 = 40 seconds, time t3 = 60 seconds, .... Then, the control unit of the electronic device sequentially determines the area with the highest proportion of black among the areas A to P shown in FIG. 58(1) at each of times t1, ..., tn, and determines the information OI (OI1 to OIM (where M is a positive integer)) to be output based on the determination result. For example, if the area with the highest proportion of black among the areas A to P shown in FIG. 58(1) at each of times t1, ..., tn is all A (area A shown in FIG. 58(1)), the control unit of the electronic device determines OI1 as the information OI to be output. Also, for example, if, at each of times t1, ..., tn, among areas A to P shown in Figure 58(1), the area with the highest proportion of black is all P (area P shown in Figure 58(1)), the control unit of the electronic device determines OIM as the information OI to be output.

[0392] The interval between the times t1, ..., tn and the value of n may be preset, or may be set by the user according to the user's preference. The value of M is determined based on the number of regions dividing the rectangle corresponding to the table on which the food and drink are arranged, and the value of n.

[0393] [An example of determining the information to be output using image information about food and drink laid out on a table] Fig. 59 is a diagram showing the relationship between images (images of food and drink arranged on table 1010) that can be acquired by the electronic device in the tenth embodiment and areas A to P shown in Fig. 58(1). Note that the images of food and drink arranged on table 1010 shown in Fig. 59(1) to (6) are the same as the images shown in Fig. 57(1) to (6).

[0394] Fig. 60 is a diagram showing an example of determining information to be output by the electronic device in the tenth embodiment. Fig. 59 and Fig. 60 show an example of using times t1 to t6 as the elapsed time when image information is acquired using an image acquisition unit of the electronic device.

[0395] The images of food and drink arranged on a table 1010 shown in Fig. 59(1) to (6) are taken at times t1 to t6.

[0396] FIG. 60(1) shows an example of storing the area with the highest black ratio among areas A to P shown in FIG. 58(1) at times t1 to t6 in association with times t1 to t6.

[0397] The control unit of the electronic device acquires a feature amount for an image included in a rectangular region (a rectangular region corresponding to table 1010) in an image acquired at a timing of time t1. Specifically, the control unit of the electronic device acquires a feature amount for each image included in the rectangular region in an image acquired at a timing of time t1 (each image included in regions A to P shown in FIG. 58(1)). For example, the control unit of the electronic device obtains information on a specific color (e.g., black) as a feature amount for each image included in regions A to P. For example, the control unit of the electronic device determines each pixel in each image included in the image acquired by the image acquisition unit as either a black region or a white region. For example, this can be determined by a binarization process (for example, a process in which a threshold value is set in advance, and if the pixel value is greater than the threshold value, it is determined to be white, and if the pixel value is less than the threshold value, it is determined to be black). Then, the control unit of the electronic device obtains the ratio of the black region and the white region in each image included in regions A to P as a feature amount.

[0398] Next, the control unit of the electronic device determines the area with the highest proportion of black areas from among the images contained in areas A to P. In the example shown in Fig. 59(1), the area with the highest proportion of black areas from among the images contained in areas A to P is area M because only container 1016 is present on table 1010. Therefore, the control unit of the electronic device determines area M as the area with the highest proportion of black areas from the images acquired at time t1.

[0399] Similarly, the control unit of the electronic device determines the area with the highest proportion of black areas for each image (images of food and drink arranged on table 1010) acquired at times t2 to t6. The control unit of the electronic device then sequentially stores the determined feature amounts (areas with the highest proportion of black areas) for each image acquired at times t1 to t6 in the memory unit in association with times t1 to t6. An example of this storage is shown in FIG. 60(1). That is, the storage example shown in FIG. 60(1) is the storage example for the examples shown in FIGS. 59(1) to (6).

[0400] Fig. 60(2) shows an example of extraction when extracting feature amounts that match the feature amounts shown in Fig. 60(1) (information associating times t1 to t6 with areas A to P) from the determination information shown in Fig. 58(2). Note that the determination information shown in Fig. 60(2) is the same as the determination information shown in Fig. 58(2), and some of the determination information shown in Fig. 58(2) is omitted.

[0401] Next, the control unit of the electronic device determines information to be output based on the acquired feature amount. Specifically, the control unit of the electronic device extracts a feature amount that matches the acquired feature amount (information associating times t1 to t6 with areas A to P shown in FIG. 60(1)) from the judgment information shown in FIG. 58(2). For example, as shown in FIG. 60(2), the feature amount shown in FIG. 60(1) matches the feature amount shown in the dashed rectangle 1021. Therefore, the control unit of the electronic device extracts the feature amount shown in the dashed rectangle 1021 from the judgment information shown in FIG. 60(2) as the feature amount that matches the feature amount shown in FIG. 60(1). In addition, the control unit of the electronic device determines OITT1 as information OI (information to be output) corresponding to the extracted feature amount.

[0402] Next, the control unit of the electronic device causes the content to be output based on the information stored in the storage unit in association with OITT1 determined as the information OI to be output.

[0403] In the above, an example in which black is used as the specific color has been shown, but other colors may be used. In addition, when multiple colors can be distinguished, the information to be output may be determined using the multiple colors. For example, for an image acquired at time t1 (an image of food and drink arranged on the table 1010), a region with the highest ratio of a first color (e.g., red) may be determined, for an image acquired at time t2, a region with the highest ratio of a second color (e.g., green) may be determined, for an image acquired at time t3, a region with the highest ratio of a third color (e.g., blue) may be determined, and for an image acquired at time t4, a region with the highest ratio of a fourth color (e.g., yellow) may be determined. Similarly, the specific color region may be determined using different colors or the same color.

[0404] Also, audio information to be output may be determined using other determination information. For example, image patterns as shown in Fig. 12 and Fig. 43 may be used. Also, content may be output using information acquired about other drinks (e.g., soft drinks, carbonated drinks, beer, alcohol) other than drinks (e.g., coffee, black tea) produced by soaking substances such as coffee beans, black tea, etc. until the liquid becomes delicious. For example, content may be determined based on the transition of image information (e.g., image of the surface, image of the side) about the drink poured into a container (cup).

[0405] In the above, an example has been shown in which information about food and drink (e.g., surface transitions) is obtained mainly using an image sensor (image sensor, image capture element), but other sensors may be used to obtain information about food and drink. For example, as described with reference to FIG. 37, information about food and drink may be obtained and used using various sensors (e.g., a temperature sensor capable of measuring temperature, an odor sensor (odor sensor) capable of measuring odor, a sound sensor (e.g., a microphone) capable of measuring sound, a weight sensor capable of measuring weight, and a taste sensor capable of measuring taste). For example, the content to be output can be determined based on the transitions of each piece of obtained information.

[0406] Thus, the electronic device and information processing device (for example, the server 410 (or a plurality of devices constituting it)) shown in the first to tenth embodiments are examples of an information processing device, information processing system, and communication system that use transitions in information about food and drink to control output of content from an output unit. Also, the electronic device and information processing device (for example, the server 410 (or a plurality of devices constituting it)) shown in the first to tenth embodiments are examples of an information processing device, information processing system, and communication system that use transitions in information about food and drink to control execution of at least one process from a plurality of processes (for example, content output, game process) different from a process related to evaluation of food and drink (for example, a process of generating evaluation information such as delicious, good-looking, beautifully presented, etc., and a process of outputting the evaluation information) and a process related to generation of food and drink (for example, a process of generating information for providing cooking advice, and a process of outputting the advice). These controls are performed, for example, by a control unit provided in each device.

[0407] Also, as shown in the fifth embodiment, the server may collect information from a plurality of electronic devices and determine information to be provided to each device based on the collected results. For example, a user may use an electronic device to obtain image information on food in one container (e.g., ramen or beef bowl in a bowl, or food served on a plate (e.g., pasta, meat dish, fish dish)) and transmit it to the server, and the server may determine content to be provided to the user based on the image information and transmit it to the electronic device. In this case, the server may determine content to be provided to each user based on the image information transmitted from the plurality of users and transmit it to the electronic device of each user. That is, the electronic device and information processing device (e.g., server 410 (or a plurality of devices constituting it)) shown in the first to tenth embodiments are examples of electronic devices, information processing devices, information processing systems, and communication systems that obtain information on food and drink for each food and drink in a plurality of containers and perform control to provide content determined based on the obtained information to the electronic devices of each person associated with the plurality of containers. Furthermore, the electronic devices and information processing devices (e.g., server 410 (or multiple devices constituting it)) shown in the first to tenth embodiments are examples of electronic devices, information processing devices, information processing systems, and communication systems that acquire information about each food and beverage contained in multiple containers, and control the electronic devices of each person associated with the multiple containers to execute processing determined based on the acquired information.

[0408] As described above, each of the electronic devices and information processing devices (e.g., server 410) shown in the first to tenth embodiments is an example of an electronic device, information processing device, information processing system, and communication system that determines content (e.g., information unrelated to the drink) based on information acquired about the generated drink (information acquired from the drink) when the drink is generated from a substance (e.g., a powdered substance, a liquid substance, a solid substance) and a liquid to generate the drink and storing the drink in a container, when the generated drink is placed in the container, or after the generated drink is placed in the container.

[0409] Moreover, the electronic devices and information processing devices (e.g., server 410) shown in the first to tenth embodiments are examples of electronic devices, information processing devices, information processing systems, and communication systems that acquire and use information about a liquid that is collected in a container that collects the liquid that drips from an extractor for extracting a drink. For example, the information about the liquid can be used to determine the content to be provided to the user.

[0410] Furthermore, the server 410 (or multiple devices constituting it) shown in the fifth embodiment is an example of an information processing device, information processing system, or communication system that acquires information about the generated drink (information acquired from the drink) for each drink placed in multiple containers when the drink is generated from a substance (e.g., a powdered substance, a liquid substance, a solid substance) and a liquid for generating the drink and the drink is stored in a container, when the generated drink is placed in the container, or after the generated drink is placed in the container, and determines content (e.g., information unrelated to the drink) to be provided to each person associated with the multiple containers based on each piece of acquired information.

[0411] Furthermore, the server 410 (or multiple devices constituting it) shown in the fifth embodiment is an example of an information processing device, information processing system, or communication system that acquires information about the extracted beverage for each beverage placed in multiple containers when the extracted beverage is placed in a container or after the extracted beverage is placed in a container, and determines content to be provided to each person associated with the multiple containers based on each piece of acquired information.

[0412] Each control operation shown in the first to tenth embodiments is executed based on a program for making a computer execute each processing procedure, and each control operation can be understood as an invention of a method or a program. For example, each control operation shown in the first to tenth embodiments can be realized by storing a program related to each control operation shown in the first to tenth embodiments in a storage medium of an information processing device such as an electronic device or a server, reading the program from the storage medium, and making the information processing device execute the read program. Also, the program may be stored in an external storage medium (e.g., a CD (Compact Disc), a DVD (Digital Versatile Disc), a memory card), etc. of the information processing device and used.

[0413] The first to tenth embodiments are merely examples for realizing the invention described in the claims, and are not limited to these, and various modifications can be made without departing from the spirit of the invention. [Explanation of symbols]

[0414] 1, 200, 600, 610, 620, 630, 640, 650 Coffee Extractor 10, 210, 601, 611, 621, 631, 641, 651 Dripper 11, 211 Storage unit 12, 212 Flange part 20, 220, 320, 350, 360, 431, 441, 442, 443, 451, 452, 660, 700, 710, 720, 725, 750, 810, 840, 900~904 Electronic equipment 21, 221, 321, 481 Lenses 22, 222, 322, 482, 811 Image acquisition unit 23, 223, 323, 352, 412, 483 Control unit 24, 224, 324, 353, 413, 484 storage section 25, 225, 354, 361, 485, 841, 960 Audio output section 30, 603, 613, 623, 633, 643, 652 containers 40 Communication Systems 213 Legs 226, 355, 486 Input section 227, 356, 487, 812, 850, 911, 950 Display section 229, 357, 488 User Interface Stands 240, 602, 612, 622, 632, 642 325, 351, 411 Communications Department 358 Location information acquisition unit 400 Communication Systems 410 Server 420 Network 490 Location information setting section 604, 614, 624, 634, 644, 661 Sensors 653 Installation stand 820 Coffee Cup 830 Installation equipment 910 Display device

Claims

1. This game device obtains a plurality of image information relating to the surface of a drink stored in a container when another liquid is poured into the drink, and executes a game-related process by utilizing the transition of the other liquid appearing on the surface of the drink contained in the plurality of image information.

2. An electronic device acquires multiple image information regarding the surface of a drink when another liquid is poured into a drink stored in a container, and performs processing to output content from an output unit by utilizing the transition of the other liquid appearing on the surface of the drink contained in the multiple image information.

3. An electronic device that performs processing to output content unrelated to both the beverage and the extraction of the beverage from an output unit using image information regarding the surface of the beverage when another liquid is poured into the beverage stored in a container.

4. acquiring a plurality of image information relating to a surface of a beverage stored in a container when another liquid is poured into the beverage; and executing a process related to a game by utilizing a transition of the other liquid appearing on the surface of the drink included in the plurality of image information. Information processing methods.

5. Obtaining a plurality of image information relating to a surface of a beverage stored in a container when another liquid is poured into the beverage; and executing a process for outputting content from an output unit by utilizing a transition of the other liquid appearing on the surface of the drink included in the plurality of image information. Information processing methods.

6. An information processing method including executing a process for outputting content unrelated to both the beverage and the extraction of the beverage from an output unit using image information regarding the surface of the beverage when another liquid is poured into the beverage stored in a container.

7. acquiring a plurality of image information relating to a surface of a beverage stored in a container when another liquid is poured into the beverage; and executing a process related to a game by utilizing a transition of the other liquid appearing on the surface of the drink included in the plurality of image information. A program for causing a computer to execute the following.

8. Obtaining a plurality of image information relating to a surface of a beverage stored in a container when another liquid is poured into the beverage; and executing a process for outputting content from an output unit by utilizing a transition of the other liquid appearing on the surface of the drink included in the plurality of image information. A program for causing a computer to execute the following.

9. A program for causing a computer to execute a process for outputting content unrelated to both the beverage and the extraction of the beverage from an output unit using image information regarding the surface of the beverage when another liquid is poured into the beverage stored in a container.

Images