Cutlery-Type Information Collection Device, Information Collection and Presentation System, Method, and Program

Abstract

To provide a system which can promote improvement in an eating habit.SOLUTION: An information collection presentation system comprises: a cutlery type information collection device 2 which includes means 5 for measuring the acceleration and angular velocity of cutlery, means 6 for capturing a tip image of the cutlery and behavior analysis means 21 for analyzing the behavior about an eating habit or cooking of a user using the cutlery based on the collected data; a content presentation device 3 which includes means 32 for analyzing basic information, means 31 for setting a correspondence between contents and the behavior analysis result, basic contents and a contents generation rule, means 33 for deciding the basic arrangement of the contents based on the acquired data, means 34 for generating the contents on the basis of the behavior analysis result and a correspondence between the preset behavior analysis result and the contents setting and means 35 for presenting the contents; and a basic information input device 4 which includes means 41 for inputting the basic information about the eating habit or cooking of the user.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technology for improving people's eating habits.

Background Art

[0002] Eating habits are major lifestyle behaviors that have a profound impact on physical and mental health. However, for busy modern people, improving their eating habits is not easy. For example, it is said that the increase in digital intervention associated with the development of smartphones and mobile devices has led to unhealthy eating habits such as unbalanced diets and fast meals. On the other hand, due to the impact of the COVID-19 pandemic, people have lost the opportunity to eat together, and the frequency of eating alone has increased. To maintain a strong immune system and avoid or minimize chronic diseases and infections, it is very important to consume a balanced diet. However, eating alone is known to have some negative effects on the physical and mental health of all generations, including young people and the elderly. Specifically, people who eat alone tend to consume less fruits and vegetables and more unhealthy fast food and junk food that can be prepared quickly. Furthermore, eating alone is considered a risk factor for depression. Against this background, solutions are sought to guide people's eating habits in a better direction.

[0003] As a technology for improving eating habits, an image processing device capable of managing ingested calories is known (see Patent Document 1). This device calculates the calories from the images of food and drink, and displays an image that reduces appetite as the calorie intake increases, thereby controlling the user's appetite. In the device disclosed in Patent Document 1, the calories of a plurality of foods and drinks can be calculated respectively. However, although the total calories of each food and drink can be calculated, the calories per bite cannot be calculated, and the eating speed and the like cannot be detected either. Therefore, there is a problem that it is difficult to finely control the user's diet.

[0004] As a technology capable of calculating the calories of food and drink per mouthful, a dietary intake regulating device equipped with various sensors such as motion sensors is known in a meal carrier in the form of a general tableware (see Patent Document 2). In the device disclosed in Patent Document 2, not only can the energy intake amount be calculated, but it is also supposed that the number of chews of food during the user's meal can be detected by sensors. In the device disclosed in Patent Document 2, a configuration including a photosensor is disclosed. However, the photosensor provided in the device only detects the change in the amount of light in the tableware food carrier part and cannot detect the type of food and drink. Therefore, there is a problem that detailed information about the user's eating pattern cannot be accurately collected. Also, although it can sense the user's dietary intake status, as a method of communicating with the user, it only visually displays warnings and notifications on the user tableware interface, and there is a problem that it does not enhance the motivation for the user to improve their diet.

[0005] Despite the fact that the importance of a healthy diet with a balanced nutrition and eating slowly is widely known, regarding the question of why the implementation of a good diet is not promoted for many people, it is said that among the "Motivation", "Ability", and "Triggers" required for "Behavior" change, "Motivation" and "Triggers" are lacking. That is, although there is the "ability" to have a healthy diet, the "trigger" to start and the "motivation" to continue are insufficient, so the implementation of a good diet is not promoted.

[0006] As a technique focusing on promoting behavioral changes in eating habits, there is known a dietary life management device that recognizes an image of a eating utensil from image data acquired by a camera unit and detects and collects information regarding the user's dietary life from the image (see Patent Document 3). This device presents data such as the number of chewing times, chopstick-holding time, chopstick-placing time, and total meal time during a meal to the user, and also presents a target number of chewing times and meal time, thereby enhancing the motivation for improving the dietary life. Also, regarding the dietary life, it is assumed that advice such as "You're eating too fast" can be presented to the user. However, presenting quantitative evaluations such as the number of chewing times and meal time to the user, as in the dietary life management device disclosed in Patent Document 3, has a problem of actually lowering the user's motivation. Also, advice such as "You're eating too fast" is a qualitative content provision, but since it is an intervention method that is effective only during a specific time such as during a meal, the sustainability of the effect is low. Moreover, specific advice during a meal is for guiding the user, and there is also a risk of giving stress to the user and lowering the motivation for improving the dietary life.

[0007] Therefore, there is known an eating determination system that enables the user to improve their dietary life while enjoying it (see Patent Document 4). This is mainly targeted at children who have preferences for food, and determines the eating state from the measurement results by an acceleration sensor and a touch sensor provided on a eating utensil such as a fork. Then, according to the eating state, it is said that the motivation for the child's eating can be improved by outputting voices such as "You'll finish eating soon" and "You're eating in a good way, amazing" from the speaker of the information device. Also, in the eating determination system of Patent Document 4, different from Patent Document 3, by reproducing sounds that onomatopoeically represent the names of foods and ingredients (for example, when the food is a carrot, "nin nin nin" etc.) or sounds that make quizzes on foods and ingredients rhythmic (for example, voices such as "What color is what you're eating now? Green? Orange?"), it is possible to improve the dietary life while enjoying it. However, regarding the eating determination system of Patent Document 4 as well, like the dietary management device of Patent Document 3, it is an intervention method that only exhibits effects during specific times such as during meals, so the sustainability of the effects is low. Also, regarding content such as outputting sounds that onomatopoeically represent the names of ingredients, basically, since it is content that is directly related to meals, for example, in the case of a child who dislikes carrots, the output of content related to carrots may make the child feel more stressed and may instead reduce the motivation to improve their diet.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0009] Thus, regarding the collection of dietary information, although there are technologies that use various sensors to track eating manners and estimate meal menus using image recognition technology, it is difficult to collect detailed meal context information that focuses on each bite during a meal, such as how much of which ingredient is picked up and at what speed it is being eaten. In addition, regarding devices for presenting information content to users, many prior arts adopt quantitative visualization using charts, graphs, and statistical reports. However, such quantitative visualization may reduce the user's motivation. Also, since existing technologies adopt intervention methods that are effective only during specific times such as during meals, the sustainability of the effects is low. Furthermore, even for qualitative content, directly presenting content highly related to meals may reduce the user's motivation.

[0010] In view of such a situation, an object of the present invention is to provide a cutlery-type information collection device, an information collection and presentation system, a method, and a program that can promote the improvement of eating habits.

Means for Solving the Problems

[0011] That is, the cutlery-type information collection device of the present invention includes inertial measurement means for measuring the acceleration and angular velocity of the cutlery, camera means for imaging the tip image of the cutlery, and behavior analysis means for analyzing the behavior of the user using the cutlery related to eating habits or cooking based on the collection data collected by the inertial measurement means and the camera means. By comprehensively analyzing the motion data obtained from the inertial measurement means and the image data obtained from the camera means for imaging the tip image of the cutlery, it becomes possible to analyze the user's behavior with higher accuracy. For example, when analyzing the eating pattern, detailed eating information of the user (such as how much of what ingredients were picked up, in what order, and at what speed they were eaten) can be collected.

[0012] Here, the cutlery is not limited to tableware such as knives, forks, and spoons, but also includes chopsticks, straws, etc., and also includes cutting tools such as kitchen knives for household use.

[0013] The cutlery-type information collection system of the present invention Ka is a system composed of a cutlery-type information collection device and a content presentation device, The cutlery-type information collection device includes inertial measurement means for measuring the acceleration and angular velocity of the cutlery, camera means for imaging the tip image of the cutlery, and behavior analysis means for analyzing the behavior related to the eating habits of the user using the cutlery based on the collected data collected by the inertial measurement means and the camera means. The content presentation device includes content generation means for generating content based on the behavior analysis result by the behavior analysis means and the correspondence between the preset behavior analysis result and content setting, and means for presenting the content. And basic information analysis means for analyzing basic information which is an image in which the entire food or drink to be measured is imaged, content setting means for setting basic content of the content and setting generation rules of the content, and configuration determination means for determining the basic configuration of the content regarding the correspondence between the type of food or drink and the color scheme of the coloring, and the correspondence between the amount of food or drink and the range of the coloring based on the data obtained from the content setting means and the basic information analysis means. It is provided with. The content generated by the content generation means is content that presents visual or auditory information. Visual means still images such as paintings and photos, moving images captured by animations and video cameras, human voices, animal voices, sounds of musical instruments, natural sounds such as waves and wind, electronic sounds, etc. It also includes combinations of any of these types of information, such as games. That is, different from the direct and quantitative information presentation by graphs and the like in the prior art, it is possible to effectively enhance the user's motivation by indirectly reflecting it as a qualitative expression such as art or a story. Also, by providing the content presentation means, the user can obtain qualitative information from the content presented by the content presentation device, making it easier to present information to the user.

[0014] The information collection and presentation system of the present invention Ki Further includes a basic information input device having means for inputting basic information. ru This is preferable. By providing the basic information input device, it becomes possible to obtain the information that is the basis for generating content. As the basic information input device, terminals such as smartphones, tablet terminals, or PCs can be widely used.

[0015] When the content presentation device is provided with basic information analysis means, the basic information regarding the user's diet or cooking associated with the collected data is analyzed, enabling subsequent behavior analysis based on the collected data. By providing the content presentation device with content setting means, the content that forms the basis of content generation is determined. Also, it becomes possible to set rules according to the needs of the user. The setting of the basic content may be arbitrarily selected by the user from a plurality of pre-stored basic contents, or may be set automatically. By providing the content presentation device with configuration determination means, it becomes possible to determine the most suitable configuration in consideration of all of the base content, the set rules, and the input basic information.

[0016] In the information collection and presentation system of the present invention, cutlery is a dining utensil including chopsticks, a fork, or a spoon. Ki When the base content is line drawing content and the content is drawing content with coloring content added on the line drawing content, it is preferable to improve the user's eating habits by changing the coloring content based on the eating style of the user using the dining utensil. Thereby, the quality of the user's eating habits can be reflected as a qualitative visual expression by art or a story. For example, a painting-like picture is displayed on the content presentation device, and according to the user's detailed eating information, the degree of completion of the painting (the pieces to be painted and the painting method) changes. At this time, when a meal with a good nutritional balance is consumed in the correct way of eating, a beautiful picture is completed, and when a meal with an unbalanced nutritional balance is consumed in an unhealthy way of eating such as eating quickly, the colors to be painted are biased and the picture is not completed. By routinely looking at the content presentation device placed in the place where the user cooks or eats (kitchen or living room), a healthy eating habit is motivated, and it becomes possible for the user to feel a sense of accomplishment when consuming a well-balanced meal menu in the correct way of eating.

[0017] In the information collection and presentation system of the present invention, the configuration determination means determines the correspondence relationship between the type of food and drink and the color scheme of the coloring, the behavior analysis means analyzes the type of food and drink imaged from the tip image of the dining utensil, and the content generation means determines the color and position of the coloring content based on the type of food and drink. woDetermination su is preferably done. Thereby, the user can sense the tendency of the types of food and drink consumed by himself / herself from the qualitative content presented by the content presentation device, and can correctly enhance the motivation for a healthy diet.

[0018] In the information collection and presentation system of the present invention, the configuration determination means determines the correspondence relationship between the amount of food and drink and the coloring range, the behavior analysis means analyzes the amount of food and drink from the size of the food and drink imaged from the tip image of the eating utensil, and the content generation means determines the coloring range in the coloring content based on the amount of food and drink wo Determination su is preferably done. Thereby, the user can sense the tendency of the amount of food and drink consumed by himself / herself from the content presented by the content presentation device, and the improvement of the awareness of his / her own diet, such as refraining from overeating, is promoted.

[0019] In the information collection and presentation system of the present invention, the behavior analysis means Acceleration , Angular velocity and from the tip image of the eating utensil, analyzes at least any one of the actions of the time to carry a mouthful of food and drink to the mouth, the chewing time for a mouthful, the interval between each mouthful, and the time for a meal, and the setting of the content generation rule in the content Setting means sets a rule regarding the coloring mode including at least any one of the thickness or length of the drawn line, the accuracy of the coloring range for the target area, and the color patches in the coloring content for the action, and according to the set state, based on the calculation result, it is preferable that the coloring mode including at least any one of the thickness or length of the drawn line, the accuracy of the coloring range for the target area, and the color patches in the coloring content in the coloring content is determined. As a result, the user can become aware of their eating habits such as having a quick breakfast from the content presented by the content presentation device, and it promotes the improvement of the awareness of correct eating, such as eating slowly over time. Since such information presentation is not in the form of being shown in a graph or receiving a warning by voice as in the prior art, it does not overly stress the user and can smoothly promote the improvement of eating habits.

[0020] The information collection and presentation system of the present invention further includes an information terminal and a server. The information terminal transmits basic content to the server, and the server Content presentation transmits the basic content received from the information terminal to the device, and preferably stores the basic content and Generated the content. With the above configuration, not only the user who uses the information collection and presentation device but also other users can upload basic content to the server, and it becomes possible to collect and use a large number of basic contents from a large number of users. Also, for example, when the content is drawing content, by uploading the generated content to the server, the user can leave the progress of improving their eating habits in the form of drawing content, feel a sense of accomplishment, and can also enjoy it as their own work. The content such as the basic content and drawing content uploaded to the server may be ranked according to the number of downloads and evaluations by the user, or may be configured to give a reward to the user who uploaded it.

[0021] In the information collection and presentation system of the present invention, the cutlery is a cooking utensil including a knife, the basic information is the overall image of the food before cooking, or information regarding the name of the dish or the name of the food ingredients, the basic content is the recipe information of the dish, the configuration determination means determines a recipe suitable for the basic information based on the basic information and the recipe information, the content is cooking process content composed of at least any one of text, image, and voice, and it may support the user's cooking by changing the cooking process content based on the cooking mode of the user using the cooking utensil. With such a configuration, when cooking progresses, it becomes possible to use the content presentation device to sequentially display the next cooking method recipe.

[0022] The information collection and presentation method of the present invention includes an inertial measurement step of measuring the acceleration and angular velocity of the cutlery, and imaging the tip image of the cutlery Imaging step, based on the collection data collected by the inertial measurement step and the imaging step, analyzing the behavior of the user using the cutlery regarding ni their eating habits Behavior analysis step, and is an image in which the entire food or drink to be measured is imaged step of inputting the basic information to be used, analyzing the basic information Basic information analysis step, a content setting step of setting the basic content of the content and setting the generation rule of the content, and based on the data obtained from the content setting step and the basic information analysis step, Regarding the correspondence between the type of food or drink and the color scheme of the coloring, and the correspondence between the amount of food or drink and the range of the coloring step of determining the basic configuration of the content, By the behavior analysis step a content generation step of generating content based on the behavior analysis result and the basic configuration, and a step of presenting the content Causing the computer to perform are included.

[0023] The program of the present invention causes the computer to function as the behavior analysis means in the cutlery type information collection device of the present invention. Further, the program of the present invention causes the computer to function as all the means provided in the content presentation device in the information collection and presentation system of the present invention.

Effects of the Invention

[0024] According to the cutlery-type information collection device, information collection and presentation system, method, and program of the present invention, there is an effect that can promote the improvement of eating habits.

Brief Description of the Drawings

[0025]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0026] Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. Note that the scope of the present invention is not limited to the following examples and illustrated examples, and numerous changes and modifications are possible.

Example

[0027] FIG. 1 shows a functional block diagram of the diet improvement system of Example 1. As shown in FIG. 1, the diet improvement system 1 is composed of a cutlery-type information collection device 2, a content presentation device 3, and a basic information input device 4. The cutlery-type information collection device 2 includes an inertial measurement means 5, a camera means 6, and a behavior analysis means 21. The inertial measurement means 5 is attached to a cutlery (not shown) and measures the acceleration and angular velocity of the cutlery. The camera means 6 is also attached to the cutlery and captures an image of the tip of the cutlery. In this embodiment, the cutlery refers to chopsticks.

[0028] The behavior analysis means 21 analyzes the user's behavior related to the diet using the cutlery based on the collection data collected by the inertial measurement means 5 and the camera means 6. The content generation means 34 generates content based on the behavior analysis result by the behavior analysis means 21 and the correspondence relationship between the preset behavior analysis result and the content setting.

[0029] The content presentation device 3 includes a content setting means 31, a basic information analysis means 32, a configuration determination means 33, a content generation means 34, and a content presentation means 35. Further, the basic information input device 4 includes a basic information input means 41. The content setting means 31 sets the correspondence relationship between the content and the action analysis result, sets the basic content of the content, and sets the generation rule of the content. It includes a basic content setting means 31a for setting the basic content that is the basis of the content, and a content generation rule setting means 31b for setting the generation rule of the content. The basic information analysis means 32 analyzes the basic information regarding the user's diet or cooking. In this embodiment, the basic information is an image in which the entire food or drink to be measured is imaged. The configuration determination means 33 determines the basic configuration of the content based on the data obtained from the content setting means 31 and the basic information analysis means 32.

[0030] Figure 2 shows a configuration diagram of the diet improvement system according to Embodiment 1. As shown in Figure 2, the diet improvement system 1 is composed of a sensor-embedded chopstick 2, a digital canvas 30a, a smartphone 40, a gateway device 81, and a local network 82. The sensor-embedded chopstick 2 has an inertial measurement unit (IMU) 50 and a camera 60 attached to one of the pair of chopsticks (20a, 20b), specifically to chopstick 20b. The inertial measurement unit 50 is provided with inertial sensors (3-axis acceleration, 3-axis gyroscope, 3-axis geomagnetism) to measure the movement of chopstick 20b. Also, the camera 60 is attached so as to be able to image the tip image of chopstick 20b, and images the food ingredient 11 picked up by the chopsticks (20a, 20b). The inertial measurement unit 50 or the camera 60 may be integrated with chopstick 20b, or may be detachable by screwing or the like. For example, it may be structured such that the inertial measurement unit 50 and the camera 60 can be attached to another fork, spoon, knife, etc. according to the meal menu for use.

[0031] The inertial measurement unit 50 is connected to the gateway device 81 via Bluetooth (registered trademark), which is a wireless communication technology. The camera 60 is connected to the gateway device 81 using a wired cable. The connection method between the sensor integration device 2 and the gateway device 81 is not limited to such a method. For example, both may be connected wirelessly. The sensor integration device 20 and the gateway device 81 function as the cutlery-type information collection device 2. That is, the inertial measurement unit 50 functions as the inertial measurement means 5, and the camera 60 functions as the camera means 6. In addition, the gateway device 81 includes the behavior analysis means 21.

[0032] The digital canvas 30a functions as the content presentation device 3 and includes the content setting means 31, the basic information analysis means 32, the configuration determination means 33, the content generation means 34, and the content presentation means 35. The content presentation device 3 is not limited to the digital canvas, and display terminals such as tablet terminals can be widely used. In addition, the smartphone 40 functions as the basic information input device 4 and includes the basic information input means 41. The basic information input device 4 is not limited to the smartphone, and information terminals such as tablet terminals and PCs can be used. The gateway device 81, the digital canvas 30a, and the smartphone 40 are configured to transmit and receive data using the local network 82. The local network 82 performs wireless communication via Wi-Fi (registered trademark), but it may also be by other wireless communication methods or by wired communication.

[0033] Figure 3 shows a schematic flowchart of the dietary improvement system according to Example 1. As shown in Figure 3, first, content setting is performed using content setting means 31 (step S01). Specifically, basic content is set using basic content setting means 31a. In this example, the basic content refers to line drawing content. Although a basic color scheme is set in advance for the line drawing content, the specific color scheme is determined by configuration determination means 33. Also, although details will be described later, content generation rule setting means 31b sets the rules for content generation.

[0034] Next, basic information is input using the basic information input device 4, that is, the smartphone 40 (step S02). The basic information here refers to an image in which the entire food or drink to be measured is imaged. The input of the basic information is performed by using a camera (not shown) provided in the smartphone 40 to image an image of the entire food or drink and transmitting the image to the digital canvas 30a through the local network 82. Figure 4 is an image diagram of a meal. (1) shows the meal image of Example 1, and (2) shows the meal image of the comparative example. As shown in Figure 4(1), in the meal image 42a captured using the smartphone 40, the food ingredients (11a to 11h) are imaged on the tray 12.

[0035] The input basic information, that is, the meal image, is analyzed using basic information analysis means 32 (step S03). In the analysis of the basic information, the name of the food ingredient and the main color that constitutes the food ingredient are analyzed. As for the name of the food ingredient, it is possible to analyze only the food ingredients used in the dish, or to analyze the name of the dish in which the food ingredient is used, or to analyze both the food ingredient and the dish name. Note that image recognition technology using known machine learning is used in the analysis by the basic information analysis means 32 and the behavior analysis means 21. For example, in the meal image 42a shown in Fig. 4(1), the food ingredients 11a are "white rice (white)", the food ingredients 11b are "hijiki (black)", the food ingredients 11c are "burdock kinpira (brown)", the food ingredients 11d are "blueberries (dark blue)", the food ingredients 11e are "omelette (yellow)", the food ingredients 11f are "spinach (green)", the food ingredients 11g are "salmon (dried salmon color)", and the food ingredients 11h are "pickled plums (red)" and are analyzed as such. Thus, in the case of a meal menu using many food ingredients, food ingredients of various colors will be analyzed.

[0036] On the other hand, in the meal image 42b shown in Fig. 4(2), on the tray 12, the food ingredients (11a, 11b, 11e, 11i) are imaged. By the basic information analysis means 32, the food ingredient 11i is analyzed as "fried chicken (brown)". Thus, in the case of a meal menu using a relatively small number of food ingredients, the color scheme will have fewer types of colors.

[0037] Based on the meal image 42a captured using the basic information input device 4, that is, the smartphone 40, and the content setting by the content setting means 31, the configuration determination means 33 is used to determine the configuration (step S04). Among the content setting means 31, in the basic content setting means 31a, line drawing content that is the basis for content generation is set. The setting of the line drawing content may be to select any one from the line drawing content stored in advance in the digital canvas 30a, or to create and use the line drawing content by oneself using the smartphone 40 or the digital canvas 30a. Among the content setting means 31, in the content generation rule setting means 31b, for operations such as the time to carry a single serving of food or drink to the mouth, the time to chew a single serving, the interval between each bite, and the time for a single meal, rules regarding the thickness or length of the drawn lines, the accuracy of the coloring range for the target area, and the coloring patterns such as color spots in the coloring content can be set. Therefore, for example, if the time taken to bring a single mouthful of food or drink to the mouth exceeds a predetermined threshold value and is short, that is, it is determined that the speed of bringing it to the mouth is fast, a setting can be made such that the lines of the sketch content that will become the underpainting protrude and are colored. Also, if the time for chewing a single mouthful exceeds a predetermined threshold value and is short, that is, it is determined that the chewing is not sufficient, a setting can be made such that the colored areas bleed.

[0038] Generally, it is said that it takes about 15 minutes for the satiety center in the brain to sense an increase in blood sugar level. If the speed of bringing food or drink to the mouth is fast or the chewing is not sufficient, the speed of the entire meal will be fast, and there is a problem that the user will overeat before feeling fullness. By making the above-described settings, when eating too quickly, for the generated content, the beautiful picture will not be completed due to the color protruding or bleeding, and it is possible to indirectly encourage correct eating. From the content set in the basic content setting means 31a and the content generation rule setting means 31b and the meal image 42a captured using the smartphone 40, a specific configuration, that is, the correspondence between the type of food or drink and the color-matching for coloring, and the correspondence between the amount of food or drink and the range of coloring are determined.

[0039] FIG. 5 is an explanatory diagram of the determination of the configuration of the eating habit improvement system according to the first embodiment, where (1) is an image diagram of the digital canvas before the configuration determination, (2) is an image diagram of the basic content, and (3) is an image diagram of the completed content. As shown in FIG. 5(1), the display 36 of the digital canvas 30a before the configuration determination is blank and displays nothing. In FIG. 5(2), the sketch content which is the basic content is displayed, and mountains (51a, 51b), moon 52, clouds 53 and background 54 are drawn. In this embodiment, only one type of sketch content is displayed, but it is not limited to such sketch content, and a plurality of types of sketch content may be selectable in advance, or the user may create the sketch content by himself / herself.

[0040] In Fig. 5(3), color painting content is added to the line drawing content, and it has become a completed state as the drawing content. When the user correctly consumes a balanced diet, it is designed to be beautifully colored as shown in Fig. 5(3). Specifically, mountain 51a is red 61, mountain 51b is green 62, moon 52 is yellow 63, cloud 53 is gray 64, and background 54 is light blue 65. These color schemes are configured to have a corresponding relationship with the diet image 42a. For example, in the diet image 42a, if it is determined that the food ingredient 11g is "salmon (dried salmon color)" and the amount it occupies in all food ingredients is large, a relatively large area like mountain 51a in the line drawing content is allocated. And since "salmon" is mainly composed of colors with a reddish tint, colors with a reddish tint such as red, peach, and dried salmon color are allocated.

[0041] Also, an appropriate color scheme can be determined for each area in the line drawing content, and based on the color scheme, each food ingredient can be allocated. For example, the area determined to be mountain 51b in the line drawing content is determined to preferably have a color closer to green, and among the food ingredients extracted from the diet image 42a, the food ingredient "spinach (green)", that is, food ingredient 11f, which has a color closer to green, will be allocated. Also, the area determined to be moon 52 in the line drawing content is determined to preferably have a color closer to yellow, and among the food ingredients extracted from the diet image 42a, the food ingredient "omelette (yellow)", that is, food ingredient 11e, which has a color closer to yellow, will be allocated. Although not shown here, the correspondence relationship between the amount of food and drink and the range of color painting, such as how much color will be painted if a certain amount is eaten, has also been determined. For example, when a large amount is consumed, it will be colored over a wide range, and when a small amount is consumed, it will be colored over a narrow range.

[0042] After the content configuration is determined, data is collected using the inertial measurement means 5 and the camera means 6, that is, the inertial measurement unit 50 and the camera 60 (step S05). The following Table 1 shows the recognition results of the sensor-embedded chopsticks 20.

[0043]

Table 1

[0044] As shown in Table 1 above, it was found that the detection of the timing of bringing food to the mouth was 84%, the type of food ingested (45 foods included in the dataset) was 95%, the color of the food ingested (red, green, yellow, white, black, purple, brown) was 98%, and the size of the food ingested (S, M, L) was 93% accurate.

[0045] Based on the data collected by the inertial measurement means 5 and the camera means 6, that is, the inertial measurement unit 50 and the camera 60, the behavior of the user is analyzed (step S06). FIG. 6 shows an explanatory diagram of the data analysis of the dietary improvement system of Example 1. As shown in FIG. 6, in the inertial measurement unit 50, by detecting the peak values (13a, 13b) of the obtained acceleration data and angular velocity data, it is determined that food or drink has been brought to the mouth. In addition, in the camera 60 attached to the sensor-embedded chopsticks 20, a large number of images are captured in real time during measurement. The tip images (43a to 43g) of the sensor-embedded chopsticks 20 are part of these captured images. For example, the tip image 43a captures the state where the sensor-embedded chopsticks 20 are inserted into the bowl, and the tip images (43b, 43e) capture the state where the food 11a is picked up with the sensor-embedded chopsticks 20. Also, the tip images (43c, 43f) capture the state where the food 11a is put into the user's mouth.

[0046] The behavior analysis means 21 discriminates the type and color of the food 11a from the image in which the food 11a is picked up with the sensor-embedded chopsticks 20, such as the tip images (43b, 43e), and calculates the amount of the food 11a picked up by the sensor-embedded chopsticks 20 from the size of the captured food 11a. The action analysis means 21 determines that the user 9 has carried food or drink to the mouth when an image in which the user 9 puts the food 11a into the user's mouth and the food 11a disappears, such as the tip image (43c, 43f), is captured. That is, since the timing at which the user 9 carries food or drink to the mouth is determined using data obtained from both the inertial measurement unit 50 and the camera 60, it is possible to detect highly accurate data.

[0047] Based on the configuration determined by the configuration determination means 33 and the analysis result by the action analysis means 21, content is generated using the content generation means 34 (step S07). The content in this embodiment is drawing content in which color painting content as shown in FIG. 5(3) is added to the line drawing content shown in FIG. 5(2). The content generation means 34 generates drawing content in which color painting content is added to the line drawing content based on the type, amount, and timing of the food or drink ingested by the user 9.

[0048] After the content is generated in the content generation means 34, the content presentation means 35 presents the content (step S08). In this embodiment, the drawing content is presented on the display 36 of the content presentation device 30a. FIG. 7 is an image diagram of content presentation of the dietary life improvement system according to the first embodiment, where (1) shows the state before a meal, (2) shows the state immediately after the start of the meal, (3) shows the state after a certain period of time has elapsed since the start of the meal, and (4) shows the state at the end of the meal. When the user has not put anything in the mouth yet, as shown in FIG. 7(1), only the line drawing content is displayed on the content presentation device 30a. When the user starts eating, as shown in FIGS. 7(2) or (3), colored content is added to the line drawing content. Specifically, in this embodiment, first, when the user eats 11 g of food (salmon, dried salmon color), as shown in FIG. 7(2), the mountain 51a is painted red 61a. Next, when the user eats an additional 11 g of food (salmon, dried salmon color), as shown in FIG. 7(3), the mountain 51a is further painted red 61b. Similarly, when the user eats 11 f of food (spinach, green), the mountain 51b is painted green 62, and when the user eats 11 e of food (omelette, yellow), the moon 52 is painted yellow 63.

[0049] Generally, there is a known psychological phenomenon that people tend to remember more strongly things that they could not achieve or things that were interrupted than things they have achieved (Zeigarnik effect). From the state where the digital canvas 30a shown in FIG. 7(1) is not colored to the state where a little color is applied as shown in FIG. 7(2), it is considered that the user's awareness of the uncolored areas becomes stronger, and an effect is obtained in which the user has a stronger awareness of wanting to color the uncolored areas. For example, among the food menu, if the user does not eat the food they dislike, the area assigned to the food will not be colored. However, due to this psychological effect, it becomes possible to make the user aware that they want to eat even the food they dislike to complete the picture. Also, there is a known psychological phenomenon that when people try to achieve a goal, their motivation improves when they feel that they are moving forward towards the goal (Endowment Progress effect). From the state where the digital canvas 30a shown in FIG. 7(1) is not colored to the state where a little color is applied as shown in FIG. 7(2), and further to the state where more color is applied as shown in FIG. 7(3), it can be expected that the user will feel that they are moving forward towards the goal and their motivation to complete the drawing content will improve.

[0050] In addition to having a balanced diet, it is also important to eat in the right way, such as eating slowly. Therefore, as described above, in the content generation rule setting means 31b, when correct eating habits are not followed, settings are made such that coloring is not beautifully done. This makes it possible to change the way of coloring the coloring content added to the line drawing content according to the user's eating speed and degree of chewing. FIG. 8 is an image diagram of content presentation of the eating habit improvement system according to the first embodiment, where (1) shows the case where the color bleeds out, and (2) shows the case where the color bleeds. In the content generation rule setting means 31b, when the time to carry a mouthful of food or drink to the mouth exceeds a predetermined threshold value and is judged to be short, that is, the speed of carrying it to the mouth is fast, if settings are made such that the lines of the preliminary sketch line drawing content bleed out and are colored, then in the behavior analysis means 21, for example, when it is determined that the user carried 11 g of food material (salmon, dried salmon color) to the mouth at a high speed, as shown in FIG. 8(1), the mountain 51a is painted with red 61c, but the end of the red 61c is painted so as to protrude beyond the line representing the mountain 51a.

[0051] Also, in the content generation rule setting means 31b, when the chewing time for one mouthful exceeds a predetermined threshold value and is judged to be short, that is, the chewing is not sufficient, if settings are made such that the colored area bleeds, then in the behavior analysis means 21, for example, when it is determined that after the user carried 11 g of food material (salmon, dried salmon color) to the mouth at a high speed, the user did not chew sufficiently and then picked up chopsticks for another food material or carried another food material to the mouth, as shown in FIG. 8(2), the mountain 51a is painted with red 61c, but the end of the red 61c will bleed. In this way, when the user finishes a meal in a good balance and slowly over time, as shown in FIG. 7(4), beautiful coloring is applied to the line drawing content, and the drawing content is completed.

[0052] (Evaluation experiment on the effectiveness of the eating habit improvement system) An evaluation experiment was conducted on the effectiveness of the eating habit improvement system of Example 1. Figure 14 shows the results of the evaluation experiment on the effectiveness of the eating habit improvement system. The horizontal axis represents each subject (P 1 ~P 10 ), and the vertical axis represents the meal time (minutes) until finishing eating a single meal. The subjects (P 1 ~P 10 ) are all men in their 20s with a tendency to eat quickly. "NI" in the figure means "without intervention", "GI" means "intervention using a graph", "PI 1 " means "intervention using a painting", and "PI 2 " means "intervention using a painting and having a color mixing effect". Figure 15 shows an image diagram of the intervention used in the evaluation experiment. (1) shows the image of "intervention using a graph", (2) shows the image of "intervention using a painting", and (3) shows the image of "intervention using a painting and having a color mixing effect". That is, "intervention using a graph" is to present a graph as shown in Figure 15(1) to the subject, and when taking a single bite of each color of food or drink, a scale is displayed on the graph. For example, in Figure 15(1), it shows that 5 bites of "red" food or drink have been taken.

[0053] "Intervention using a painting" is to present drawing content as shown in Figure 15(2), and there is no "color mixing effect". For example, when the mountain 51h is painted, it is painted in bright red 61. On the other hand, "having a color mixing effect" means that in the content generation rule setting means 31b, a setting is made such that "when the time interval for carrying a single bite of food or drink to the mouth exceeds a predetermined threshold value, that is, when it is determined that the eating speed of a single bite is fast, the color to be painted will mix with other colors". Specifically, as in the drawing content shown in Figure 15(3), even when the mountain 51h is being painted, it does not necessarily become bright red 61. For example, when it is determined that the eating speed of a single bite is fast, the color to be painted will mix with other colors (here, blue), and an intervention such as being painted in purple 66 will be made.

[0054] Under the above conditions, dietary data for each subject were collected a total of four times (each on a different day). In order to suppress differences in physical conditions such as fasting levels, subjects were requested in advance to be in a fasting state for the four experiments. Generally, it is said that it is desirable to take more than 15 minutes for a single meal. From the results, it can be observed that in the subjects (P 4 ~P 9 ), the meal time was slightly increased by the PI 1 intervention. Furthermore, it was confirmed that due to the PI 2 intervention, in most of the subjects, namely subjects (P 1 ~P 6 ,P 8 ~P 10 ), the meal time exceeded 15 minutes, which is the standard for an ideal meal time. Also, interestingly, it was confirmed that the PI 2 intervention was more effective especially for participants with a tendency to eat quickly (within 10 minutes under the condition without intervention). As a result of the Wilcoxon signed-rank test, no significant difference was confirmed between the NI and GI conditions, but significant differences were confirmed between NI and PI 1 , and between NI and PI 2 . From this, it is considered that the qualitative visualization using the art of this example has a higher behavior modification effect than the visualization using a graph.

Example

[0055] Figure 9 shows the configuration diagram of the dietary improvement system of Example 2. As shown in Figure 9, the dietary improvement system 1a is composed of the sensor-integrated chopsticks 2, digital canvases (30a, 30b), smartphone 40, gateway device 81, and local network 82, and is the same as the dietary improvement system 1 of Example 1 except that it is provided with the digital canvas 30b. The digital canvas 30b is composed of a display with a larger screen than the digital canvas 30a. FIG. 16 shows an example of the content used in the digital canvas 30a, where (1) represents line drawing content and (2) represents drawing content. FIG. 17 shows an example of the content used in the digital canvas 30b, where (1) represents line drawing content and (2) represents drawing content. The line drawing content in FIG. 17(1) is larger in size and more complexly depicted than the line drawing content in FIG. 16(1), so the number of areas to be drawn is larger. As shown in the drawing content in FIG. 17(2), it enables more diverse color combinations and a wider range of coloring compared to the drawing content in FIG. 16(2). Utilizing such differences, the digital canvas 30a presents drawing content for one meal of the user, while the digital canvas 30b presents drawing content for one week of the user.

[0056] FIG. 10 is an image diagram showing content presentation of the diet improvement system according to the second embodiment, where (1) shows the state before a meal, (2) shows the state after one meal is finished, and (3) shows the state after one week of meals is finished. As shown in FIG. 10(1), on the digital canvas 30b, the line drawing content for coloring for one week is larger in size than the line drawing content shown in FIG. 8(1), and mountains (51c to 51g), moon 52, clouds (53a, 53b), background 54, trees 55, and river 56 are displayed. When one meal is finished from the state before the meal, as shown in FIG. 10(2), it becomes a state where a part of a single picture is painted. Here, a part of mountain 51c is painted red 61, mountain 51e is painted yellow - green 69, and river 56 is painted light blue 65. However, for example, one area such as mountain 51c can be assigned for each meal.

[0057] When coloring content for one week, that is, 21 servings, as shown in FIG. 10(3), a painting as large drawing content with coloring content applied on the line drawing content is completed on the digital canvas 30b. Here, the mountain 51c is painted red 61, the mountain 51d is painted light yellow 67, the mountain 51e is painted yellow-green 69, the mountain 51f is painted purple 66, the mountain 51g is painted green 62, the moon 52 is painted yellow 63, the clouds (53a, 53b) are painted gray 64, the background 54 is painted light blue 65, the trees 55 are painted brown 68, and the river 56 is painted light blue 65. In this way, by not only presenting the content for each day but also presenting the content completed on a weekly basis using another content presentation device, it is possible to experience short-term goals, pleasures, and a sense of achievement, as well as medium- and long-term goals, pleasures, and a sense of achievement, and continuously improve the diet.

Example

[0058] FIG. 11 shows a functional block diagram of the diet improvement system according to Example 3. As shown in FIG. 11, the diet improvement system 10 according to Example 3 includes the diet improvement system 1 according to Example 1, a server 7, and an information terminal 9, and can transmit and receive data via the Internet 8. The information terminal 9 may be used by a user who uses the diet improvement system 1 or by other users. As the information terminal 9, a wide range of terminals such as a smartphone, a tablet terminal, and a PC can be used. Note that one information terminal such as a smartphone may be used as the basic information input device 4 and also as the information terminal 9.

[0059] FIG. 12 shows a configuration diagram of the diet improvement system according to Example 3. As shown in FIG. 12, the diet improvement system 10 according to Example 3 includes the server 7 and a PC 90 in addition to the configuration of the diet improvement system 1 according to Example 1, and can transmit and receive data via the Internet 8.

[0060] Figure 13 shows a schematic flowchart of the dietary improvement system of Example 3. As shown in Figure 13, first, using the information terminal 9, basic content is uploaded to the server 7 (step S11). The basic content may be line drawing content that is the basis of the drawing content as in Example 1, or it may be a self-made cooking recipe described in images or text, or it may be audio played by various musical instruments, etc. The upload of the basic content may be performed only for the purpose of the user's own use, or it may be uploaded for the purpose of allowing other users who use the dietary improvement system 10 to use it.

[0061] Next, the dietary improvement system 1 downloads the basic content from the server 7 (step S12). Specifically, it downloads to the content presentation device 3 and stores the data in a storage device (not shown). When making this configuration determination, the basic content stored in the content presentation device 3 is read. The dietary improvement system 1 performs data collection and content presentation by the same method as in Example 1 (step S13).

[0062] The dietary improvement system 1 uploads the generated content to the server 7 (step S14). For example, the user can periodically upload the drawing content generated and presented by eating to the server 7 to accumulate data. Thereby, the user can leave the progress of improving their eating habits in the form of drawing content, feel a sense of accomplishment, and also enjoy it as their own work. The content uploaded by the user can also be used, such as being made public to other users via the Internet 8. By receiving evaluations from other users, the user's motivation to improve their diet can be further enhanced. Note that it may be a specification to make the download of the basic content paid and pay a predetermined reward from the service provider to the user who uploaded the basic content.

[0063] (Other Examples) 1) Figure 18 shows an image diagram of content presentation of a dietary life improvement system according to another embodiment. In Example 2, the device for displaying drawing content for one meal and the device for displaying drawing content for one week were separated into two digital canvases (30a, 30b). However, different from such a configuration, in one digital canvas 30a, both pieces of drawing content may be displayed. Figure 18 shows an example in which drawing content for one week is displayed on the digital canvas 30a. 2) The content presented on the content presentation device may be designed using characters for children and used as a food education support system. 3) By changing the purpose of the information collection side from the specific recognition of food ingredients to the detailed description of the actions when picking up a ball with chopsticks, etc., it may also be used as a rehabilitation (delicate training) support system for the elderly that can be used in the medical and nursing care fields. Delicate training is training for practicing actions using fine fingertips, and rehabilitation using chopsticks that can stimulate various muscles in the hand is the mainstream. By making such chopsticks sensorized, the progress and improvement degree of the actions of the training implementer can be collected as specific data.

Industrial Applicability

[0064] The present invention is useful for a system for promoting a healthy dietary life targeting the adult population, a food education support system for children, and a rehabilitation training support system for the elderly.

Explanation of Reference Numerals

[0065] 1, 1a, 10 Dietary life improvement system 2 Spoon-type information collection device 3 Content presentation device 4 Basic information input device 5 Inertial measurement means 6 Camera means 7 Server 8 Internet 9 Information terminal 11, 11a~11i Food ingredients 12 Tray 13a, 13b Peak values 20 Sensor-Embedded Chopsticks Chopsticks 20a, 20b 21 Behavior Analysis Means Digital Canvas 30a, 30b 31 Content Setting Means 31a Basic Content Setting Means 31b Content Generation Rule Setting Means 32 Basic Information Analysis Means 33 Configuration Determination Means 34 Content Generation Means 35 Content Presentation Means 36 Display 40 Smartphone 41 Basic Information Input Means Meal Images 42a, 42b Tip Images 43a~43g 50 Inertial Measurement Unit Mountains 51a~51h 52 Moon Clouds 53, 53a, 53b 54 Background 55 Trees 56 River 60 Camera Red 61, 61a~61d 62 Green 63 Yellow 64 Gray 65 Light Blue 66 Purple 67 Light Yellow 68 Brown 69 Yellow-Green 81 Gateway Device 82 Local Network 90 PC

Claims

1. A system composed of a cutlery-type information collection device and a content presentation device, wherein the cutlery-type information collection device has inertial measurement means for measuring the acceleration and angular velocity of the cutlery, camera means for imaging the tip image of the cutlery, and behavior analysis means for analyzing the behavior related to the eating habits of the user using the cutlery based on the collection data collected by the inertial measurement means and the camera means, and is equipped with, wherein the content presentation device has content generation means for generating content based on the behavior analysis result by the behavior analysis means and the correspondence relationship between the preset behavior analysis result and content setting, content presentation means for presenting the content, basic information analysis means for analyzing basic information which is an image in which the entire food and drink to be measured is imaged, content setting means for setting the basic content of the content and the generation rule of the content, and configuration determination means for determining the basic configuration of the content regarding the correspondence relationship between the type of food and drink and the color scheme of the coloring, and the correspondence relationship between the amount of food and drink and the range of the coloring based on the data obtained from the content setting means and the basic information analysis means, An information collection and presentation system characterized by comprising the above.

2. The information collection and presentation system according to claim 1, further comprising a basic information input device provided with means for inputting the basic information.

3. The cutlery is a tableware including chopsticks, a fork or a spoon, the basic content is line drawing content, and the content is drawing content with coloring content added on the line drawing content, The information collection and presentation system according to claim 1, characterized in that the eating habits of the user are improved by changing the coloring content based on the eating pattern of the user using the tableware.

4. The configuration determination means determines the correspondence relationship between the type of food and drink and the color scheme of the coloring, the behavior analysis means analyzes the type of food and drink imaged from the tip image of the tableware, The information collection and presentation system according to claim 3, characterized in that the content generation means determines the color and position of the coloring content based on the type of food and drink.

5. The configuration determination means determines the correspondence relationship between the amount of food and drink and the range of the coloring, The action analysis means analyzes the amount of food and drink from the size of the imaged food and drink in the tip image of the eating utensil. The content generation means determines the coloring range in the coloring content based on the amount of food and drink, according to the information collection and presentation system described in claim 3 or 4.

6. The action analysis means analyzes at least one of the actions including the time to carry a mouthful of food and drink to the mouth, the chewing time for one mouthful, the interval between each mouthful, and the time for one meal, from the acceleration, the angular velocity, and the tip image of the eating utensil. The setting of the content generation rule in the content setting means sets a rule regarding the coloring mode including at least one of the thickness or length of the drawn line, the accuracy of the coloring range for the target area, and the color patches in the coloring content for the actions, and determines the coloring mode including at least one of the thickness or length of the drawn line, the accuracy of the coloring range for the target area, and the color patches in the coloring content based on the analysis result of at least one of the actions including the time to carry a mouthful of food and drink to the mouth, the chewing time for one mouthful, the interval between each mouthful, and the time for one meal by the action analysis means according to the set state, according to the information collection and presentation system described in any one of claims 3 to 5.

7. Further comprising an information terminal and a server. The information terminal transmits the basic content to the server. The server transmits the basic content received from the information terminal to the content presentation device and stores the basic content and the generated content. The information collection and presentation system according to any one of claims 1 to 6, characterized in that.

8. An inertial measurement step of measuring the acceleration and angular velocity of the cutlery. An imaging step of imaging the tip image of the cutlery. An action analysis step of analyzing the actions related to the eating habits of the user using the cutlery based on the collected data collected by the inertial measurement step and the imaging step. A step of inputting basic information which is an image in which the entire food and drink to be measured is imaged. A basic information analysis step of analyzing the basic information. A content setting step of setting the basic content of the content and setting the generation rule of the content. Based on the content setting step and the data obtained from the basic information analysis step, a step of determining the basic composition of the content regarding the correspondence between the type of food and drink and the color painting color scheme, and the correspondence between the amount of food and drink and the range of the color painting; A content generation step of generating the content based on the behavior analysis result from the behavior analysis step and the basic composition; A step of presenting the content; An information collection and presentation method characterized by causing a computer to perform the above.

9. A program for causing a computer to function all the means provided in the content presentation device in the information collection and presentation system according to any one of Claims 1 to 7.

Images