Method for constructing phytochemical database and phytochemical measurement system including the same

Abstract

To provide a method for constructing a database useful for measuring the content of phytochemical components of a green salad dish based on an image of the food dish.SOLUTION: Detecting a color region corresponding to a color of each green salad ingredient in an image of the food to obtain an area of each color region, calculating an area ratio of each color region to a total color region, obtaining a reference content of a phytochemical ingredient per unit area ratio of each food ingredient from the area ratio and a content of the phytochemical ingredient of each food ingredient calculated from a predetermined content of the phytochemical ingredient per unit weight of each food ingredient and a used amount of each food ingredient, and calculating a mean value of reference contents for a plurality of the foods; The item and color of the food material, the kind name of the phytochemical component contained in the food material and the average value of the reference content are registered in a phytochemical database in association with each other.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a phytochemical database, in particular, a method for constructing a phytochemical database used in measuring the phytochemical component content of a vegetable salad dish based on an image of the vegetable salad dish, and a phytochemical measurement system equipped with the phytochemical database. [Background technology]

[0002] In recent years, it has become common for customers at restaurants and other food service establishments to take photos of the food they have ordered with their smartphones and upload the images to social networking sites such as Instagram.

[0003] In addition, health consciousness is on the rise, and more and more people are paying attention to the nutritional value and calorie content of food when eating.

[0004] Furthermore, phytochemicals, which are plant-derived chemical substances, have recently been attracting attention for their beneficial properties in maintaining human health, such as antioxidant effects, and there has been a gradual increase in customer demand at restaurants to know what phytochemical components are contained in the food they order.

[0005] Therefore, if a customer at a restaurant could simply take a photo of the vegetable salad dish they ordered with their smartphone and easily find out what phytochemical components are contained in that dish and in what amounts, this would be extremely convenient for the customer and could also serve as a topic of conversation that could spark lively conversation.

[0006] In addition, restaurants can expect to attract more customers by adding entertainment value to their meals.

[0007] However, when trying to achieve this, in conventional technology, the phytochemical component content per unit weight (100g) of each ingredient (vegetable) before cooking has already been compiled into a database (see, for example, Non-Patent Document 1), but for the following reasons, it is not possible to measure the phytochemical component content based on an image of a vegetable salad dish simply by using the data in this database.

[0008] In other words, if you simply want to calculate the phytochemical content of a vegetable salad dish, you just need to multiply the amount of each ingredient used in the vegetable salad dish by the data in the database above.

[0009] On the other hand, if we start with an image of a vegetable salad dish, first of all, a vegetable salad dish is not one in which each ingredient is individually separated and arranged two-dimensionally on a plate; rather, each ingredient is washed, chopped, and arranged on a plate. Therefore, in the photographed image, the ingredients will be bent or curled and stacked on top of each other, or adjacent different ingredients will be stacked on top of each other.

[0010] Therefore, for example, the area of ​​the image portion of each ingredient detected from this image does not necessarily correspond one-to-one to the amount of that ingredient used.

[0011] Furthermore, even if the same vegetable salad dish is photographed with the same smartphone (camera), the shooting distance (the distance from the dish to the camera) and the tilt of the dish within the camera's field of view differ for each photographer (customer), so the area of ​​the image portion of each ingredient detected from the image differs for each photographer.

[0012] Therefore, it is necessary to construct a new database that takes into consideration the relationship between the processed image data and the data in the existing database. [Prior art documents] [Non-patent literature]

[0013] [Non-Patent Document 1] National Agriculture and Food Research Organization / Food Research Division / Functional ingredient content data (https: / / www.naro.go.jp / laboratory / nfri / contens / ffdb / ffdb.html) Summary of the Invention [Problem to be solved by the invention]

[0014] Therefore, an object of the present invention is to provide a method for constructing a database useful for measuring the phytochemical component content of a vegetable salad dish based on an image of the vegetable salad dish.

[0015] Another object of the present invention is to provide a phytochemical measurement system equipped with a database constructed by this method. [Means for solving the problem]

[0016] In order to solve the above problems, according to the present invention, there is provided a method for constructing a phytochemical database to be used in measuring the phytochemical content of a vegetable salad dish based on an image of the vegetable salad dish, the method comprising the steps of: (1) determining at least one type of phytochemical component to be measured for each ingredient of the vegetable salad dish, and estimating the content of the phytochemical component to be measured for each ingredient in the vegetable salad dish from the predetermined content of the phytochemical component to be measured per unit weight of each ingredient and the amount of each ingredient used; (2) classifying the ingredients of the vegetable salad dish by color and setting a plurality of filters in the HSV color space that can individually detect the colors; and (3) photographing the vegetable salad dish, applying each of the plurality of filters to the obtained image, and detecting color areas in the image that correspond to the filters. The method includes the steps of: (1) measuring the area of ​​each detected color region and dividing the area of ​​each color region by the sum of the areas of the color regions to determine the area ratio of each color region; (2) dividing the estimated content by the corresponding area ratio for each color region and for each type of phytochemical component to be measured to determine the standard content of the phytochemical component to be measured in the food material corresponding to the color region; (3) sequentially performing steps (4) to (5) for each of the plurality of vegetable salad dishes to determine the average value of the standard content; and (4) correlating the type of food material and the color of the corresponding color region with the type name of the phytochemical component to be measured contained in the food material and the average value of the standard content, and registering them in the phytochemical database.

[0017] Furthermore, in order to solve the above-mentioned problems, according to the present invention, there is provided a system for measuring the phytochemical content of a vegetable salad dish based on an image of the vegetable salad dish, the system comprising: a server equipped with a phytochemical database constructed by the above-mentioned method; and at least one user terminal equipped with a camera and connected to the server for data communication, wherein the server comprises an image receiving unit that receives image data of the vegetable salad dish from the user terminal; a color region detecting unit that applies a plurality of filters in an HSV color space that can individually detect the colors of ingredients of the vegetable salad dish to the image data, and detects color regions in the image that correspond to the filters; and a color region detecting unit that measures the area of ​​each color region detected by the color region detecting unit and divides the area of ​​the color region by the sum of the areas of all color regions. a measuring unit that searches the phytochemical database based on the color of each color area detected by the color area detection unit, extracts the corresponding food item and the name of the type of phytochemical component to be measured contained in the food item, and the average value of the standard content of the phytochemical component, and calculates the actual measured content of the phytochemical component by multiplying the average value of the standard content by the area ratio of the detected color area; and a measurement result transmitting unit that transmits the data on the food item, the name of the type of phytochemical component to be measured contained in the food item, and the calculated actual measured content of the phytochemical component extracted by the measuring unit to the user terminal. [Effects of the Invention]

[0018] According to the present invention, the ingredients of a vegetable salad dish are classified by color, color areas corresponding to the color of each ingredient are detected in an image of the dish to determine the area of ​​each color area, and the proportion of the area (area ratio) that each color area accounts for to the total color area is calculated.The phytochemical component content per unit area ratio (standard content) of each ingredient is determined from this area ratio, the predetermined content of phytochemical components per unit weight of each ingredient, and the content of phytochemical components of each ingredient calculated from the amount of each ingredient used.Furthermore, multiple versions of the dish are prepared, and the standard content of phytochemical components of each ingredient is determined as before, and the average value of the standard contents is calculated.The type of ingredient and the color of the corresponding color area are associated with the type name of the phytochemical component contained in the ingredient and the average value of the standard content, and registered in a phytochemical database.

[0019] In this way, the image of a vegetable salad dish is processed, the color areas of each ingredient in the dish are detected, and the area ratio of each color area is calculated. The color of the ingredients is then used to search a phytochemical database to extract the type names and average standard contents of the relevant phytochemical components. The extracted average standard contents are then multiplied by the area ratio to calculate the measured value of the phytochemical component content (actual measured content), thereby measuring the type and content of the phytochemical components contained in each ingredient.

[0020] In this case, a physical quantity called the area ratio of the color regions is introduced, and the content of phytochemical components contained in the ingredients is calculated (measured) through the area ratio.Therefore, even if images are obtained with different magnifications (reductions) and different tilts of the dish within the camera's field of view for each photographer (customer), there is little variation in the measured values ​​of the phytochemical component content, making it possible to perform measurements that are sufficiently satisfactory for adding entertainment to meals (it is sufficient to know the types of phytochemical components contained in vegetable salad dishes and the approximate content percentages of those phytochemical components). [Brief explanation of the drawings]

[0021] [Figure 1]FIG. 1 is a flow diagram illustrating a method for constructing a phytochemical database according to one embodiment of the present invention. [Figure 2] (A) is a table showing the ingredients used in vegetable salad dishes to which the method of Figure 1 is applied and the predetermined content of phytochemical components to be measured per 100g of each ingredient, and (B) is a table showing the estimated content of phytochemical components to be measured for each ingredient shown in (A). [Figure 3] (A) is an image of a sample vegetable salad dish to which the method of Figure 1 is applied, (B) is a diagram showing the color areas detected by applying a green filter to the image of (A), and (C) is a pie chart showing the area ratios of the color areas detected in the image of (A). [Figure 4] FIG. 1 shows the structure of a phytochemical database constructed by the method of the present invention. [Figure 5] FIG. 1 is a block diagram showing the schematic configuration of a phytochemical measurement system equipped with a phytochemical database constructed by the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, the configuration of the present invention will be described based on a preferred embodiment with reference to the accompanying drawings. FIG. 1 is a flow diagram illustrating a method for constructing a phytochemical database according to one embodiment of the present invention.

[0023] Referring to Figure 1, according to the present invention, first, at least one type of phytochemical component to be measured is determined for each ingredient in a vegetable salad dish, and the estimated content of the phytochemical component to be measured for each ingredient in the vegetable salad dish is calculated based on the predetermined content of the phytochemical component to be measured per unit weight of each ingredient and the amount of each ingredient used (step S1 in Figure 1).

[0024] In this case, the predetermined content of phytochemical components can be obtained from publicly known literature or websites (e.g., https: / / www.naro.go.jp / laboratory / nfri / contens / ffdb / ffdb.html).

[0025] In this example, a sample of vegetable salad dish as shown in FIG. 3(A) was prepared. As can be seen from this diagram, the vegetable salad dish uses seven ingredients: tomatoes, carrots, spinach, yellow peppers, red cabbage, burdock, and onions.

[0026] Figure 2(A) is a table showing the ingredients used in the vegetable salad dish of Figure 3(A) and the predetermined content of phytochemical components to be measured per 100g of each ingredient.

[0027] For example, for tomatoes, the phytochemical component to be measured is lycopene, and the specified content of lycopene per 100 g of tomatoes is 3.53 mg; for carrots, the phytochemical components to be measured are lycopene and beta-carotene, and the specified contents of lycopene and beta-carotene per 100 g of carrots are 0.02 mg and 8.60 mg, respectively.

[0028] Figure 2(B) is a table showing the estimated contents of phytochemical components to be measured for each of the food ingredients shown in Figure 2(A).

[0029] As shown in Figure 2(B), in this example, the amount used for each ingredient in the vegetable salad dish is 50.00 g. Therefore, for example, for tomatoes, the estimated lycopene content is 3.53 mg × (50.00 g / 100 g) = 1.77 mg, and for carrots, the estimated lycopene content is 0.02 mg × (50.00 g / 100 g) = 0.01 mg, and the estimated beta-carotene content is 8.60 mg × (50.00 g / 100 g) = 4.30 mg.

[0030] According to the present invention, the ingredients of the vegetable salad dish are then classified by color, and a plurality of filters in the HSV color space that can individually detect the colors are set (step S2 in FIG. 1).

[0031] This step S2 and the subsequent step S3 are performed using a personal computer on which an appropriate known image processing program (in this embodiment, OpenCV manufactured by Intel Corporation is used) is installed.

[0032] The classification of ingredients by color is performed so that different types of ingredients are classified into different colors. Some food ingredients may have different colors in parts, for example, the color at the base and the tip of the leaves may be clearly different. In such cases, the color that occupies the largest proportion of the food ingredient's overall area is used as the color of that food ingredient.

[0033] As can be seen from Figures 2(A), (B) and 3(A), in this embodiment, food ingredients are classified into seven colors: red (tomato), orange (carrot), green (spinach), yellow (yellow pepper), purple (red cabbage), dark brown (burdock) and white (onion), and correspondingly, seven types of filters are set: red color filters, orange color filters, green color filters, yellow color filters, purple color filters, black color filters and white color filters.

[0034] Specifically, the filter is set by appropriately setting the ranges of the color value, brightness value, and saturation value.

[0035] According to the present invention, the vegetable salad dish is then photographed, and the above-mentioned multiple filters are each applied to the acquired image to detect color areas in the image corresponding to the filters. The area of ​​each detected color area is measured, and the area of ​​each color area is divided by the sum of the areas of the color areas to determine the area ratio (percentage) of each color area (step S3 in Figure 1).

[0036] FIG. 3B is a diagram showing a state in which a green color region is detected by applying a green color filter to the sample image of FIG. 3A. FIG. 3(C) shows a pie chart of the area ratio of each color region calculated for the sample image of FIG. 3(A).

[0037] According to the present invention, the standard content of the phytochemical component to be measured in the food material corresponding to the color region is further determined by dividing the estimated content by the corresponding area ratio for each color region and for each type of phytochemical component to be measured (step S4 in Figure 1).

[0038] According to the present invention, steps S3 and S4 are further carried out in sequence for each of the plurality of vegetable salad dishes to determine the average value of the reference contents (step S5 in FIG. 1).

[0039] In this case, it is preferable to prepare as many vegetable salad dishes as possible, and it is also preferable to change the shooting distance (the distance between the camera and the dish) and the position (rotation angle) of the dish relative to the center of the camera lens for each photograph of a vegetable salad dish.

[0040] Then, the type of food material and the color of the corresponding color area are associated with the name of the type of phytochemical component contained in the food material to be measured and the average value of the standard content of the phytochemical component, and are registered in the phytochemical database (step S6 in Figure 1).

[0041] FIG. 3 shows the structure of the phytochemical database constructed in this way.

[0042] In this way, the image of a vegetable salad dish is processed, the color areas of each ingredient in the dish are detected, and the area ratio of each color area is calculated. The color of the ingredients is then used to search a phytochemical database to extract the type names and average standard contents of the relevant phytochemical components. The extracted average standard contents are then multiplied by the area ratio to calculate the measured value of the phytochemical component content (actual measured content), thereby measuring the type and content of the phytochemical components contained in each ingredient.

[0043] In this case, a physical quantity called the area ratio of the color regions is introduced, and the content of phytochemical components contained in the ingredients is calculated (measured) through the area ratio.Therefore, even if images are obtained with different magnifications (reductions) and different tilts of the dish within the camera's field of view for each photographer (customer), there is little variation in the measured values ​​of the phytochemical component content, making it possible to perform measurements that are sufficiently satisfactory for adding entertainment to meals (it is sufficient to know the types of phytochemical components contained in vegetable salad dishes and the approximate content percentages of those phytochemical components).

[0044] FIG. 4 is a block diagram showing the schematic configuration of a phytochemical measurement system equipped with a phytochemical database constructed by the method of the present invention.

[0045] Referring to Figure 4, the phytochemical measurement system of the present invention includes a server 1 having a phytochemical database 2 constructed by the above-mentioned method, and at least one user terminal 4 connected to the server 1 via a communication network 3 and equipped with a camera.

[0046] The server 1 is equipped with an image receiving unit 5 that receives image data of a vegetable salad dish from a user terminal 4, a color area detection unit 6 that applies a plurality of filters in the HSV color space that can individually detect the colors of the ingredients of the vegetable salad dish to the image data to detect color areas in the image that correspond to the filters, and an area ratio calculation unit 7 that measures the area of ​​each color area detected by the color area detection unit 6 and calculates the area ratio of each color area by dividing the area of ​​each color area by the sum of the areas of all color areas.

[0047] The server 1 further includes a measurement unit 8 that searches the phytochemical database 2 for each color area detected by the color area detection unit 6 based on the color of the color area, extracts the corresponding food item, the name of the type of phytochemical component to be measured contained in the food item, and the average value of the standard content of the phytochemical component, and calculates the actual measured content of the phytochemical component by multiplying the average value of the standard content by the area ratio of the detected color area, and a measurement result transmission unit 9 that transmits the data on the food item, the name of the type of phytochemical component to be measured contained in the food item, and the calculated actual measured content of the phytochemical component extracted by the measurement unit 8 to the user terminal.

[0048] The phytochemical measurement system of the present invention is provided, for example, for each restaurant serving food, and if multiple types of vegetable salad dishes are served, multiple phytochemical databases 2 are provided in the server 1 accordingly.

[0049] In this way, when a customer takes a photo of the vegetable salad dish they have ordered using the camera on their user terminal 4 (usually the customer's smartphone) and sends the acquired image to the server 1 of the phytochemical measurement system, the server 1 sends to the user terminal 4 the name of the type of phytochemical component to be measured for each color region (ingredient) and data on the actual measured content.

[0050] The data received from the server 1 is then processed appropriately in the user terminal 4, and the type and color of the food ingredients, as well as the type and amount of phytochemical components contained in the food ingredients, are displayed on the user terminal 4, for example, in the form of a pie chart. [Explanation of symbols]

[0051] 1 Server 2 Phytochemical Database 3. Communication Network 4. User terminal 5 Image receiving unit 6 Color region detection section 7 Area ratio calculation section 8 Measuring part 9 Measurement result transmission unit

Claims

1. A method for constructing a phytochemical database to be used in measuring the phytochemical component content of a vegetable salad dish based on an image of the vegetable salad dish, comprising: (1) determining at least one type of phytochemical component to be measured for each ingredient of the vegetable salad dish, and calculating an estimated content of the phytochemical component to be measured in each ingredient of the vegetable salad dish from a predetermined content of the phytochemical component to be measured per unit weight of each ingredient and the amount of each ingredient used; (2) classifying ingredients of the vegetable salad dish by color and setting a plurality of filters in the HSV color space that can individually detect the colors; (3) taking a photograph of the vegetable salad dish, applying each of the filters to the acquired image, detecting color regions in the image corresponding to the filters, measuring the area of ​​each detected color region, and dividing the area of ​​each color region by the sum of the areas of the color regions to obtain an area ratio of each color region; (4) calculating the standard content of the phytochemical component to be measured in the food material corresponding to the color region by dividing the estimated content by the corresponding area ratio for each color region and for each type of phytochemical component to be measured; (5) sequentially performing steps (3) to (4) for each of the plurality of vegetable salad dishes to obtain an average value of the reference content; (6) A method characterized by including a step of correlating the type of food material and the color of the corresponding color area with the type name of the phytochemical component to be measured contained in the food material and the average value of the standard content, and registering them in the phytochemical database.

2. A system for measuring the phytochemical component content of a vegetable salad dish based on an image of the vegetable salad dish, comprising: A server equipped with the phytochemical database constructed by the method of claim 1; At least one user terminal connected to the server so as to be able to communicate data with the server and equipped with a camera; The server an image receiving unit that receives data of an image of the vegetable salad dish from the user terminal; a color region detection unit that applies a plurality of filters in an HSV color space to the image data, each of which can individually detect the colors of ingredients of the vegetable salad dish, and detects color regions in the image corresponding to the filters; an area ratio calculation unit that calculates an area ratio of each color region detected by the color region detection unit by measuring the area of ​​the color region and dividing the area of ​​the color region by the sum of the areas of all the color regions; a measurement unit that searches the phytochemical database based on the color of each color region detected by the color region detection unit, extracts the corresponding foodstuff type, the name of the type of phytochemical component to be measured contained in the foodstuff, and the average value of the standard content of the phytochemical component, and calculates the actual content of the phytochemical component by multiplying the average value of the standard content by the area ratio of the detected color region; A system characterized by comprising a measurement result transmission unit that transmits to the user terminal the item of food material extracted by the measurement unit, the name of the type of phytochemical component to be measured contained in the food material, and the calculated actual measured content of the phytochemical component.

Images