Area management device and program

Abstract

In the prior art, in the recommendation of foodstuffs with respect to a storage, the nutrients of the current inventory are not considered, there is a possibility of becoming a cause of inventory excess, or the past intake nutrients are not considered, and it is not clear how long a period of time the purchase deficiency can be prevented. In order to solve the above problem, in a refrigerated storage (1) for storing foodstuffs which is an example of a region management apparatus of the present application, there are provided: an identifying section (124) which identifies a usage state of a management region of the refrigerated storage (1); a health index calculating section (126) which, based on the usage state, determines nutrients which are recommended to be ingested for a prescribed user, and calculates a health index corresponding to the nutrients; and a display section (128) which displays the health index.

Description

Technical Field

[0001] This invention relates to an area management device for processing information regarding the storage and preservation of items, including food ingredients (hereinafter referred to as storage). In particular, it relates to a technology for providing information corresponding to the usage status of inventory within the storage area. Furthermore, the area management device includes, in addition to the storage room itself, a computer, a portable terminal, a network server, and an in-storage management system (area management system) comprising at least one of these components for its management. Moreover, the storage room can be anything capable of storing items, including cold storage, storage cabinets, floor storage, food cabinets, food storage rooms, shelves, etc. Background Technology

[0002] Currently, in line with increased health awareness, there are technologies that provide recommendations on insufficient nutritional value and ingredients. For example, there are technologies that calculate the nutritional intake based on a user's past diet during a certain period, and then calculate the deficient nutrients based on the difference between the nutritional intake and the recommended intake, thus calculating an appropriate diet (Patent Document 1). Patent Document 1 describes "calculating the total nutritional value of the diet used in any period and showing a diet that provides nutritional value that is insufficient from a nutritional balance perspective."

[0003] In addition, there is a technology that considers the current nutrient inventory and recommends prioritizing the purchase of low-cost ingredients to supplement deficient nutrients (Patent Document 2). Patent Document 2 describes a "cold storage facility that simultaneously displays priority consumption ingredients based on the overall cost-effectiveness evaluation of the inventory ingredients and recommends purchasing ingredients that achieve a balance between price and the consumer's supplementation of deficient nutrients, i.e., the best cost-effectiveness".

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2002-251518

[0007] Patent Document 2: Japanese Patent Application Publication No. 2006-90646 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] Here, suggestions regarding insufficient nutritional value and ingredients can be made more suitable for users by considering usage conditions such as the inventory in the storage facility (in-store inventory). However, Patent Document 1 does not consider the current inventory of nutrients, which could potentially lead to overstocking. Furthermore, Patent Document 2 does not consider past nutrient intake; while it may help curb overstocking, it is unclear how long-term it can prevent insufficient purchases. As mentioned above, Patent Documents 1 and 2 fail to provide suggestions that align with the user's intake status.

[0010] Methods for solving problems

[0011] To address the aforementioned issues, this invention utilizes a management marker indicating the user's nutritional status, taking into account recommended future nutrient intake. More preferably, based on recommended nutrient intake corresponding to the storage facility's usage, health indicators corresponding to the nutrients are calculated. Furthermore, recommended dietary information regarding nutrient intake, corresponding to the health management marker including health indicators or nutritional parameters, is generated.

[0012] More specifically, the present invention includes the following structures (1) and (2).

[0013] (1) An area management device for processing information about food storage, comprising: an identification unit for identifying the usage status of the management area for storage; a health index calculation unit for determining, based on the usage status, a nutrient recommended for a specified user and calculating a health index corresponding to that nutrient; and an output unit for outputting the health index.

[0014] (2) An area management device for processing information about food storage, comprising: an identification unit for identifying the usage status of the storage management area; a health management mark calculation unit and a health indicator calculation unit for determining, based on the usage status of the storage management area, a recommended nutrient intake for a specified user and calculating a health management mark representing the nutritional status of the user; a suggestion information generation unit for generating suggestion information about a recipe for consuming the determined nutrient in accordance with the health management mark; and an output unit for outputting the suggestion information.

[0015] Furthermore, this invention includes a health management method and computer program using the area management device described in (1) and (2) above. Additionally, the area management device in (1) and (2) includes a storage facility and a computer such as a portable terminal. Furthermore, the system and subsystems including the aforementioned storage facility and computer are also included in the area management device of this invention.

[0016] Invention Effects

[0017] According to the present invention, it is possible to make recommendations regarding nutrient intake that are appropriate for the user's dietary situation.

[0018] Other issues, structures, and effects not described above will be explained through the following description of the implementation methods. Attached Figure Description

[0019] Figure 1 This is an overall structural diagram of the warehouse management system in the embodiment.

[0020] Figure 2 This is a flowchart illustrating the library management process in Example 1.

[0021] Figure 3 This is an example of a CG image captured by a fisheye camera.

[0022] Figure 4 This is an example of an image obtained by converting a fisheye camera image (CG) into a planar image.

[0023] Figure 5 This is a diagram illustrating an example of a nutritional information table of the ingredients used in the embodiments.

[0024] Figure 6 This is a diagram showing an example of a suggested message.

[0025] Figure 7 This is an example of an order screen displayed on a portable terminal.

[0026] Figure 8 This is a flowchart illustrating the library management process in the embodiment.

[0027] Figure 9 This is a diagram illustrating an example of the healthy recipe shown in Example 2.

[0028] Figure 10 This diagram illustrates an example of a camera installed inside the main body of a cold storage warehouse.

[0029] Figure 11 This is a diagram showing the structure of the portable terminal 7 that performs library management processing. Detailed Implementation

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this embodiment, a storage facility is used as an area management device. In this embodiment, the usage status of the storage facility is identified, and nutritional parameters and health indices are calculated based on it as management indicators representing the user's nutritional status, corresponding to the nutrients the user should consume. Then, based on the calculated management indicators, information regarding a recipe for consuming the nutrients the user should consume is suggested. The information regarding the recipe includes the cooking contents (the recipe itself), recommended additional ingredients, and the cooking method of the recipe.

[0031] In the embodiments described later, a cold storage room is used as an example of a storage room. However, the storage room in this embodiment is not limited to a cold storage room, but also includes shelves, etc. In addition, it is not limited to specific storage rooms and shelves or other furniture or appliances, and can also be set as an area for storing food consumed by users (hereinafter sometimes referred to as a management area). For example, it can be set as the entire area of ​​a user's own home. In the embodiments, the interior of the storage room is described as an example of a management area. Sometimes, the food stored in the management area that is subject to management is referred to as managed food.

[0032] Furthermore, as an example of the imaging department, a camera will be used for explanation. However, this embodiment is not limited to cameras, and can also be widely applied to text recognition of sensor information such as weight sensors and mycotoxin detectors, IC tag information, and packaging information.

[0033] In this embodiment, the identification unit manages the management area, such as identifying the usage status of ingredients in the storage room and ingredients consumed by users. In the following embodiments, the identification unit uses the image captured by the camera unit for identification, but a structure that identifies input corresponding to the user's input to a portable terminal 7 such as a smartphone and the storage room itself can also be adopted. In this embodiment and the following embodiments, ingredients are used as an example of items for explanation. In addition, ingredients can be raw materials, cooked food, and seasonings. Furthermore, the storage management system (area management system) in the following embodiments and the various devices and combinations thereof described later are included in the area management device of the present invention.

[0034] (Example 1)

[0035] First, use Figures 1 to 7 Example 1 is described below. Figure 1 This is an overall structural diagram of the warehouse management system (area management system) in Embodiment 1 and Embodiment 2 described later. The warehouse management system is formed by interconnecting the cold storage 1 with a portable terminal 7, a network server 8, and a computer 9 via a network CN. The following describes each device.

[0036] First, the cold storage 1 includes a control unit 10 and a cold storage main body 20, which serves as the "storage main body".

[0037] Additionally, the portable terminal 7, as an external device, is a terminal used by the user of the cold storage 1. As described later, the portable terminal 7 displays a list of food orders sent from the cold storage 1. The portable terminal 7 can be implemented using an information processing device such as a tablet, smartphone, or PC. Furthermore, while the main processing in Embodiments 1 and 2 is performed by the cold storage 1 (control unit 10), it can also be performed by the portable terminal 7, or the cold storage 1 and the portable terminal 7 can share the workload. Regarding this, using... Figure 11 This will be described later.

[0038] Additionally, the network server 8 includes, for example, online supermarkets and recipe websites. The cold storage 1, through communication with it, can purchase ingredients and obtain cooking methods, recipes, etc. Furthermore, the computer 9 is a computer used to publish machine learning models and the like for performing information summaries as described in Examples 1 and 2.

[0039] Next, details of the cold storage 1 will be explained. A camera 50 for filming the interior of the cold storage is installed on the upper part of the main body 20. Furthermore, the camera's location is not limited to outside the storage; it can also be inside. Figure 10 This diagram illustrates an example of cameras 50a to 50e installed inside the main body 20 of the cold storage unit. Cameras 50a and 50b are examples of cameras installed above the door. Camera 50c is an example of a camera installed in the upper part of the main body. Generally, each camera can also be installed in a location other than the upper part of the interior of the main body 20. As examples, the positions of cameras 50d and 50e can be shown. Camera 50d is installed in the center of the door, and camera 50e is installed in the lower part of the interior. Furthermore, the installation position and number of cameras are not limited to [specific locations / specific locations]. Figure 10 For example, especially when setting up more than one unit.

[0040] Additionally, the control unit 10 for controlling the cold storage 1 includes, for example, a processor 11, a storage device 12, a communication unit (I / F) 14 connected to the network CN, and an I / O interface 13 (I / O) in the figure. Furthermore, the storage device 12 includes a main storage device composed of volatile or non-volatile memory, and an auxiliary storage device composed of flash memory or hard disk drive, etc.

[0041] It is also possible to send part or all of the computer programs and data stored in the storage device 12 to the outside via the communication network CN. Conversely, it is also possible to send and store computer programs and data from an external computing unit 9 or the like to the storage device 12 via the communication network CN.

[0042] In addition, it is also possible to connect a storage medium MM such as a flash memory or hard disk drive to the control unit 10, and transfer part or all of the computer program and data between the storage device 12 and the storage medium MM.

[0043] The storage device 12 stores a prescribed computer program for implementing the imaging unit 121, image conversion unit 123, recognition unit 124, table control unit 125, health index calculation unit 126, suggestion information generation unit 127, display unit 128, ordering unit 129, and inventory control unit 130. Additionally, the storage device 12 includes an image buffer 122. However, the image buffer 122 can also be a separate structure.

[0044] Then, the processor 11 executes the respective computer programs to implement the various functional units (except for the image buffer 122 in 121-130 described above). That is, Figure 1 Each functional unit of the storage device 12 is equivalent to a program. Therefore, each unit can be referred to as a computer program, and the processing and functions of each unit, described later, are implemented by the processor 11 using each computer program. However, each unit can also be implemented using dedicated hardware or an FPGA (Field Programmable Gate Array). Furthermore, these computer programs can be configured as one or fewer than the number shown in the figure. In this case, each functional unit can be configured as a computer module (also simply called a module).

[0045] As described above, processor 11 functions as a functional unit providing predetermined functions by executing processing according to a computer program. For example, processor 11 functions as an image conversion unit 123 by executing processing according to an image conversion program. The same applies to other computer programs. Furthermore, processor 11 also functions as a functional unit providing the functions of multiple processes executed by each computer program. In addition, in this embodiment, the computer program is executed by only processor 11, but it can also be executed by multiple processors.

[0046] The imaging unit 121 acquires camera images from the camera 50 via the I / O interface 13 and saves the acquired camera images to the image buffer 122. The camera 50, which is the "camera unit", is configured as a fisheye camera. The camera 50 has, for example, a fisheye or wide-angle lens.

[0047] The image conversion unit 123 converts camera images captured with a fisheye or wide-angle lens into planar images. Since known techniques can be used to unfold images captured with a fisheye or wide-angle lens into planar images, the description is omitted.

[0048] The recognition unit 124 comprises a learning-based image recognition unit and a rule-based image recognition unit. The learning-based image recognition unit and the rule-based image recognition unit each recognize ingredients from a library of converted planar images. The learning-based image recognition unit includes, for example, a machine learning model such as deep learning that has been pre-learned, and outputs the recognition results of the ingredients included in the planar image when the input planar image is received.

[0049] Furthermore, the rule-based image recognition unit identifies food items from a rule-based database when inputting a planar image. The rule-based image recognition unit performs region segmentation on the input planar image, labels items within each region, and outputs the recognized label content as text. Additionally, both the learning-based and rule-based image recognition units can utilize known techniques.

[0050] The recognition unit 124 in this embodiment uses image recognition to identify the usage status of the cold storage 1, but it is not limited to the above. In this case, the imaging unit 121, image buffer 122, and image conversion unit 123 can be omitted. Then, a structure for recognition in the recognition unit 124 is provided instead of them. For example, if a weight sensor is used, a functional unit is provided to perform processing to correspond the weight to the food ingredients.

[0051] In addition, the control section 125 controls the nutritional composition of each food ingredient as specified in the food nutrition composition table T1 (for reference). Figure 5 The table control unit 125 accesses the food nutrition facts table T1 and uses its results in other functional units. The food nutrition facts table T1 is located inside the table control unit 125. Furthermore, the nutritional components in the food nutrition facts table T1 are the nutritional benchmarks (benchmark values) used when calculating the health index using the health index calculation unit 126. For example, the values ​​from the Ministry of Education, Culture, Sports, Science and Technology's "Japanese Food Standard Composition Table 2020 Edition" can be cited as examples of these nutritional components.

[0052] However, the nutritional information table T1 for ingredients is not limited to this example; the nutritional information for each ingredient can also be created independently by the ingredient manufacturer or other entities.

[0053] In addition, the health index calculation unit 126 calculates the health index based on the in-store health index (managed area health index) and the consumer health index. This calculation includes preliminary preparation. This preliminary preparation consists of the following two stages. In the first stage, the health index calculation unit 126 refers to the values ​​of nutrients, i.e., nutritional components, of the food ingredients identified by the table control unit 125. Examples of such nutrients include protein, fat, and carbohydrates. However, Examples 1 and 2 can also be widely applied to nutrients such as vitamins and minerals.

[0054] In the second stage, the health index calculation unit 126 sums up the identified ingredients within the categories of protein, fat, and carbohydrates, and calculates the total proportion of protein, fat, and carbohydrates. This proportion can be, for example, the proportion of the protein, fat, and carbohydrates of the ingredients of interest as a percentage of the total energy, assuming each ingredient is converted into energy.

[0055] Furthermore, when calculating the health index within the database, the health index calculation unit 126 compares the total proportions of protein, fat, and carbohydrates with the baseline values ​​to calculate the nutritional parameters for each database. In this embodiment, the baseline value is (Equation 1-1) representing the value of the "Dietary Intake Standards for Japanese People" (2020 edition) issued by the Ministry of Health, Labour and Welfare, as an example.

[0056] p*:f*:c*=13~20:20~30:50~65……(Equation 1-1)

[0057] in:

[0058] p*+f*+c*=100……(Equation 1-2)

[0059] Here, p* represents the baseline value for protein intake, f* represents the baseline value for fat intake, and c* represents the baseline value for carbohydrate intake. However, the baseline values ​​are not limited to those set by the Ministry of Health, Labour and Welfare; they can also be set independently by food manufacturers, etc. Input from users, such as gender and age, can also be accepted and considered when setting the baseline values. Additionally, the baseline values ​​for each nutrient can be set by the user.

[0060] Here, nutritional parameters represent information about the nutrients in the managed ingredients, such as those in the warehouse. Therefore, the higher the value, the more likely the user is to be advised to purchase ingredients containing that nutrient. Furthermore, nutritional parameters consist of warehouse nutritional parameters and consumption nutritional parameters. The nutritional parameter for a particular nutrient represents the proportion of two or more nutrients relative to the total. In this embodiment, as described above, proteins, fats, and carbohydrates are used as the three nutrients in the description.

[0061] Furthermore, the in-keto health index is calculated by subtracting the absolute values ​​of each in-keto nutrient parameter (protein, fat, carbohydrates) from 100. This calculation uses equations (2-1) to (2-5). Additionally, the in-keto health index, as expressed in equation (2-6), is a value between 0 and 100. However, the mathematical formula is not limited to equations (2-1) to (2-6), and other mathematical formulas may be used.

[0062] Health index within the library = 100 - |np ip |-|np if |-|np ic |……(Equation 2-1)

[0063] np ip =x t2 -p*……(Equation 2-2)

[0064] np if =y t2 -f*……(Equation 2-3)

[0065] np ic =z t2 -c*……(Equation 2-4)

[0066] in:

[0067] x t2 +y t2 +z t2 =100……(Equation 2-5)

[0068] 0 ≤ Health index within the library ≤ 100…… (Equation 2-6)

[0069] Here, np ip It is the intra-pool nutritional parameter of protein, np if It is the intracellular nutrient parameter of fat, np ic This refers to the in vivo nutrient parameters of carbohydrates. Additionally, x... t2 It represents the protein content of the currently identified ingredients in the storage area, y t2 This refers to the percentage of fat in the currently identified ingredients in the warehouse, z t2 It represents the proportion of carbohydrates in the currently identified ingredients in the storage area.

[0070] In other words, the absolute value of the in-store nutrient parameter for a certain nutrient indicates the extent to which the proportion of nutrients in the food stored in the managed area deviates from the baseline proportion. Therefore, it can be understood that the absolute value of the in-store nutrient parameter for a certain nutrient indicates the extent to which the user's intake of that nutrient deviates from the baseline from now until the end of consuming all the food in the managed area, assuming the user consumes all the food in the managed area.

[0071] The in-store health index is obtained by subtracting the absolute values ​​of the in-store nutritional parameters of each nutrient from an arbitrarily determined maximum value (100 in this example). Therefore, the in-store health index represents the nutritional balance of the food stored in the managed area, that is, the degree of nutritional balance that users will consume if they live with the food as it is currently managed.

[0072] In addition, the Consumer Health Index is calculated by subtracting the absolute values ​​of each consumer nutrient parameter (protein, fat, carbohydrates) from 100. This calculation uses equations (3-1) to (3-5).

[0073] In addition, the consumer health index, as expressed in (Equation 3-6), is a value between 0 and 100. However, the mathematical formula is not limited to (Equation 3-1) to (Equation 3-6), and other mathematical formulas can also be used.

[0074] First, determine the managed food consumed from any past time to the present, and calculate the total energy of each nutrient (protein, fat, and carbohydrates in this example) contained in that consumed managed food. That is, calculate the total energy of each nutrient ingested by the user from any past time to the present. Next, divide each nutrient by the total energy. This allows calculation of the proportion of each nutrient ingested by the user from any past time to the present. These proportions are referred to as the consumption nutrient parameters for protein, fat, and carbohydrates, and are defined as np. cp ,np cf ,np cc Information regarding the management of these consumed ingredients can be identified using cameras installed in the storage facilities. Further details will be provided separately.

[0075] Right now:

[0076] Consumer Health Index 100-|np cp |-|np cf |-|np cc |……(Equation 3-1)

[0077] np cp = (The proportion of protein ingested by the user from any past time to the present) ... (Equation 3-2)

[0078] np cf = (The percentage of fat consumed by the user from any point in the past to the present) ... (Equation 3-3)

[0079] np cc = (The proportion of carbohydrates consumed by the user from any past time to the present) ... (Equation 3-4)

[0080] np cp +np cf +np cc =100……(Equation 3-5)

[0081] 0 ≤ Consumer Health Index ≤ 100…… (Equation 3-6)

[0082] That is, the absolute value of the consumption nutrient parameter of a certain nutrient represents the proportion of a certain nutrient consumed by the person (i.e., the user or the person living with the user) in the consumption management area from the past to the present.

[0083] The Consumption Health Index is obtained by subtracting the absolute values ​​of the consumption nutrient parameters of each nutrient from an arbitrarily determined maximum value (100 in this example). Therefore, the Consumption Health Index represents the degree to which the people in the consumption management area have maintained a balanced intake of nutrients from a specified past time to the present.

[0084] In addition, the health index is calculated by averaging the health index in the database and the consumer health index.

[0085] In this calculation, Equation 4 is used.

[0086] Health Index = (In-Stock Health Index + Consumer Health Index) / 2 ... (Equation 4)

[0087] The health index is evaluated using five levels: A (80-100): Excellent, B (60-79): Good, C (40-59): Average, D (20-39): Needs Improvement, and E (0-19): Very Needs Improvement. However, this is not the only evaluation method; manufacturers can also set their own levels. Therefore, the health index calculated using the in-stock health index and the consumer health index represents the nutritional balance of the user's intake from a past point in time until the current intake of managed food ingredients. This allows for the evaluation of nutritional balance by considering both the balance of nutrients ingested during the past specified period and the nutritional balance of the current managed food ingredients.

[0088] The health index calculated using the health index within the database and the health index of consumption is not limited to summation and averaging; it can be calculated using multiplication and averaging or other averaging methods, or any other method that does not impede the calculation.

[0089] In addition, the warehouse health index is a value calculated based on the nutrients of each ingredient identified in the warehouse. Based on the warehouse health index, for example, additional ingredients can be recommended to the user in a way that maintains a healthy nutritional balance within the warehouse.

[0090] As an example of calculating the health index within the food storage area, if the nutrient ratio of the food in the storage area identified at the current time (or other specified time) is protein:fat:carbohydrate = 20:35:45, when compared with the baseline ratios p*, f*, and c* for nutrient intake, protein is within the baseline range, fat is 5 more than the upper limit of the baseline, and carbohydrates are 5 less than the lower limit of the baseline. In this case, assuming the baseline value for fat is set as the upper limit and the baseline value for carbohydrates is set as the lower limit, in order to maintain a healthy balance within the storage area, a recommendation to purchase high-carbohydrate foods is generated and suggested. The nutrient parameter within the storage area in this example ratio is np. ip =0、np if =-5、np ic =5. Additionally, the health index within the library is 100-5-5=90.

[0091] In this way, to improve or maintain a high level of health within the warehouse, recommended ingredients should be obtained, thereby improving the nutritional balance of the ingredients. Furthermore, it becomes easier to cook nutritionally balanced menus using sufficient managed ingredients, reducing the frequency of necessary shopping trips.

[0092] The Consumer Health Index is a value calculated based on the nutritional content of various foods consumed by a user over a specific period from now until now. Based on the Consumer Health Index, it's possible to suggest suitable foods or preferred menus for users, for example, based on the nutritional balance of their consumption over time.

[0093] For time-series differences, users can freely select N days from options such as the previous day, 2 or 3 days ago, or 1 week ago. Therefore, it's possible to provide shopping suggestions that allow inventory to be held up to N days after the user's specified date.

[0094] As an example for calculating this consumer health index, consider a scenario where a user sets the time-series difference to the previous day, and the nutrient ratio of the food consumed the previous day relative to total energy intake is protein:fat:carbohydrate = 23:37:40. When compared to the baseline ratios p*, f*, and c* for nutrient intake, protein is 3 higher than the upper limit of the baseline, fat is 7 higher, and carbohydrates are 10 lower. This indicates a low proportion of carbohydrate intake. In this case, to adjust the nutritional balance of the time-series difference, it is possible to suggest high-carbohydrate foods or menus. The consumer nutrition parameter is np. cp =3、np cf =7、np cc =-10. Additionally, the Consumer Health Index is 100-3-7-10=80.

[0095] Furthermore, the health index is calculated based on inventory information of food products (in-stock health index) and the history of food consumption (consumer health index). Therefore, the health index is an index corresponding to nutrients, and it is preferable to use information that is easier to understand based on nutritional parameters.

[0096] Furthermore, based on health indices and nutritional parameters, i.e., health management indicators, the system recommends additional food purchases, taking into account existing inventory and required nutrient intake. The health indices aim to easily ensure sufficient intake of the necessary nutritional amounts of food needed in the future. Thus, health management indicators represent the user's nutritional status, and more preferably include both health indices and nutritional parameters.

[0097] When using the in-store health index and consumer health index values ​​from the examples above, the health index is calculated as: in-store health index (=90) + consumer health index (=80) / 2 = 85. Nutritional parameters are obtained by adding the in-store nutritional parameters to the consumer nutritional parameters, np. p =np ip +np cp =3, np f =np if +np cf =2, np c =np ic+np cc =-5.

[0098] In addition, the health index calculation unit 126 can calculate nutritional parameters that are the same health management indicator as the health management index, so it can be regarded as a health management indicator calculation unit.

[0099] In addition, the information generation unit 127 includes an additional ingredient decision unit 1271 and a healthy recipe information generation unit 1272. In the additional ingredient decision unit 1271, to improve the health index, based on nutritional parameters, and considering the inventory in the warehouse and the nutrients that should be consumed, it suggests the amount of additional ingredients that should be purchased. Therefore, it is easy to ensure, in terms of nutrition, the necessary and sufficient amount of ingredients that should be consumed in the future from the warehouse.

[0100] In the examples given above, the health index is 85 and the nutritional parameter is NP. p =3、np f =2、np c =-5, therefore it is recommended to buy foods with a high protein and fat content and a low carbohydrate content in order to improve the health index. Eggs are a good example of foods to consider as additional purchases.

[0101] Furthermore, the healthy recipe information generation unit 1272 displays recipes that can be made with ingredients from the database and suggests recipes with a higher health index. The indicators used for additional purchases are unrelated to the health index; one or a combination of the database's health index, the consumer health index, or multiple indicators can be used. This allows for the suggestion of rich recipes that effectively provide optimal nutrients, and makes it easy to create recipes that minimize food waste. Recipe information can be obtained from the internet or created independently by manufacturers, etc. Recipe information can be obtained using known technologies. Regarding the display of recipe information, it is envisioned that it be arranged in order of health index, but it is not limited to this; it can also be displayed in a single overview without prioritization.

[0102] Furthermore, the suggestion information generation unit 127 may also be equipped with a cooking method determination unit that determines the cooking method of the dish shown in the recipe. Furthermore, the suggestion information generation unit 127 may also adopt a structure that includes at least one of an additional ingredient determination unit 1271, a healthy recipe information generation unit 1272, and a cooking method determination unit. Additionally, the cooking method determination unit preferably obtains the cooking method from an external device, similar to the healthy recipe information generation unit 1272.

[0103] Furthermore, the display unit 128 displays health indices, in-store health indices, consumer health indices, current in-store nutritional balance, nutritional balance of previously consumed ingredients, in-store ingredients, additional purchased ingredients, and healthy recipes on a display device such as an LCD screen installed on the cold storage 1 (not shown). Alternatively, it may display only one item, not all of the above. Here, the display unit 128 may also be implemented as an output unit such as I / O interface 13 and I / F 14.

[0104] Additionally, in the ordering unit 129, recommended additional purchases of ingredients to improve the health index are displayed on a screen such as the portable terminal 7 or a cold storage LCD screen, and the displayed ingredients are ordered automatically. Alternatively, ingredients can be ordered semi-automatically from an online supermarket by the user pressing an order button. The indicators used for ordering are not limited to the health index; they can also be the in-stock health index or the consumer health index. Furthermore, the ordering unit 129 can also be omitted from the processing.

[0105] In addition, the internal control unit 130 controls an electric motor and compressor (not shown) to control the temperature and humidity inside the cold storage 1.

[0106] Next, use Figure 3 and Figure 4 The images captured by camera 50 will be explained. Figure 3 This diagram illustrates an example of a fisheye camera image GC captured by camera 50. In this embodiment, camera 50 is a fisheye lens. Figure 4 The image shown is a fisheye camera image GC captured by it. Here, it is difficult to directly use the fisheye camera image GC to enable the recognition unit 124 to perform food identification. That is, because the machine learning model learns using images without distortion during the learning phase, it is difficult to directly identify the fisheye camera image GC distorted by the fisheye or wide-angle lens 52.

[0107] Therefore, the image conversion unit 123 converts the distorted image captured by the fisheye lens into a planar image. For this purpose, learning is performed in cooperation with the aforementioned computer 9. Here, Figure 4 This diagram illustrates an example of an image obtained by converting a fisheye camera image GC into a planar image. For this purpose, the image conversion unit 123 generates a recognition image G30 by combining the distortion-removed right door unfolded image G32, the front unfolded image G33, the left door unfolded image G34, and the upper unfolded image G31.

[0108] As a method to remove distortion, well-known camera calibration techniques can be used. For example, feature points before and after distortion correction can be extracted from an image with a known distortion-corrected pattern, such as a target plate, and the parameters of the fisheye camera can be estimated based on the position coordinates of these feature points.

[0109] That concludes the description of the images captured by camera 50. Next, the processing described in this embodiment will be explained.

[0110] Figure 2 This is a flowchart illustrating an example of warehouse management processing performed by the control unit 10. First, in step S16, the imaging unit 121 acquires a visible light image captured by the camera 50.

[0111] In addition, the imaging unit 121 preferably stores the acquired visible light image as a camera image GC in the image buffer 122.

[0112] Next, in step S17, the image conversion unit 123 reads the camera image GC captured in step S16 from the image buffer 122 and unfolds the fisheye image into a planar image.

[0113] Next, in step S18, the learning-based image recognition unit of the recognition unit 124 receives the recognition image G30 generated by the image conversion unit 123 as input, and enables the machine learning model obtained from the computer 9 to perform image recognition of the food ingredient. Additionally, the rule-based image recognition unit of the recognition unit 124 receives the recognition image G30 generated by the image conversion unit 123 as input, performs image recognition of the food ingredient based on rules, and outputs the content of the recognized label. Furthermore, the processing order of the learning-based image recognition and the rule-based image recognition is not limited to the above, and they can be performed in parallel.

[0114] Next, in step S19, the health index is calculated in the health index calculation unit 126 using the food nutrition facts table T1. Furthermore, this health index preferably includes both the stock health index and the consumer health index. Figure 5 This is an example of a diagram representing the nutritional information of a food ingredient, Table T1.

[0115] The proportion of nutrients in the stored ingredients identified by the identification unit 124 is calculated based on the nutritional composition table T1. Therefore, in this step, the health index calculation unit 126 calculates the health index of the stored ingredients based on the nutrients identified from the stored ingredients. The health index of the stored ingredients is calculated based on the user's current inventory information. The purpose of the health index of the stored ingredients is to maintain nutritional balance in the stored ingredients in order to recommend additional ingredients to the user based on the consumer health index.

[0116] Next, in step S20, the health index calculation unit 126 calculates a consumption health index based on the user's food consumption history over a certain period from now on. To this end, the health index calculation unit 126 can calculate the consumption health index based on the temporal differences (changes) of the food in the inventory and the food consumed by the user. For this calculation, the identification unit 124 preferably determines the food consumed by the user, i.e., the consumed food, based on the temporal changes of the food in the inventory.

[0117] Here, the purpose of the Consumer Health Index is to provide users with suitable food recommendations based on the consumption patterns over time, in conjunction with the Inventory Health Index. For the time-series difference, users can freely select N days from options such as the previous day, 2 or 3 days ago, or 1 week ago. Therefore, it allows for shopping recommendations based on whether the inventory can sustain the user's purchases up to N days into the future.

[0118] Next, in step S21, the health index calculation unit 126 calculates a health index based on the user's current inventory of food ingredients (in-stock health index) and the food ingredient consumption history (consumption health index). The health index is preferably an index corresponding to the nutrients recommended for the user's intake, and more preferably information that facilitates ensuring the necessary and sufficient amount of food ingredients to be consumed in the future in terms of nutrition.

[0119] Next, in step S22, the additional food ingredient determination unit 1271 of the recommendation information generation unit 127 determines the additional food ingredients. As an example, the additional food ingredient determination unit 1271, in order to improve the health index, determines recommended food ingredients and their quantities based on nutritional parameters, considering the inventory in the warehouse and the nutrients that should be consumed, and outputs the recommended ingredients. Thus, it is easy to ensure, in terms of nutrition, that the necessary and sufficient amount of food ingredients that should be consumed in the future are available in the warehouse.

[0120] Here, a specific example of step S22 is shown below. The additional ingredient determination unit 1271 learns of the deficiencies in various nutrients. For example, protein: less than 30, fat: less than 70, etc. Next, the additional ingredient determination unit 1271 searches for matching ingredients from the ingredient nutrition facts table T1 for the nutrient with the largest deficiency (fat in the above example). For example, nuts (fat: 47.6) are found.

[0121] Then, the ingredient determination unit 1271 adds the retrieved nutrients from the ingredients and calculates a new deficiency value. In the example above, the deficiency becomes: protein: less than 10.2, and fat: less than 17.6.

[0122] Next, the supplementary ingredient decision unit 1271 searches for ingredients with the most deficient nutrient. In this case, the supplementary ingredient decision unit 1271 searches for ingredients from the ingredient nutrition facts table T1 to meet the requirement of fat: 17.6. Furthermore, in the search for ingredients with the most deficient nutrient, if any nutrient is in excess, the supplementary ingredient decision unit 1271 can either cancel the searched ingredients and search for the next most deficient nutrient, or directly use the search results. Alternatively, it can use a structure that determines whether the user accepts the decision to cancel.

[0123] Alternatively, the additional ingredient determination unit 1271 can also determine the ingredients based on the proportion of deficiencies in each nutrient using a combination optimization algorithm. In this case, it is preferable to perform the calculation in a way that minimizes the amount (number, weight) of the ingredients.

[0124] Next, in step S23, the display unit 128 displays the health index, the in-stock health index, the consumption health index, the current in-stock nutritional balance, the nutritional balance of previously consumed ingredients, the in-stock stored ingredients, additional ingredients, healthy recipes, cooking methods, etc., on the display device or portable terminal 7. Alternatively, it is not necessary to display all of the above; only one item may be displayed.

[0125] Here, we will explain this display example. First, Figure 6 This is a diagram illustrating an example of displaying suggested information. This display can be executed using either a display device or a portable terminal 7; the case of displaying using the portable terminal 7 will be described. As described above, the portable terminal 7 can be implemented using a smartphone. The portable terminal 7 is not limited to smartphones; it can also be a tablet, wearable device, laptop, etc. Furthermore, the warehouse management application running on the portable terminal 7 communicates with the cold storage 1 periodically or irregularly via the network CN. Additionally, for this application (warehouse management application 720), [the following is used...] Figure 11 This will be described later.

[0126] Next, regarding Figure 6 Please provide an explanation. Figure 6 The display example shown is divided into an upper left section 61, a lower left left section 62, a lower right left section 63, and a right section 64. The upper left section 61 displays the health index and overall evaluation. The lower left left section 62 displays information on available ingredients (health index) and nutritional balance. The lower right left section 63 displays information on consumed ingredients (health index) and nutritional balance. The right section 64 shows recommendations for additional ingredients to be purchased to improve the health index, considering the available inventory and the necessary nutrients. The recommended information displayed here is only information about dietary menus for nutrient intake and is not limited to specific dietary recommendations. Figure 6 .

[0127] Additionally, among the recommended additional purchase items containing high levels of nutrients, priority can be given to displaying or only items that the user has historically consumed frequently. As an example, for recommendations and... Figure 6The following explains different scenarios regarding additional food items. For example, consider a scenario where the user frequently consumes natto and eggs, which are high in protein. In this case, the list of high-protein foods would be displayed in the order of sausage, pork, ham, natto, and eggs. However, if the user's food consumption history indicates a particular tendency to consume natto and eggs, then the next display could show natto, eggs, ham, pork, sausage, or only natto and eggs. This concludes the explanation of the suggested information display examples. Return to... Figure 2 Continuing with the explanation of warehouse management processes.

[0128] Next, in step S24, the ordering unit 129 displays the suggested additional ingredients on the portable terminal 7 or display device and automatically orders the displayed ingredients. Alternatively, the user can semi-automatically order ingredients from the online supermarket by pressing the order button. Alternatively, the ordering unit 129 may be omitted from the process.

[0129] Here, we will explain the display example in this step. Figure 7 This diagram illustrates an example of an ordering screen displayed on the portable terminal 7. The ordering screen is displayed on the touch panel 703 as a food order list 73. This food order list 73 includes suggested additional food items received from the cold storage 1.

[0130] Below the food order list 73, the order target 74 and the order button 75 are displayed. When the user of cold storage 1 is detected to have interacted with the order button 75, the content of the food order list 73 is sent to the order target 74. Additionally, Figure 7 The example shown is of a single order target 74, but multiple order targets 74 can also be displayed, which can be selected by the user when placing an order. Alternatively, an order target can be selected from a pre-registered pool of order targets, corresponding to the type of ingredients.

[0131] In addition, the timing of generating the food order list 73 in the additional food decision-making unit 1271 is not limited to taking pictures of the warehouse with camera 50, and can be carried out regularly or irregularly.

[0132] Furthermore, an example is shown where the camera unit 121 takes a picture of the interior of the cold storage 1, performs image recognition of the ingredients, and generates and sends an ingredient order list 73, but this is not a limitation. For example, the control unit 10 may also store the ingredient order list 73 calculated based on the recognition results of the camera image GC in the storage device 12, and send the latest ingredient order list 73 to the portable terminal 7 upon receiving a request for the ingredient order list 73. Thus, the user of the cold storage 1 can quickly know which ingredients need to be purchased even when out and about by referring to the portable terminal 7.

[0133] Furthermore, the machine learning model of the recognition unit 124 can be updated to the latest machine learning model in response to changes or additions to the packaging of the ingredients. For example, a machine learning model received from a server not shown can be updated to the machine learning model of the recognition unit 124.

[0134] As described above, in this embodiment, a health index can be calculated using the user's item consumption history (consumption health index) and the current information on items in the user's storage (storage health index). Taking into account the inventory level in the storage and the necessary nutrients, the system can recommend additional food purchases. Regarding the item consumption history (consumption health index), by allowing the user to freely select the number of days in the past time series, shopping recommendations can be made to ensure that the inventory in the storage can last until the selected number of days later.

[0135] (Example 2)

[0136] use Figure 8 and Figure 9 Example 2, which recommends healthy recipes to improve health index, will be described. This example focuses on the differences from Example 1. In this example, recipes with a higher health index are recommended to the user from among recipes that can be made using ingredients available in the database.

[0137] Figure 8 This is a flowchart illustrating an example of the library management process performed by the control unit 10 in this embodiment. Steps S16-S18 and S24 of this process are... Figure 2 Steps S16-S18 and S24 are the same, so their description is omitted. In this process, steps S25 and S26 are newly added. Furthermore, the structure of this embodiment can be the same as that of Embodiment 1.

[0138] First, in step S25, the healthy recipe information generation unit 1272 displays recipes that can be made with the identified ingredients in the database, arranged in order of health index, and suggests recipes with higher health indices. This suggests rich, healthy recipes that effectively provide preferred nutrients and are easy to prepare while minimizing food waste. Alternatively, the recipes can be arranged not by health index, but by the health index in the database or the health index of the consumer.

[0139] Next, in step S26, the display unit 128 displays the stored ingredients and healthy recipes on the portable terminal 7 or display device, in addition to the health index and additional ingredients. Figure 9 This is an example of a healthy recipe overview 65 shown in this step. Figure 9The display unit 128 is divided into a left side section 66 and a right side section 67. The left side section 66 displays an overview of the ingredients in the storage area. The right side section 67 displays recipes that can be made using the ingredients in the storage area, ordered from highest to lowest health index.

[0140] In addition, users can freely specify the types of healthy recipes corresponding to their priorities such as health, endurance, and activity. For example, there are three categories: Category 1, Category 2, and Category 3. Category 1 emphasizes balance and includes staple food, main dish, side dish, soup, milk / dairy products, and fruit. Category 2 allows users to choose a focus on fish, meat, or vegetables. Category 3 allows users to choose from various options such as: High Health (for couples / families wanting to eat healthily every day), Medium Health (for those wanting to eat less about health indicators), Endurance (for children wanting to eat more after participating in activities), and Celebrations (for birthdays, passing exams, etc.).

[0141] This embodiment, thus configured, achieves the same effect as the first embodiment. Furthermore, in this embodiment, by suggesting recipes that improve the health index based on the ingredients in the database, it is possible to suggest rich recipes that effectively provide preferred nutrients, and to prepare recipes in a way that easily reduces food waste.

[0142] The warehouse management processes in the above embodiments are executed by the control unit 10 of the cold storage 1, but they can also be executed by other devices. Hereinafter, a variation of warehouse management processing using a portable terminal 7 will be described. Figure 11 The diagram shows the structure of a portable terminal 7 that performs library management processing in this modified example. The portable terminal 7 in this example includes a processor 701, a storage device 702, a touch panel 703, and a communication unit 704. As described above, the portable terminal 7 can be implemented using an information processing device (computer) such as a smartphone.

[0143] Then, the processor 701 and the storage device 702 have the same Figure 1 The control unit 10 shown has the same functions. Additionally, the touch panel 703 functions as an input / output unit. Furthermore, the communication unit 704 connects to the network CN. This connection can be wireless or wired.

[0144] Here, the warehouse management application 720 (warehouse management program), which stores in storage device 702 and performs the processing of this modified example, will be described. The warehouse management application 720 consists of an identification module 721, a table control module 722, a health index calculation module 723, a suggestion information generation module 724, an ordering module 725, and a warehouse control instruction module 726. Furthermore, the suggestion information generation module 724 consists of an additional ingredient decision module 7241 and a health recipe information generation module 7242. These constitute a single computer program (application), but each module can also be composed of independent computer programs, or some modules can be combined into a single computer program.

[0145] In addition, each module executes and Figure 1 The functional units shown have the same function. That is, they have the following correspondence.

[0146] Recognition Module 721: Recognition Unit 124

[0147] Table control module 722: Table control unit 125

[0148] Health Index Calculation Module 723: Health Index Calculation Section 126

[0149] Suggestion Information Generation Module 724: Suggestion Information Generation Unit 127

[0150] Additional ingredient decision module 7241: Additional ingredient decision department 1271

[0151] Healthy Recipe Information Generation Module 7242: Healthy Recipe Information Generation Department 1272

[0152] Ordering Module 725: Ordering Department 129

[0153] Warehouse control indication module 726: Warehouse control unit 130

[0154] In addition, each module performs the same processing as the corresponding functional unit as described above, but the warehouse control instruction module 726 preferably manages the usage status of ingredients in the warehouse.

[0155] For example, the warehouse control instruction module 726 obtains user input and information read from the food ingredient code, and the identification unit 124 identifies the corresponding food ingredient.

[0156] Furthermore, the library management application 720 is preferably distributed to the portable terminal 7 via the network CN. Therefore, the network CN is implemented using the Internet.

[0157] Based on the above embodiments and variations, a health index can be calculated using the user's item consumption history (consumption health index) and the current information on items in the user's storage (storage health index), and suggestions can be made regarding the amount of food to be purchased, taking into account the inventory in the storage and the nutrients that should be consumed. Regarding the item consumption history (consumption health index), by allowing the user to freely select the number of days of past time differences, shopping suggestions can be made to ensure that the inventory in the storage can last for the selected number of days.

[0158] This concludes the description of the present invention. However, the present invention is not limited to the embodiments described above, but includes various modifications. For example, the embodiments described above are provided for a better understanding of the present invention and are not limited to having all the structures described above.

[0159] It is possible to replace a portion of the structure of one embodiment with the structure of another embodiment. It is also possible to add the structure of another embodiment to the structure of one embodiment. For a portion of the structure of each embodiment, it is possible to delete, add other structures, or replace it with other structures.

[0160] The aforementioned structures, functions, processing units, and processing modules can be partially or entirely implemented in hardware, for example, through design in integrated circuits. Alternatively, the aforementioned structures and functions can be implemented in software by a processor interpreting and executing programs that implement each function. The programs, tables, files, and other information implementing each function can be stored in a storage device. This storage device includes non-volatile semiconductor memory, hard disk drives, SSDs (Solid State Drives), and other computer-readable non-transitory data storage media such as IC cards, SD cards, and DVDs.

[0161] In addition, the control lines and information lines shown are those deemed necessary for the description, but do not necessarily represent all control lines and information lines on the product. In fact, it can be assumed that almost all structures are interconnected.

[0162] Furthermore, the above embodiments can be appropriately combined, and such combinations are also included within the scope of the present invention.

[0163] Explanation of reference numerals in the attached figures

[0164] 1: Cold storage

[0165] 7: Portable terminal

[0166] 8: Network Server

[0167] 9: Computer

[0168] 10: Control Department

[0169] 12: Storage device

[0170] 121: Filming Department

[0171] 20: Main body of the cold storage

[0172] 50: Camera

[0173] 123: Image Conversion Department

[0174] 124: Identification Department

[0175] 125: Table Control Department

[0176] 126: Health Index Calculation Department

[0177] 127: Suggestion Information Generation Department

[0178] 128: Display Section

[0179] 129: Ordering Department.

Claims

1. A regional management device for processing information regarding the storage of food ingredients, characterized in that, have: The identification unit identifies the managed ingredients stored in the management area and the consumed ingredients stored in the management area, as a measure of the usage status of the management area. The health index calculation unit determines the recommended nutrients for a specified user based on the usage status, and calculates the health index corresponding to the nutrient using the health index of the managed area corresponding to the nutrient of the managed food and the health index of the consumer corresponding to the nutrient of the consumed food. The suggestion information generation department determines that it includes recommended ingredients for improving the health index and additional ingredients in appropriate quantities; and The output section contains the aforementioned health index and suggested additional ingredients. The health index calculation unit Calculate the consumption nutrient parameters representing the nutrients in the consumed food and the management area nutrient parameters representing the nutrients in the managed food. The consumer health index is calculated using the aforementioned consumer nutrition parameters. The health index of the management area is calculated using the nutritional parameters of the management area. The health index is calculated by averaging the health index of the managed area and the health index of the consumer. This health index represents the nutritional balance that the user is expected to consume from a specified time until the end of the intake of the managed ingredients.

2. The area management device as described in claim 1, characterized in that: The health index calculation unit also refers to a storage unit that stores a food nutrient composition table representing the nutritional components of each food ingredient, and uses the food nutrient composition table to determine the nutrients of the managed food ingredient and the consumed food ingredient.

3. The area management device as described in claim 1, characterized in that: The health index calculation unit calculates the nutritional content of the food ingredients consumed in the past period, as the consumption health index.

4. A regional management device for processing information regarding the storage of food ingredients, characterized in that, have: The identification unit identifies the managed ingredients stored in the management area and the consumed ingredients stored in the management area, as a measure of the usage status of the management area. The health management mark calculation unit determines the recommended intake of nutrients for the specified user based on the usage status of the storage management area, and calculates a health management mark representing the nutritional status of the user. The health index calculation unit uses the health index of the managed area corresponding to the nutrients of the managed food and the health index of the consumer food corresponding to the nutrients of the consumer food to calculate the health index corresponding to the nutrient. The suggestion information generation unit generates suggestion information about recipes for consuming the identified nutrients, in accordance with the health management mark. and The output unit outputs the suggested information. The health index calculation unit Calculate the consumption nutrient parameters representing the nutrients in the consumed food and the management area nutrient parameters representing the nutrients in the managed food. The consumer health index is calculated using the aforementioned consumer nutrition parameters. The health index of the management area is calculated using the nutritional parameters of the management area. The health index is calculated by averaging the health index of the managed area and the health index of the consumer. This health index represents the nutritional balance that the user is expected to consume from a specified time until the end of the intake of the managed ingredients.

5. The area management device as described in claim 4, characterized in that: The suggestion information generation unit has at least one of the following: an additional ingredient decision unit for deciding on additional ingredients, a healthy recipe information generation unit for generating the recipe, and a cooking method determination unit for determining the cooking method of the recipe.

6. The area management device as described in claim 5, characterized in that: The health management indicator calculation unit, as the health management indicator, calculates at least one of the nutritional parameters representing the recommended intake of nutrients and the health indicators corresponding to the recommended intake of nutrients.

7. The area management device as described in any one of claims 4-6, characterized in that: The output unit outputs the suggested information and the health management symbol.

8. A program product, characterized in that: The program product includes a program that enables one or more processors to perform the processing of the area management device as described in any one of claims 1 to 7.

Images