Method for managing food ingredients based on artificial intelligence

Abstract

An artificial intelligence-based food ingredient management method comprises the steps of: acquiring one or more images of a food ingredient storage area of ​​a cafeteria; generating inventory information including identification information and estimated inventory quantities for each of one or more food ingredients within the food ingredient storage area based on one or more images using a vision AI model; acquiring facility profile data and user data associated with the cafeteria; generating menu data including the required quantity of each of one or more food ingredients based on the inventory information, facility profile data, and user data using a menu generation model; calculating an additional order quantity by subtracting the estimated inventory quantity included in the inventory information from the required quantity for each of one or more food ingredients; and transmitting order data including the additional order quantity to a supplier system.

Description

Technology Field

[0001] The present disclosure relates to an artificial intelligence-based food ingredient management method.

[0002] The present disclosure relates to the field of artificial intelligence (AI) technology, and more specifically, to image-based object recognition technology using a vision AI model, machine learning-based time series forecasting technology, and decision support technology using an optimization algorithm.

[0003] The present disclosure relates to a computer-implemented method for automating the inventory recognition, menu composition, and ordering processes of food ingredients in the field of institutional food service, and in particular to computer vision-based inventory identification, the generation of customized menus reflecting user health information, and data-based optimization of food ingredient ordering. Background Technology

[0005] In order to provide nutritionally balanced meals to hundreds of users every day, cafeterias must repeatedly perform numerous tasks, such as checking food inventory, planning menus, and ordering ingredients.

[0006] However, conventional cafeteria operations rely on a method in which nutritionists or cooks manually check the food storage area to determine inventory, compose menus based on empirical judgment, and manually calculate the required amount of ingredients to place orders with suppliers. In this manual-based process, human error may occur during the inventory assessment process, it is difficult to systematically reflect users' health status or allergy information in the menu composition, and there is a risk of repeated waste losses due to over-ordering of ingredients or a failure to meet the required number of meals due to under-ordering.

[0007] In particular, conventional inventory tracking methods mostly rely on images taken at a single point in time or visual inspection, making it difficult to accurately determine the quantity of food ingredients stored in overlapping or irregular forms (e.g., nets, sacks, etc.). Furthermore, if meal count forecasting relies on the nutritionist's empirical estimation, fluctuations in meal count caused by seasonal variations, daily deviations, or internal events are not systematically reflected, which may lead to repeated over-ordering or under-ordering of food ingredients.

[0008] Furthermore, as users' allergy and health status information is reflected unsystematically in meal planning, problems may arise where allergy-causing ingredients are included or nutritional compositions unsuitable for the user's health condition are provided. A technical approach is required to unify inventory recognition, meal planning, and ordering processes by integrally applying vision AI technology and time-series forecasting models to food ingredient management in institutional catering environments.

[0009] The aforementioned technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and does not necessarily refer to known technology disclosed to the general public prior to the filing of the present invention.

[0010] The information described above is intended to enhance understanding of the background of the present invention and may include information that does not constitute prior art. The problem to be solved

[0012] The present disclosure provides an artificial intelligence-based food ingredient management method, apparatus, system, and computer program for solving the aforementioned problems. means of solving the problem

[0014] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.

[0015] An artificial intelligence-based food ingredient management method performed by one or more processors according to one embodiment of the present disclosure comprises: acquiring one or more images of a food ingredient storage area of ​​a cafeteria; generating inventory information including identification information and an estimated inventory quantity for each of one or more food ingredients within the food ingredient storage area based on one or more images using a vision artificial intelligence model; acquiring facility profile data and user data associated with the cafeteria; generating menu data including the required quantity of each of one or more food ingredients based on the inventory information, facility profile data, and user data using a menu generation model; calculating an additional order quantity by subtracting the estimated inventory quantity included in the inventory information from the required quantity for each of one or more food ingredients; and transmitting order data including the additional order quantity to a supplier system.

[0016] According to one embodiment of the present disclosure, the step of generating inventory information includes: generating identification information including the location and type of each of one or more food ingredients by performing object detection on one or more images; calculating an estimated inventory amount based on the size of the area occupied by each of one or more food ingredients within one or more images for each of one or more food ingredients; excluding from the inventory information food ingredients for which the identification confidence score output by the vision artificial intelligence model for each of one or more food ingredients is less than a preset threshold; and summing the estimated inventory amounts of food ingredients identified as the same type at multiple locations within one or more images.

[0017] According to one embodiment of the present disclosure, user data includes at least one of allergy information or health status information for each of a plurality of users of a cafeteria, and the method further includes the step of excluding food ingredients corresponding to the constraint from the diet data by reflecting the allergy information or health status information included in the user data as a constraint.

[0018] According to one embodiment of the present disclosure, facility profile data includes industry group information of a workplace to which a cafeteria belongs and work intensity information of the workplace, and the step of generating diet data includes generating diet data to satisfy standard nutritional requirements corresponding to the industry group information and work intensity information, and the standard nutritional requirements include target ranges for calories, carbohydrates, protein, and fat, respectively.

[0019] According to one embodiment of the present disclosure, the step of generating meal data includes the step of calculating a predicted number of meals based on past meal history data of a cafeteria and work schedule data of a workplace to which the cafeteria belongs, and the step of calculating the required amount of each of one or more food ingredients based on the unit requirement amount of each of one or more food ingredients and the predicted number of meals. The predicted number of meals may be calculated by multiplying a weighted moving average, which applies weights based on temporal proximity to each actual number of meals in each of a plurality of past periods included in the past meal history data, by the ratio of the number of people scheduled to report to work included in the work schedule data to the average number of people reporting to work in a plurality of past periods.

[0020] Various aspects and features of the invention are defined in the appended claims. Combinations of features of the dependent claims may be appropriately combined with features of the independent claims, not only as explicitly presented in the claims.

[0021] Additionally, one or more selected features of any one embodiment described in this disclosure may be combined with one or more selected features of any other embodiment described in this disclosure, and such alternative combination of features is possible if it at least partially alleviates one or more technical problems discussed in this disclosure or at least partially alleviates technical problems discernible from this disclosure by a person skilled in the art, and furthermore, such combination is possible if the specific combination or permutation of the embodiment features thus formed is not understood by a person skilled in the art to be incompatible.

[0022] In any described example implementation, two or more physically distinct components may alternatively be integrated into a single component if such integration is possible, provided that the same function is performed by the single component thus formed. Conversely, a single component of any embodiment described in this disclosure may alternatively be implemented by two or more distinct components that achieve the same function, where appropriate.

[0023] The purpose of certain embodiments of the present invention is to solve, mitigate, or eliminate at least one of the problems and / or disadvantages associated with the prior art, at least partially. Certain embodiments are intended to provide at least one of the advantages described below. Effects of the invention

[0025] According to various embodiments of the present disclosure, by using a vision artificial intelligence model to automatically generate inventory information from images of food storage areas, generating optimal meal data that reflects facility profile data and user data, and accurately ordering only the necessary food ingredients by subtracting existing inventory, cost losses due to excess food ingredient orders can be reduced and the manual workload of dietitians can be reduced.

[0026] According to various embodiments of the present disclosure, the location, type, and quantity of food ingredients within an image are automatically detected using a vision artificial intelligence model, and the detection results are visually provided to the user in the form of bounding boxes and identification labels. This automates the task of checking food ingredient inventory, which was previously performed manually by the user, and reduces the time required for inventory checking and human error.

[0027] According to various embodiments of the present disclosure, by reflecting allergy information or health status information included in user data as constraints and automatically excluding unsuitable food ingredients from the diet, safety accidents caused by the provision of allergy-causing food ingredients can be prevented, and it can be made possible to provide a customized diet suitable for the user's health condition.

[0028] According to various embodiments of the present disclosure, by automatically calculating standard nutritional requirements corresponding to the industry group and work intensity of the workplace to which the cafeteria belongs and generating meal data that satisfies the standard nutritional requirements, it becomes possible to provide customized nutrition suitable for the user's physical activity level, and the nutrition design process, which previously relied on the empirical judgment of a nutritionist, can be standardized.

[0029] According to various embodiments of the present disclosure, by combining past meal history data and work schedule data to calculate the predicted number of meals and precisely calculating the required amount of each food ingredient based on the predicted number of meals, the process of determining the order quantity, which previously relied on the empirical estimation of a nutritionist, can be standardized based on data, and the failure to meet the meal requirement due to under-ordering of food ingredients or the disposal of leftover food ingredients due to over-ordering can be minimized.

[0030] According to various embodiments of the present disclosure, a specific user interface is provided that integrally displays an image of a food material storage area and an additional order form on the screen of a user terminal and transmits order data to a supplier system in response to the user's confirmation input, thereby enabling a series of processes including verification of food material inventory recognition results, review of order details, and order transmission to be performed on a single screen.

[0031] The present disclosure is not a simple mathematical operation, but a technical data processing process performed by a computing device, which achieves the technical effect of precisely controlling the physical inventory status of food ingredients within a cafeteria and reducing waste. Furthermore, it includes the effect of resource optimization through the efficiency of the food ingredient supply chain and waste reduction.

[0032] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims. Brief explanation of the drawing

[0034] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. FIG. 1 is a diagram showing an overview of an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 2 is a block diagram showing the internal configuration of a user terminal and a server in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 3 is a flowchart relating to an artificial intelligence-based food ingredient management method performed by a server in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 4 is a diagram showing an example of an inventory recognition result by a vision artificial intelligence model in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 5 is a diagram illustrating an example of the relationship between user data and meal data in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 6 is a diagram showing an example in which industry group information and work intensity information included in facility profile data are reflected in diet data in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 7 is a diagram illustrating an example of a flow for calculating the required amount of food ingredients in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. FIG. 8 is a drawing showing an example of a user interface for an artificial intelligence-based food ingredient management service in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure. Specific details for implementing the invention

[0035] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. A singular expression may include a plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this disclosure. Terms used in this disclosure that are defined in a general dictionary may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure. In some cases, even terms defined in this disclosure are not to be interpreted to exclude the embodiments of this disclosure.

[0036] In the various embodiments of the present disclosure described below, a hardware-based approach is described as an example. However, since the various embodiments of the present disclosure include techniques using both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.

[0037] Hereinafter, various embodiments are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, since the technical concept of the present disclosure can be modified and implemented in various forms, it is not limited to the embodiments described in this specification. In describing the embodiments disclosed in this specification, if it is determined that specifically describing related prior art could obscure the essence of the technical concept of the present disclosure, such specific description of prior art is omitted. Identical or similar components are assigned the same reference numerals, and redundant descriptions thereof are omitted.

[0038] When an element is described in this specification as being "connected" to another element, this includes not only cases where they are "directly connected" but also cases where they are "indirectly connected" with another element in between. When an element is described as "comprising" another element, this means that, unless specifically stated otherwise, it does not exclude other elements in addition to the other elements but may include additional elements.

[0039] Some embodiments may be described by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. The functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks of the present disclosure may be implemented as algorithms executed on one or more processors. The functions performed by the functional blocks of the present disclosure may be performed by a plurality of functional blocks, or the functions performed by a plurality of functional blocks in the present disclosure may be performed by a single functional block. Additionally, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc.

[0040] Additionally, in this disclosure, expressions such as "greater than" or "less than" have been used to determine whether specific conditions are satisfied or fulfilled; however, this is merely for illustrative purposes and does not exclude descriptions of "greater than" or "less than." Conditions described as "greater than" may be replaced with "greater than," conditions described as "less than" with "less than," and conditions described as "greater than and less than" with "greater than and less than."

[0041] FIG. 1 is a diagram showing an overview of an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0042] Referring to FIG. 1, the user (110) may be a person in charge of food ingredient management, such as a nutritionist or a cook at a cafeteria. In addition to a nutritionist or a cook, the user (110) may be any person in charge of managing the inflow and outflow and inventory status of food ingredients, such as an operations manager or an inventory manager at a cafeteria.

[0043] The user terminal (120) may include a smartphone, tablet PC, etc. equipped with a camera function. The user terminal (120) may refer to any computing device capable of wired and / or wireless communication and capable of running applications. For example, the user terminal (120) may include a smartphone, tablet PC, PDA (personal digital assistants), or industrial rugged terminal, etc. The internal configuration of the user terminal (120) may be described in detail in FIG. 2.

[0044] The food storage area (130) may refer to any space within the cafeteria where food ingredients are stored, such as a refrigerator, freezer, drying warehouse, or shelf. The food storage area (130) is not limited to a single location, and if there are multiple food storage areas within the cafeteria, images can be acquired individually for each food storage area. For example, if there is a refrigerator for food ingredients that require refrigeration and a shelf for dried food that can be stored at room temperature, the user (110) can acquire multiple images by sequentially photographing each storage space.

[0045] The food ingredients (132) refer to individual food ingredient items stored in the food ingredient storage area (130) and may include vegetables, fruits, meat, dairy products, grains, processed foods, etc. The food ingredients (132) may be stored in packaging units (e.g., boxes, bottles, bags, cans) or in an unpackaged state (e.g., individual vegetables, fruits). The types and quantities of the food ingredients (132) may vary depending on the size of the cafeteria, the type of menu provided, and the number of users.

[0046] A user (110) can use a user terminal (120) to photograph food ingredients (132) within a food ingredient storage area (130). One or more images captured by the user terminal (120) can be transmitted to a server (described later in FIG. 2) and analyzed by a vision artificial intelligence (AI) model, and based on the analysis results, a series of processes including generating inventory information, generating meal data, and transmitting order data can be performed. The specific internal configuration of the server is described in FIG. 2, and each step of the AI-based food ingredient management method performed by the server can be described in detail in FIG. 3.

[0047] One or more images may include still images or frames extracted from a video. When a user (110) films a food storage area (130) as a video using the camera of a user terminal (120), the user terminal (120) or the server may extract frames from the video at a predetermined interval (e.g., 1 frame per second) and use them as one or more images.

[0048] According to various embodiments of the present disclosure, a vision artificial intelligence model may automatically recognize the type and quantity of food ingredients (132) based on an image captured by a user terminal (120), generate meal data based on the recognition results, calculate an additional order quantity considering the existing inventory, and transmit the order data to a supplier system. Since each step of food ingredient recognition, meal creation, and order quantity calculation is automated by a computer, human error and workload in the inventory counting, meal composition, and ordering processes, which previously relied on the empirical judgment and manual work of a nutritionist, can be reduced.

[0049] According to one embodiment of the present disclosure, a dedicated application installed on a user terminal (120) may provide a shooting guide. The dedicated application may display an overlay on the display that guides the entire food storage area (130) to be included in the shooting frame while the camera of the user terminal (120) is activated. For example, the overlay may include guide lines that visually indicate whether the top, bottom, left, and right boundaries of the food storage area (130) are included within the frame. If the dedicated application determines that the food storage area (130) is not sufficiently included within the frame, it may display a message to the user (110) guiding adjustment of the shooting position or angle. The dedicated application may analyze the sharpness of the captured image and provide a notification recommending re-shooting if shaking or out-of-focus is detected.

[0050] According to another embodiment of the present disclosure, a user (110) can increase recognition accuracy by taking multiple images from different angles. For example, the user (110) can take one or more images from the front, top, and side of the food storage area (130), respectively. When multiple images taken from different angles are transmitted to a server, the vision AI model can combine the multiple images to additionally detect food ingredients (132) that are difficult to identify because they are obscured or overlapped by only frontal shooting, and the accuracy of the estimated inventory amount of food ingredients (132) can be improved. A specific procedure for generating inventory information using a vision AI model can be described in detail in FIG. 4.

[0051] According to another embodiment of the present disclosure, instead of a user terminal (120), a camera fixedly installed in the food storage area (130) may be able to periodically take pictures. The fixedly installed camera may be mounted inside a refrigerator or shelf and may automatically take pictures of the food storage area (130) at preset time intervals (e.g., every 1 hour, every 4 hours, every 12 hours) and transmit the images to a server. When using a fixedly installed camera, the inventory status can be periodically updated without manual shooting by the user (110). An embodiment in which shooting by the fixedly installed camera and the user terminal (120) is used in combination may also be possible, and images periodically taken by the fixed camera and images manually taken by the user (110) as needed can be integratedly analyzed on the server.

[0052] FIG. 2 is a block diagram showing the internal configuration of a user terminal (210) and a server (230) in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0053] Referring to FIGS. 1 and FIGS. 2, the user terminal (210) may correspond to the user terminal (120) of FIGS. 1. The user terminal (210) may refer to any computing device capable of wired and / or wireless communication and capable of running applications, etc. For example, the user terminal (210) may include a smartphone, a mobile phone, a tablet PC, a wearable device, a PDA (personal digital assistant), etc. According to one embodiment of the present disclosure, the user terminal (210) may be a general-purpose smartphone installed with an application developed exclusively for food ingredient management.

[0054] As illustrated, the user terminal (210) may include memory (211), a processor (212), a communication module (213), and an input / output interface (214). The server (230) may include memory (231), a processor (232), a communication module (233), and an input / output interface (234). The processor (212, 232) may include an AI (artificial intelligence) dedicated accelerator such as a CPU (central processing unit), GPU (graphics processing unit), NPU (neural processing unit), or TPU (tensor processing unit).

[0055] As illustrated in FIG. 2, the user terminal (210) and the server (230) can communicate information and / or data through the network (220) using their respective communication modules (213, 233).

[0056] The network (220) may include any communication network that enables communication between a user terminal (210) and a server (230). Depending on the installation environment, the network (220) may be composed of, for example, a wired network such as Ethernet or a wired home network (power line communication), a mobile communication network, a wireless local area network (WLAN), a wireless network such as Wi-Fi, Bluetooth, or ZigBee, or a combination of a wired network and a wireless network. The network (220) may include an LTE (long term evolution) network, a 5G (5th generation) network, or a private wireless network within the cafeteria (e.g., Wi-Fi dedicated to the cafeteria), and multiple types of networks may be used in combination depending on the size and environment of the cafeteria. The communication method is not limited and may include not only a communication method utilizing the communication network that the network (220) may include, but also short-range wireless communication between the user terminal (210) and the server (230).

[0057] The memory (211, 231) may include any non-transitory computer-readable recording medium. According to one embodiment of the present disclosure, the memory (211, 231) may include a permanent mass storage device such as a read-only memory (ROM), a disk drive, a solid-state drive (SSD), or a flash memory. As another example, a permanent mass storage device such as a ROM, SSD, a flash memory, or a disk drive may be included in a user terminal (210) or a server (230) as a separate permanent storage device distinct from the memory (211, 231).

[0058] The memory (211, 231) may store an operating system and at least one program code. The memory (231) of the server (230) may store parameters of a vision artificial intelligence model, parameters of a meal creation model, facility profile data, user data, food ingredient master data (names, categories, standard unit requirements, etc. for each food ingredient), supplier information, etc. The food ingredient master data may include a unique identification code, name, category (e.g., vegetables, meat, dairy products, grains, etc.), standard unit requirements (e.g., required weight per serving), standard unit price information, etc. for each food ingredient available for use in a cafeteria. Software components may be loaded from a computer-readable recording medium separate from the memory (211, 231). The separate computer-readable recording medium may include computer-readable recording media such as a floppy drive, disk, tape, DVD / CD-ROM drive, or memory card. As another example, software components may be loaded into memory (211, 231) via a communication module (213, 233) rather than a computer-readable recording medium.

[0059] The processor (212, 232) can process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (212, 232) by memory (211, 231) or a communication module (213, 233). For example, the processor (212, 232) can execute instructions received according to program code stored in a recording device such as memory (211, 231).

[0060] The processor (232) of the server (230) can generate inventory information by applying a vision artificial intelligence model to an image received from a user terminal (210), generate meal data using a meal generation model based on facility profile data and user data, calculate additional order quantities, and transmit order data to a supplier system. The specific details of each step performed by the processor (232) may be described in detail below in FIG. 3.

[0061] According to one embodiment of the present disclosure, the processor (232) may include a GPU or an NPU for performing inference operations of a vision artificial intelligence model and may support parallel processing of a plurality of images received from a user terminal (210). The processor (232) may perform image preprocessing (e.g., resolution normalization, color correction, etc.), detection of food ingredients and quantity estimation by a vision artificial intelligence model, calculation of an optimal meal plan by a meal plan generation model, calculation of additional order quantities, etc. in a pipeline manner.

[0062] The communication module (213, 233) may provide a configuration or function for the user terminal (210) and the server (230) to communicate with each other via the network (220). The user terminal (210) and the server (230) may be provided with a function to communicate with other systems (e.g., a vendor system, a separate cloud system) using their respective communication modules (213, 233). For example, a request or data generated by the processor (212) of the user terminal (210) according to program code stored in a recording device such as memory (211) may be transmitted to the server (230) via the network (220) under the control of the communication module (213). Conversely, meal data, order data, inventory information, etc. provided under the control of the processor (232) of the server (230) may be received by the user terminal (210) through the communication module (213) of the user terminal (210) via the communication module (233) and the network (220).

[0063] The input / output interface (214, 234) may be a means for interfacing with an input / output device (not shown). The input device may include, for example, a keyboard, a mouse, a microphone, a touchscreen, etc., and the output device may include a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (214, 234) may be a means for interfacing with a device in which the functions for input and output are integrated into one, such as a touchscreen. The input / output interface (214) of the user terminal (210) may be utilized to receive input for the user to view captured images and to review diet data or order data.

[0064] In FIG. 2, the input / output interface (214, 234) is shown as an element configured separately from the processor (212, 232), but an embodiment in which the input / output interface (214, 234) is included in the processor (212, 232) is also possible. The user terminal (210) and the server (230) may include more components than those shown in FIG. 2. For example, the user terminal (210) may further include components such as a camera module, a GPS module, an ambient light sensor, and an accelerometer. The server (230) may further include a dedicated communication interface with a vendor system (e.g., an EDI (electronic data interchange) linkage module, an API (application programming interface) gateway, etc.).

[0065] According to one embodiment of the present disclosure, the processor (212) of the user terminal (210) receives user input (e.g., touch of a shooting button, touch of a menu approval button, etc.) through the input / output interface (214) while a food ingredient management application is in operation, and can transmit data corresponding to the received user input to the server (230) through the communication module (213) and network (220). The processor (212) can display inventory information, menu data, or order data received from the server (230) through an output device such as a display. Specific examples of user interfaces (UI) displayed on the user terminal (210) can be described in FIG. 8.

[0066] An embodiment in which the server (230) is configured as a cloud-based distributed computing system may be possible. In a cloud-based distributed computing environment, the memory (231) and processor (232) of the server (230) may be distributed across a plurality of physically separated server devices and operated as an integrated virtual instance. For example, the inference computation of a vision artificial intelligence model may be performed on a GPU cluster, the optimization computation of a diet generation model may be performed on a separate computation node, and communication with the user terminal (210) may be managed through an API gateway.

[0067] An embodiment may be possible in which the user terminal (210) itself is equipped with on-device AI functions to perform some analysis locally. For example, if the processor (212) of the user terminal (210) includes an NPU, the user terminal (210) can analyze images captured on its own without the support of the server (230) to pre-classify the types of food ingredients (132) or determine the quality of the image (clarity, brightness, etc.). The results of the analysis performed locally on the user terminal (210) are transmitted to the server (230) and can be used as an auxiliary tool for precise analysis by the vision AI model of the server (230).

[0068] FIG. 3 is a flowchart relating to an artificial intelligence-based food ingredient management method performed by a server (230) in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0069] FIG. 3 is described together with FIG. 2. The flowchart (300) may include a plurality of steps (S310 to S360) performed by the processor (232) of the server. Specific data processing methods at each step may be described in detail in FIG. 4 to 7.

[0070] The processor (232) can acquire one or more images of the food storage area (130) of the cafeteria (S310). One or more images may be received via a network (220) from a user terminal (210) or directly from a camera fixedly installed in the food storage area. The images may be still image formats such as JPEG, PNG, or BMP, or frames extracted from a video. When multiple images are received, the processor may manage timestamp and shooting location information for each image as metadata.

[0071] The processor (232) can generate inventory information by applying a vision artificial intelligence model to one or more images obtained in step S310 (S320). The vision artificial intelligence model may include an object detection model, an image classification model, an instance segmentation model, etc., and architectures such as YOLO (you only look once), Faster R-CNN (faster region-based convolutional neural network), Mask R-CNN, and DETR (detection transformer) may be used, but the architecture of the vision artificial intelligence model may not be limited to the examples described above. The parameters of the vision artificial intelligence model may be stored in the memory (231) of the server, and a model that has been pre-trained or fine-tuned based on training data regarding the types of food ingredients and packaging forms handled at the cafeteria may be used. The inventory information may include identification information and / or an estimated inventory quantity. The identification information may include the type, name, category, etc. of the food ingredients. The estimated inventory quantity may be expressed in units such as quantity, weight, or volume. The specific procedure for generating inventory information can be explained in detail in Fig. 4.

[0072] The processor (232) can obtain facility profile data and user data associated with the cafeteria (S330). The facility profile data may include information on the industry group of the workplace to which the cafeteria belongs, work intensity information, work hours information, size of the cafeteria, number of meals per day, etc. The user data may include allergy information, health status information, preference information, age, gender, etc. of each cafeteria user. The facility profile data and user data may be stored in advance in the memory (231) of the server, or may be obtained in real time through an API (application programming interface) from an external system (e.g., a workplace personnel management system, a cafeteria management system, etc.). The specific method of reflecting the facility profile data and the specific method of reflecting the user data may be described in detail in FIG. 6 and FIG. 5, respectively.

[0073] The processor (232) can generate meal data including the required amount of each food ingredient for the cafeteria based on the inventory information generated in step S320, the facility profile data obtained in step S330, and user data using a meal generation model (S340). The meal generation model may include a machine learning model, a rule-based engine, or an optimization algorithm (e.g., linear programming, genetic algorithm, reinforcement learning, etc.). The meal data may include a list of food ingredients included in each of one or more meal candidates and the required amount of each food ingredient. The meal data may include meal compositions for one day or multiple days. The meal generation model may receive the estimated inventory amount included in the inventory information as input and configure the meal in a way that prioritizes the use of existing inventory. The parameters of the meal generation model may be stored in the memory (231) of the server and may be periodically updated based on the operational history data of the cafeteria. The predicted number of meals may be used to calculate the required amount of each food ingredient, and the method for calculating the predicted number of meals is described in detail in FIG. 7.

[0074] The processor (232) can calculate the additional order quantity by subtracting the estimated inventory quantity included in the inventory information from the required quantity for each food ingredient (S350). According to one embodiment of the present disclosure, the additional order quantity of the i-th food ingredient can be calculated based on Equation 1.

[0075]

[0076] Referring to mathematical formula 1, Q_order_i can indicate the additional order quantity for the i-th ingredient, R_i the required quantity of the i-th ingredient included in the meal data, and I_i the estimated inventory quantity of the i-th ingredient included in the inventory information. The max function can reflect that additional orders are unnecessary for ingredients whose estimated inventory quantity is greater than or equal to the required quantity. For example, if the required quantity (R) of onions is 15 kg and the estimated inventory quantity (I) is 7 kg, the additional order quantity can be calculated as 8 kg. On the other hand, if the required quantity (R) of potatoes is 10 kg and the estimated inventory quantity (I) is 12 kg, the additional order quantity is calculated as 0 kg, so no additional order may occur.

[0077] According to another embodiment of the present disclosure, it may be possible to calculate an additional order quantity that reflects safety stock by taking into account the uncertainty of the estimated inventory quantity. The additional order quantity that reflects safety stock may be calculated based on Equation 2.

[0078]

[0079] Referring to Equation 2, S_i can indicate the safety stock quantity for the i-th food ingredient. The safety stock quantity (S_i) can be determined based on the error range of the estimated inventory quantity by the vision AI model, the variability in consumption of the food ingredient, or the supplier's delivery lead time. For example, if the average error rate of the estimated inventory quantity is 10%, the safety stock quantity (S_i) can be set to 10% of the required quantity (R_i). By reflecting the safety stock, shortages of food ingredients caused by overestimation of the estimated inventory quantity can be prevented.

[0080] The processor (232) can transmit order data including the additional order quantity calculated in step S350 to the supplier system (S360). The order data may include the additional order quantity per food ingredient, food ingredient identification information, delivery request date, etc. The supplier system may include a server, an enterprise resource planning (ERP) system, or an electronic ordering platform operated by the food ingredient supplier. The method of transmitting the order data may include API integration, electronic data interchange (EDI), email, etc.

[0081] According to one embodiment of the present disclosure, it may be possible to transmit an approval request to a dietitian terminal (e.g., user terminal (210 in FIG. 2)) before transmitting order data, and to transmit order data to a supplier system in response to the dietitian's approval input. The dietitian may review the meal data and order data displayed on the user terminal's display, and if modifications are necessary, change some items of the meal data or adjust the order quantity, and then perform an approval input. When the approval input is received, the processor (232) may transmit order data with the changes reflected to the supplier system.

[0082] According to another embodiment of the present disclosure, it may be possible to receive unit price information from a plurality of suppliers and automatically select the supplier that minimizes the total cost. The processor (232) may refer to supplier information stored in memory (231), query the unit price for each food ingredient for each of the plurality of suppliers, calculate the total cost for each supplier by multiplying the additional order quantity by the unit price, and automatically select the supplier with the lowest total cost to transmit order data. In cases where a plurality of suppliers supply the same food ingredient at different unit prices, it may be possible to individually select the supplier with the lowest unit price for each food ingredient and place split orders with different suppliers for each food ingredient.

[0083] According to another embodiment of the present disclosure, steps S310 through S360 may be repeated at a preset interval (e.g., daily or weekly). Through repeated execution, meal data and order data may be periodically updated according to the consumption status of food ingredients and changes in inventory. The frequency of repeated execution may be determined differently depending on the operating type of the cafeteria (e.g., 5-day operation, 7-day operation), the distribution characteristics of food ingredients (e.g., fresh food ordered daily, dried food ordered weekly), or the settings of the manager.

[0084] According to one embodiment of the present disclosure, a feedback loop may be implemented to collect leftover data, update user preferences, and reflect them in the generation of subsequent meal data. When data regarding the amount or type of leftovers generated after the provision of meals is collected by the server (230), the processor (232) may analyze the leftover data to update the user's preferences for each meal. For ingredients included in meals with a large amount of leftovers, the preference score may be lowered, and for ingredients included in meals with a small amount of leftovers, the preference score may be raised. The updated preference information is utilized as input to a meal generation model in the subsequent step S340, thereby generating meal data that better matches the user's preferences.

[0085] Alternatively, steps S310 through S360 do not necessarily have to be performed in the order shown in FIG. 3. For example, step S330 (acquisition of facility profile data and user data) can be performed independently of step S310 (acquisition of images) and step S320 (generation of inventory information), so step S330 can be performed in parallel with step S310. The processor (232) can parallel process the image reception and the generation of inventory information by the vision artificial intelligence model, and the acquisition of facility profile data and user data as separate threads or processes.

[0086] According to various embodiments of the present disclosure, by using a vision artificial intelligence model to automatically generate inventory information from images of food storage areas, generating optimal meal data that reflects facility profile data and user data, and accurately ordering only the necessary food ingredients by subtracting existing inventory, cost losses due to excess food ingredient orders can be reduced and the manual workload of dietitians can be reduced.

[0087] FIG. 4 is a diagram showing an example of an inventory recognition result by a vision artificial intelligence model in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0088] Referring to FIGS. 1, 3, and 4, a bounding box, an identification label, and an estimated quantity (Est. Qty) for each food ingredient detected by a vision AI model may be superimposed on an image of a food ingredient storage area (130) captured on the screen (400). The bounding box may indicate the area occupied by the food ingredient within the image. The identification label may correspond to identification information (type and name). The estimated quantity may correspond to an estimated inventory quantity. The bounding box may be displayed as a rectangular dotted or solid line overlaid on the image, and the coordinates of the bounding box may be derived from the object detection results of the vision AI model.

[0089] For example, "Label: Potato, Est. Qty: ~12" may mean that the vision AI model identified the food ingredient in the area as a potato and calculated the estimated inventory quantity as about 12. In the screen (400) shown in FIG. 4, recognition results for multiple food ingredients such as "Onion (net), Est. Qty: 6-7", "Apple Juice, Est. Qty: 1", "Canned Tuna, Est. Qty: 2", "Milk, Est. Qty: 2", "Canned Corn, Est. Qty: 5", "Potato Bag, Est. Qty: ~15" may be displayed along with individual bounding boxes. When the estimated quantity is expressed as a range (e.g., 6-7) or an approximation (e.g., ~12, ~15), it may be because the food is stored in an irregular form (e.g., onions in a net, potatoes in a sack) and it is difficult to determine the exact number.

[0090] The inventory recognition result illustrated in FIG. 4 can visually represent the specific procedure for generating inventory information performed in step S320 of FIG. 3. The processor (232) can generate identification information including the location and type of each food ingredient by performing object detection on one or more images. The generation of identification information can be performed by a process in which a vision artificial intelligence model outputs the bounding box coordinates of each food ingredient within the image and a class label corresponding to the bounding box. The class label can be mapped to the food ingredient name in the food ingredient master data stored in the server's memory (231).

[0091] The processor (232) can calculate an estimated inventory amount for each food ingredient based on the size of the area occupied by the food ingredient within the image. According to one embodiment of the present disclosure, the estimated inventory amount can be calculated based on Equation 3.

[0092]

[0093] Referring to Equation 3, Q_est_i represents the estimated inventory quantity of the i-th food ingredient, A_bbox_i represents the bounding box area occupied by the i-th food ingredient within the image (in pixels or normalized area), A_ref_i represents the reference area occupied by one unit of the i-th food ingredient (a value pre-stored in the food ingredient master data), and γ_i represents a correction factor based on the loading method of the i-th food ingredient. The correction factor γ_i is intended to reflect the difference in the area-quantity relationship between when the food ingredient is loaded in a bag form and when it is arranged individually, and can be set differently for each category of the food ingredient (e.g., bags, boxes, individual items). For example, the correction factor γ_i for canned food arranged individually may have a value close to 1.0, while the correction factor γ_i for irregularly loaded food ingredients, such as onions in a net, may have a value greater than 1.0 (e.g., 1.3 to 1.5). If the correction factor is greater than 1.0, it can be estimated that a larger quantity is loaded for the same bounding box area. A_ref_i is a value pre-registered for each food ingredient in the food ingredient master data and can be calculated based on the standard area occupied when one unit of food ingredient (e.g., one potato, one pack of milk) is photographed from the front.

[0094] For example, in the screen (400) of FIG. 4, if the bounding box area (A_bbox) occupied by the potato is 36,000 pixels, the reference area (A_ref) of one potato is 3,000 pixels, and the correction factor (γ) of the potato is 1.0, the estimated inventory quantity (Q_est) can be calculated as 1.0 × (36,000 / 3,000) = 12. If the bounding box area of ​​the onion (net) is 15,000 pixels, the reference area of ​​one onion is 3,000 pixels, and the correction factor of the onion in the net is 1.4, the estimated inventory quantity can be calculated as 1.4 × (15,000 / 3,000) = 7.

[0095] Equation 3 is exemplary, and for the calculation of the estimated inventory quantity, a deep learning-based regression model may also be used to directly output the quantity from the feature vector of the bounding box. The regression model can output the estimated quantity of the corresponding food ingredient as a real value by inputting the image patch inside the bounding box into a convolutional neural network (CNN) and passing the extracted feature vector through a fully connected layer. When using a regression model, the area-quantity relationship can be implicitly learned from the training data without the need to separately manage correction coefficients based on the loading method of the food ingredient.

[0096] According to one embodiment of the present disclosure, it may be possible to calculate the area more precisely by extracting the accurate contours of food ingredients through instance segmentation in addition to bounding boxes. An instance segmentation model (e.g., Mask R-CNN) can output a pixel-level mask for each food ingredient and can calculate an occupied area more precise than the bounding box area based on the total number of pixels in the mask. When utilizing instance segmentation, the contours of individual food ingredients can be recognized separately even when the food ingredients partially overlap, thereby improving the accuracy of the estimated inventory amount.

[0097] According to another embodiment of the present disclosure, when the same type of food ingredient is detected at multiple locations within an image, it may be possible to sum the estimated inventory quantities. For example, in the screen (400) of FIG. 4, if "potato bag" is detected at the bottom left and bottom right of the screen respectively and the estimated quantity of each is about 15, the processor (232) can sum the estimated quantities of bounding boxes having the same identification label ("potato bag") to calculate the total estimated inventory quantity of potato bags as about 30.

[0098] According to another embodiment of the present disclosure, it may be possible to increase the estimation accuracy by correcting perspective distortion according to the shooting angle. When a user (110) photographs a food storage area at an oblique angle, the bounding box area of ​​a food located far away in the image may appear smaller than the bounding box area of ​​a food of the same size located close by. A processor (232) can correct the area distortion caused by perspective distortion by estimating the vanishing point of the image and applying a scaling factor proportional to the distance between the vanishing point and the bounding box to the bounding box area.

[0099] According to one embodiment of the present disclosure, a vision artificial intelligence model may output an identification confidence score for each bounding box. Bounding boxes with an identification confidence score below a preset threshold (e.g., 0.5) may be ignored and not included in the inventory information, and only bounding boxes with an identification confidence score above the threshold may be adopted as valid detection results. When the identification confidence score has a value near the threshold (e.g., 0.5 to 0.7), it may be possible to visually distinguish the bounding box (e.g., dotted line processing or color change) on the screen (400) of the user terminal (210) and request confirmation input from the user (110).

[0100] According to various embodiments of the present disclosure, the location, type, and quantity of food ingredients within an image are automatically detected using a vision artificial intelligence model, and the detection results are visually provided to the user in the form of bounding boxes and identification labels. This automates the task of checking food ingredient inventory, which was previously performed manually by the user, and reduces the time required for inventory checking and human error.

[0101] FIG. 5 is a diagram showing an example of the relationship between user data (510) and meal data (520) in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0102] Referring to FIGS. 3 and FIGS. 5, user data (510) may correspond to user data obtained in step S330 of FIG. 3 and may include allergy information and health status information of each user. Allergy information may indicate a history of allergic reactions to specific food ingredients or food ingredient components, and may include, for example, peanuts, shellfish, milk, gluten, etc. Health status information may indicate the user's chronic diseases, dietary restrictions, etc., and may include, for example, diabetes (low-sugar diet), high blood pressure (low-salt diet), kidney disease (low-potassium diet), etc. In addition to allergy information and health status information, user data (510) may further include the user's age, gender, preference information (e.g., preferred menu, non-preferred food ingredients), etc.

[0103] Diet data (520) may correspond to diet data generated in step S340 of FIG. 3, and for each of a plurality of diet candidates (diet X, diet Y, diet Z, etc.), may include ingredients and required amounts for each ingredient. In the table of diet data (520) shown in FIG. 5, diet X may include ingredients a (required amount: Na1), ingredients b (required amount: Nb1), ingredients c (required amount: Nc1), etc., diet Y may include ingredients a (required amount: Na2), ingredients b (required amount: Nb2), ingredients d (required amount: Nd), etc., and diet Z may include ingredients e (required amount: Ne), ingredients f (required amount: Nf), ingredients g (required amount: Ng), etc. The required amount may be the required amount based on one serving, and may be converted into a total required amount through calculation with the predicted number of meals.

[0104] The processor (232) can reflect allergy information or health status information included in user data (510) as constraints, and exclude food ingredients corresponding to the constraints from diet data (520). For example, since user C has an allergy to ingredient g, the diet generation model can exclude diet Z containing ingredient g for user C or replace ingredient g with a substitute ingredient. The selection of substitute ingredients can be performed by referring to a list of replaceable ingredients pre-registered in the food ingredient master data (e.g., stored in memory (231) of FIG. 2). For example, if ingredient g is milk, soy milk or oat beverage may be selected as a substitute ingredient.

[0105] Since User B's health status is recorded as 'poor', dietary restrictions corresponding to the health status information (e.g., exclusion of high-calorie diets, assignment of low-salt diets, etc.) may be reflected as constraints. For a user whose health status information is 'poor', the processor (232) may exclude diets among diet candidates that exceed a preset upper limit for calorie content, or prioritize the assignment of diets with low sodium content. Specific criteria for dietary restrictions corresponding to the health status information (e.g., calorie upper limit, sodium upper limit, sugar upper limit, etc.) may be stored in advance in the server's memory (231).

[0106] According to one embodiment of the present disclosure, in a group meal environment where the same meal plan is provided to all users of a cafeteria, a processor (232) may set constraints by combining all allergy information of multiple users. For example, if even one of the multiple users has an allergy to a specific food ingredient, the processor may exclude the food ingredient from the entire meal plan data (520) or separately generate an alternative meal plan that does not include the food ingredient.

[0107] According to another embodiment of the present disclosure, it may be possible to limit the frequency of use of a specific allergen (e.g., peanuts) at the level of the entire cafeteria by aggregating allergy information of all users of the cafeteria. If the proportion of users among a plurality of users who have an allergy to a specific allergen exceeds a preset threshold (e.g., 5%), the processor (232) may limit the frequency of meal allocation of the said ingredient to a preset number of times per week (e.g., 1 time) or less.

[0108] According to another embodiment of the present disclosure, it may be possible to calculate a diet preference score based on user preference information (e.g., preferred menu, non-preferred ingredients) and use it as input to a diet generation model. According to one embodiment of the present disclosure, the diet preference score may be calculated based on Equation 4.

[0109]

[0110] Referring to mathematical formula 4, Pref_m is the diet preference score of the m-th diet candidate, N is the total number of users of the cafeteria, P_ji is the preference value (range 0 to 1) for the i-th food ingredient of the j-th user, D_ji is the non-preference value (range 0 to 1) for the i-th food ingredient of the j-th user, δ(i, m) is an indicator function that is 1 if the i-th food ingredient is included in the m-th diet candidate and 0 if it is not included, and w_pos and w_neg can indicate weights for preference and non-preference. The higher the value of Pref_m, the more likely it is to be a diet candidate that meets the preferences of multiple users. The diet generation model can prioritize reflecting diets with high diet preference scores among diet candidates that satisfy constraints (allergy information, health status information) in the diet data (520).

[0111] According to one embodiment of the present disclosure, user data (510) may be obtained by the user directly inputting it through an application on a user terminal (210) or by a food service manager registering it in bulk. When the user directly inputs it, the application may provide a list of allergy-causing food ingredients in the form of a checklist to guide the user to select the items that apply to them. When a food service manager registers it in bulk, the food service manager may register the user data (510) based on the user's health examination results or allergy test results. The user data (510) may be stored in the memory (231) of a server and may be periodically updated according to changes in the user's health status or allergy history.

[0112] According to various embodiments of the present disclosure, by reflecting allergy information or health status information included in user data as constraints and automatically excluding unsuitable food ingredients from the diet, safety accidents caused by the provision of allergy-causing food ingredients can be prevented, and it can be made possible to provide a customized diet suitable for the user's health condition.

[0113] FIG. 6 is a diagram showing an example in which industry group information (610) and work intensity information (620) included in facility profile data are reflected in diet data (630) in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0114] Referring to FIGS. 3 and FIGS. 6, the industry group information (610) may correspond to a portion of the facility profile data obtained in step S330 of FIG. 3. The industry group information (610) may indicate the industrial classification of the workplace to which the cafeteria belongs, and may include, for example, manufacturing, construction, office work, medical services, education, etc. The industry group information (610) may be coded based on an accredited industrial classification system such as the Korean Standard Industrial Classification (KSIC) and may be stored in the server's memory (231) when the cafeteria is registered.

[0115] Work intensity information (620) may indicate the level of physical activity of the workers at the workplace and may be classified into grades such as light work, heavy work, and heavy work. Light work may correspond to office work performed mainly while sitting, heavy work may correspond to work performed while standing or involving a moderate level of physical activity, and heavy work may correspond to work involving the carrying of heavy objects or high-intensity physical labor. Work intensity information (620) may be obtained from the workplace's personnel management system or manually set by the cafeteria manager.

[0116] According to one embodiment of the present disclosure, the processor (232) can calculate a reference calorie amount based on industry group information (610) and work intensity information (620). The reference calorie amount can be calculated based on Equation 5.

[0117]

[0118] Referring to Equation 5, E_target may indicate the standard caloric intake (kcal), E_base(c) may indicate the basic caloric intake corresponding to industry group c, and α(w) may indicate an activity correction factor corresponding to work intensity w. For example, for office work (light work), E_base may be set to 2,000 kcal and α to 1.0, so that the standard caloric intake is calculated as 2,000 kcal. For construction work (heavy work), E_base may be set to 2,000 kcal and α to 1.4, so that the standard caloric intake is calculated as 2,800 kcal. E_base may correspond to the recommended daily basic caloric intake for adults that is pre-set for each industry group, and may be set by referring to accredited nutritional standards such as the Korean Dietetic Society's Reference Intakes for Koreans. The activity correction factor α can be set to 1.0 for light work, 1.2 for medium work, 1.4 for heavy work, etc., and the factor value may be applied differently depending on the work intensity grade. As the value of α increases, the standard caloric intake increases, so a diet with a higher caloric intake can be provided to workers with high levels of physical activity.

[0119] The processor (232) can calculate the target range for each nutrient based on the reference calorie (E_target). The target range for each nutrient can be calculated based on Equation 6.

[0120]

[0121]

[0122]

[0123] Referring to mathematical formula 6, r_carb, r_prot, and r_fat may indicate the range of energy ratios of each nutrient relative to total calories (e.g., carbohydrates 55–65%, protein 7–20%, fat 15–30%). 4 and 9 may indicate the calories per gram (kcal / g) of carbohydrates / protein and fat, respectively. For example, if the standard calorie intake is 2,000 kcal, the target range for carbohydrates can be calculated as [2,000 × 0.55 / 4, 2,000 × 0.65 / 4] = [275g, 325g], the target range for protein can be calculated as [2,000 × 0.07 / 4, 2,000 × 0.20 / 4] = [35g, 100g], and the target range for fat can be calculated as [2,000 × 0.15 / 9, 2,000 × 0.30 / 9] = [33g, 67g]. In the case where the standard calorie intake is 2,800 kcal (construction work, heavy work), the target range for each nutrient increases proportionally, and the target range for carbohydrates can be calculated as [385g, 455g], the target range for protein as [49g, 140g], and the target range for fat as [47g, 93g].

[0124] Each diet (diet K, diet M, diet L) shown in the diet data (630) may include information on calories, carbohydrates, protein, and fat, and the diet generation model may configure the diets such that the nutrient composition of each diet is within the target range of the standard nutritional requirements. For example, Diet K shown in FIG. 6 may include 400 kcal of calories, 45 g of carbohydrates, 30 g of protein, and 10 g of fat; Diet M may include 373 kcal of calories, 31 g of carbohydrates, 36 g of protein, and 11 g of fat; and Diet L may include 415 kcal of calories, 56 g of carbohydrates, 21 g of protein, and 13 g of fat. Diets K, M, and L illustrate only the major components of each meal as examples; additional items (rice, soup, side dishes, etc.) not shown in the actual diet data are included, so the sum of nutrients across three daily meals (breakfast, lunch, and dinner) can meet the reference caloric (E_target) and target ranges for each nutrient.

[0125] According to one embodiment of the present disclosure, a diet generation model can generate diet data (630) by performing multi-objective optimization that simultaneously considers a plurality of evaluation indicators. The diet generation objective function can be defined based on Equation 7.

[0126]

[0127] Referring to Equation 7, S represents a diet candidate, f_nutrition(S) represents a nutritional balance score (the inverse of the deviation from the standard nutritional requirement), f_cost(S) represents a food cost score (higher score the lower the total cost), f_preference(S) represents a user preference score (e.g., based on preference information in Fig. 5), and w_n, w_c, and w_p represent the weights of each item. The diet generation model can determine the diet candidate that maximizes F(S) as the diet data (630). f_nutrition(S) may have a higher value as the sum of the nutrients included in the diet candidate S is closer to the center of the target range of the standard nutritional requirement. For example, f_nutrition(S) may be calculated in proportion to the inverse of the Euclidean distance between the median of the target range for each nutrient calculated by Equation 6 and the actual nutrient value of the diet candidate S. f_cost(S) can have a higher score as the total cost of ingredients included in diet candidate S decreases, and can be calculated, for example, as f_cost(S) = 1 / (1 + C(S)) (where C(S) is the total cost of ingredients of diet candidate S). f_preference(S) can correspond to the diet preference score (Pref_m of Equation 4) described in Fig. 5.

[0128] The weights w_n, w_c, and w_p can be set differently depending on the operational policy of the cafeteria. For example, in a cafeteria that prioritizes health management, the value of w_n may be set relatively high to prioritize nutritional balance, while in a cafeteria that prioritizes cost reduction, the value of w_c may be set relatively high to generate a menu centered on inexpensive ingredients. It may be possible to normalize the weights so that their sum is 1 (e.g., w_n + w_c + w_p = 1).

[0129] According to one embodiment of the present disclosure, if work intensity information (620) differs by time period (e.g., heavy work in the morning, light work in the afternoon), it may be possible to apply standard nutritional requirements differently for each meal time period (breakfast, lunch, dinner). For example, in a workplace where heavy work is concentrated in the morning, the standard caloric intake for lunch may be set higher than the standard caloric intake for dinner. The processor (232) may individually calculate an activity correction coefficient for each meal time period based on the work intensity information by time period and generate meal data (630) by applying different standard nutritional requirements for each meal time period.

[0130] According to another embodiment of the present disclosure, it may be possible to additionally reflect seasonal or external environmental information in the calculation of the reference caloric value. For example, during the winter season, a predetermined correction value (e.g., +100 to 200 kcal) may be added to the reference caloric value to reflect the increase in basal metabolic rate for maintaining body temperature, and during the summer season, dietary composition for fluid and electrolyte replenishment may be given priority consideration.

[0131] According to various embodiments of the present disclosure, by automatically calculating standard nutritional requirements corresponding to the industry group and work intensity of the workplace to which the cafeteria belongs and generating meal data that satisfies the standard nutritional requirements, it becomes possible to provide customized nutrition suitable for the user's physical activity level, and the nutrition design process, which previously relied on the empirical judgment of a nutritionist, can be standardized.

[0132] FIG. 7 is a diagram showing an example of a flow for calculating the required amount (750) of food ingredients in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0133] Referring to FIGS. 3 and FIGS. 7, past meal history data (710) may correspond to a portion of the input data used in the process of generating meal data in step S340 of FIG. 3. Past meal history data (710) may record the number of people who actually used the meal at each meal time during a predetermined past period at the cafeteria, and may include daily, weekday, and monthly meal data. Past meal history data (710) may be accumulated and stored in the memory (231) of a server, and the actual number of meals for that day may be automatically recorded whenever the meal provision is completed. Meal patterns by day of the week (e.g., a tendency for Monday meals to be higher than Friday meals), seasonal fluctuations in meal numbers (e.g., a decrease in meals during the summer), or fluctuations in meal numbers due to specific events (e.g., internal events, holidays) may be reflected in the past meal history data (710).

[0134] The work schedule data (720) may include the number of employees scheduled to report to work, shift schedules, holiday information, etc., of the workplace to which the cafeteria belongs. The work schedule data (720) may be obtained in real time via an API from the workplace's personnel management system or attendance management system, or may be manually entered by a cafeteria manager. In the case of a workplace with shift work, since the number of employees reporting to work for the day shift and the night shift may differ, the work schedule data (720) may include the number of employees scheduled to report to work individually for each meal time slot (breakfast, lunch, dinner).

[0135] As illustrated in FIG. 7, past meal history data (710) and work schedule data (720) can be combined to calculate the predicted meal count (730). The predicted meal count (730) can indicate the number of people predicted to use the meal at a specific future point in time (e.g., tomorrow, next week). The processor (232) can calculate the predicted meal count (730) by combining the patterns of the past meal history data (710) and the number of people scheduled to report to work from the work schedule data (720).

[0136] According to one embodiment of the present disclosure, the predicted number of formulas (730) can be calculated based on mathematical formula 8.

[0137]

[0138] Referring to Equation 8, N_pred represents the predicted number of meals, N_actual(t) represents the actual number of meals in the past t-th period, w_t represents a weight based on temporal proximity (higher weight for more recent data), T represents the number of reference periods, S_sched represents the number of people scheduled to come to work on the work schedule for that day, and S_avg represents the average number of people coming to work during the reference period. The term (S_sched / S_avg) may be a correction ratio to reflect changes in the work schedule (e.g., increase in the number of people coming to work immediately after a public holiday, decrease in the number of meals during night shift work, etc.).

[0139] For example, it can be assumed that the reference period T is 4 weeks (28 days), the weighted sum of the last week is 0.4, the weighted sum of the last 2 weeks is 0.3, the weighted sum of the last 3 weeks is 0.2, and the weighted sum of the last 4 weeks is 0.1. If the actual number of meals on the same day of the week during the last 4 weeks is 180, 175, 190, and 170 respectively, the number of people scheduled to come to work on that day (S_sched) is 200, and the average number of people coming to work during the reference period (S_avg) is 195, then the predicted number of meals (N_pred) can be calculated as (0.4 × 180 + 0.3 × 175 + 0.2 × 190 + 0.1 × 170) × (200 / 195) ≈ 179.5 × 1.026 ≈ 184. If the number of people scheduled to come to work is greater than the average number of people coming to work, the correction ratio (S_sched / S_avg) becomes greater than 1, so the predicted number of meals may be corrected upward, and if the opposite is true, it may be corrected downward.

[0140] The weight w_t can be calculated based on an exponential decay function. For example, w_t = exp(- × t) / Σ(k=1~T) exp(- It can be defined as × k), where λ indicates a damping constant. The larger the value of λ, the more rapidly the weights for recent data can increase and the more rapidly the weights for older data can decrease. The closer the value of λ is to 0, the more evenly weights can be assigned to data across all reference periods.

[0141] As illustrated in FIG. 7, the predicted number of meals (730) and the unit requirement (740) can be combined in the calculation of the required amount (750). The unit requirement (740) may indicate the amount of food ingredients required for one serving in a single meal and may be included in the meal data (e.g., 520 in FIG. 5). The unit requirement (740) may be determined based on standard unit requirements pre-registered for each food ingredient in the food ingredient master data (e.g., stored in the memory (231) in FIG. 2), or may be individually calculated for each meal by a meal creation model.

[0142] The required amount (750) can be calculated based on mathematical formula 9.

[0143]

[0144] Referring to mathematical formula 9, R_i may indicate the required amount of the i-th food ingredient, q_i may indicate the unit requirement of the i-th food ingredient (the amount of food ingredient required per person per meal), and N_pred may indicate the predicted number of meals. For example, if the unit requirement (q) of onion is 0.05 kg and the predicted number of meals (N_pred) is 184 people, the required amount (R) of onion can be calculated as 0.05 × 184 = 9.2 kg. If the unit requirement (q) of potato is 0.08 kg, the required amount (R) of potato can be calculated as 0.08 × 184 = 14.72 kg. The required amount (750) may correspond to the required amount per food ingredient included in the meal data generated in step S340 of FIG. 3, and may serve as the basis for calculating the additional order quantity in step S350 of FIG. 3.

[0145] According to one embodiment of the present disclosure, when multiple meals (breakfast, lunch, and dinner) are provided in a day, different predicted meal counts may be calculated for each meal time slot, and the required amount of each food ingredient included in the menu for each meal time slot may be calculated individually and then summed. For example, if the predicted number of meals for lunch is 184 and the predicted number of meals for dinner is 120, and onions are used for both lunch and dinner, the total required amount of onions may be calculated as (0.05 × 184) + (0.05 × 120) = 15.2 kg.

[0146] According to another embodiment of the present disclosure, it may be possible to calculate a predicted number of meals (730) from a past number of meals pattern using a machine learning-based prediction model (e.g., LSTM (long short-term memory), Prophet, etc.). The LSTM model learns the time series pattern of past number of meals history data (710) and can output a predicted number of meals that reflects the number of meals by day of the week and season, the long-term trend, and the seasonality. The Prophet model can effectively reflect the number of meals caused by irregular events such as public holidays or long weekends by modeling the trend, seasonality, and holiday effects of the time series data separately.

[0147] According to another embodiment of the present disclosure, external variables such as weather, season, and internal events may be utilized as additional inputs in the calculation of the predicted number of meals. For example, if there is a forecast of heavy rain or heavy snow, the number of employees coming to work may decrease, so the processor (232) may adjust the predicted number of meals (730) downward based on weather forecast data obtained from the weather agency API. If special events such as internal workshops or sports competitions are scheduled, the number of meals may increase compared to normal, so the predicted number of meals (730) may be adjusted upward based on internal event schedule data.

[0148] According to various embodiments of the present disclosure, by combining past meal history data and work schedule data to calculate the predicted number of meals and precisely calculating the required amount of each food ingredient based on the predicted number of meals, the process of determining the order quantity, which previously relied on the empirical estimation of a nutritionist, can be standardized based on data, and the failure to meet the meal requirement due to under-ordering of food ingredients or the disposal of leftover food ingredients due to over-ordering can be minimized.

[0149] FIG. 8 is a drawing showing an example of a user interface (UI) of an artificial intelligence-based food ingredient management service in an artificial intelligence-based food ingredient management system according to various embodiments of the present disclosure.

[0150] Referring to FIGS. 1, 3 and 8, the screen (800) may include an image area (810) and an additional order form area (820). The screen (800) may be displayed on the display of the user terminal (210) and may be rendered by a food ingredient management application installed on the user terminal.

[0151] The image area (810) can display an image of the food storage area (130) captured by the user (110) in FIG. 1. The image area (810) can display an image with the inventory recognition result (see FIG. 4) by a vision AI model applied. For example, the bounding box and identification label of each food item are superimposed on the captured image of the food storage area in the image area (810), so that the user can visually check the recognition result of the vision AI model.

[0152] The additional order form area (820) can display the additional order quantity calculated in step S350 of FIG. 3 for each food ingredient. For example, the additional order form area (820) can display order quantity information for each food ingredient, such as "Onion: I will order an additional 7kg" or "Potato: I will order an additional 20kg". The order quantity for each food ingredient displayed in the additional order form area (820) can correspond to the value calculated in step S350 based on the inventory information (estimated inventory amount) generated in step S320 of FIG. 3 and the meal data (required amount per food ingredient) generated in step S340.

[0153] The "Send Order" button at the bottom of the screen (800) can perform the function of transmitting order data to the supplier system in response to the user's confirmation input. When the user (110) touches the "Send Order" button after confirming the order quantity for each food ingredient displayed in the additional order form area (820), the processor (e.g., 212 in FIG. 2) of the user terminal (210) can transmit an order approval request to the server (230) via a communication module (e.g., 213 in FIG. 2). The server's processor (232) can transmit the order data to the supplier system in response to receiving the order approval request, according to step S360 of FIG. 3.

[0154] The UI of FIG. 8 is exemplary, and the screen composition, layout, and display items may vary depending on the implementation. For example, the vertical arrangement of the image area (810) and the additional order form area (820) may be changed to a horizontal arrangement, and the number and format of information items displayed in the additional order form area (820) may be configured differently depending on the operation policy of the cafeteria.

[0155] According to one embodiment of the present disclosure, it may be possible to automatically place an order without manual confirmation by the user via the "order transmission" button. In the automatic ordering embodiment, the processor (232) can automatically perform step S360 without separate confirmation input by the user after calculating the additional order quantity in step S350 of FIG. 3 and then transmit the order data to the supplier system. Whether automatic ordering is enabled can be set by the cafeteria manager, and even when automatic ordering is enabled, an order completion notification can be displayed on the user terminal (210).

[0156] According to another embodiment of the present disclosure, it may be possible for a user to manually modify the order quantity in the additional order form area (820) and then transmit the order. The order quantity for each food ingredient in the additional order form area (820) may be displayed as an editable input field, and the user may increase or decrease the order quantity of a specific food ingredient and then touch the "Transmit Order" button. The processor (232) may update the order data to reflect the order quantity modified by the user and then transmit it to the supplier system.

[0157] According to another embodiment of the present disclosure, an order history lookup function and a cost total display function may be added to the screen (800). The order history lookup function may display past order data (order date, order quantity per food ingredient, supplier, delivery status, etc.) in a chronological list format. The cost total display function may display the total estimated order cost at the bottom of the screen (800), calculated by multiplying the order quantity per food ingredient displayed in the additional order order area (820) by the unit price information included in the food ingredient master data (e.g., stored in the memory (231) of FIG. 2). The cost total display function may support the user in checking the estimated cost before placing an order and in making decisions to adjust the order quantity within the budget range.

[0158] According to one embodiment of the present disclosure, a meal preview area may be additionally included in the screen (800). The meal preview area may summarize and display the composition of meal candidates (menu name, main ingredients, nutrient information, etc.) included in the meal data generated in step S340 of FIG. 3. After checking the generated meal in the meal preview area, the user (110) may change or approve the meal candidates as needed.

[0159] According to various embodiments of the present disclosure, a specific user interface is provided that integrally displays an image (810) of a food storage area and an additional order form (820) on a screen (800) of a user terminal and transmits order data to a supplier system in response to a user's confirmation input, thereby enabling a series of processes leading to the verification of food inventory recognition results, review of order details, and order transmission to be performed on a single screen. A specific technical implementation can be performed in which the recognition results of a vision artificial intelligence model and the calculated order quantity are visually provided to the user through the user interface, and data transmission between the server and the supplier system is initiated by the user's physical input (touch input).

[0160] Methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software. Methods according to the embodiments may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium, or may be implemented as a computer program stored on a computer-readable recording medium in combination with hardware.

[0161] When implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). One or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. One or more programs include instructions that cause the electronic device to execute methods according to the claims or embodiments described in the specification of this disclosure.

[0162] Such programs (software modules, software) may be stored in random access memory, non-volatile memory including flash memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic disc storage devices, compact disc-ROM (CD-ROM), digital versatile discs (DVDs), or other forms of optical storage devices, magnetic cassettes. Alternatively, they may be stored in memory composed of some or all of these. Additionally, each constituent memory may include multiple units.

[0163] Additionally, the program may be stored on an attachable storage device that can be accessed via a communication network such as the Internet, Intranet, LAN (local area network), WAN (wide area network), or SAN (storage area network), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure through an external port. Additionally, a separate storage device on a communication network may be connected to a device performing an embodiment of the present disclosure.

[0164] In the specific embodiments of the present disclosure described above, the components included in the disclosure are expressed in a singular or plural form according to the specific embodiments presented. However, the singular or plural expression is selected to suit the situation presented for convenience of explanation, and the present disclosure is not limited to singular or plural components; even if a component is expressed in the plural form, it may be composed of a singular form, and even if a component is expressed in the singular form, it may be composed of a plural form.

[0165] Meanwhile, although specific embodiments have been described in the detailed description of the present disclosure, it is understood that various modifications are possible within the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof. Explanation of the symbols

[0167] 110: User 120: User terminal 130: Food storage area 132: Food ingredients

Claims

Claim 1 An artificial intelligence-based food ingredient management method performed by one or more processors, comprising: acquiring one or more images of a food ingredient storage area of ​​a cafeteria; generating inventory information including identification information and an estimated inventory quantity for each of one or more food ingredients within the food ingredient storage area based on the one or more images using a vision artificial intelligence model; acquiring facility profile data and user data associated with the cafeteria; generating menu data including the required quantity for each of the one or more food ingredients based on the inventory information, the facility profile data, and the user data using a menu generation model; and calculating an additional order quantity for each of the one or more food ingredients by subtracting the estimated inventory quantity included in the inventory information from the required quantity. The method comprises the step of transmitting order data including the additional order quantity to a supplier system; wherein the facility profile data includes industry group information of the workplace to which the food service establishment belongs and work intensity information of the workplace, and the step of generating the meal data includes generating the meal data to satisfy standard nutritional requirements corresponding to the industry group information and the work intensity information, and wherein the standard nutritional requirements include target ranges for calories, carbohydrates, protein, and fat, respectively. Claim 2 In claim 1, the step of generating the inventory information comprises: generating the identification information including the location and type of each of the one or more food ingredients by performing object detection on the one or more images; calculating the estimated inventory amount for each of the one or more food ingredients based on the size of the area occupied by each of the one or more food ingredients within the one or more images; excluding from the inventory information food ingredients for which the identification confidence score output by the vision AI model for each of the one or more food ingredients is less than a preset threshold; and summing the estimated inventory amounts of food ingredients identified as the same type at multiple locations within the one or more images. Claim 3 An artificial intelligence-based food ingredient management method according to claim 1, wherein the user data includes at least one of allergy information or health status information of each of a plurality of users of the food service facility, and the method further includes the step of excluding food ingredients corresponding to the constraint from the meal data by reflecting the allergy information or health status information included in the user data as a constraint. Claim 4 delete Claim 5 In claim 1, the step of generating the meal data comprises: a step of calculating a predicted number of meals based on past meal history data of the cafeteria and work schedule data of the workplace to which the cafeteria belongs; and a step of calculating the required amount of each of the one or more food ingredients based on the unit requirement of each of the one or more food ingredients and the predicted number of meals, wherein the predicted number of meals is calculated based on the following mathematical formula. In the above mathematical formula, N_pred is the predicted number of meals, N_actual(t) is the actual number of meals in the past t-th period included in the past meal history data, w_t is a weight based on temporal proximity, T is the number of reference periods, S_sched is the number of people scheduled to come to work on the corresponding day included in the work schedule data, and S_avg is the average number of people coming to work during the reference period, an artificial intelligence-based food ingredient management method.

Images