Information processing device, information processing method and information processing program

Abstract

To provide an information processing device, an information processing method, and an information processing program that make it possible to estimate a placement state of a food.SOLUTION: An information processing device includes: an acquisition unit that acquires image information generated by capturing images of containers on which multiple types of foods are placed; a determination unit that recognizes a food recorded in the image information acquired by the acquisition unit and, when estimating a placement state of the food, determines whether the food can be estimated using a first learned model in which learning target foods corresponding to the food and placement states of the learning target foods are learned, or whether the learning target foods corresponding to the food have not been used in the learning and therefore the food cannot be estimated using the first learned model; and a learning unit that, if the determination unit determines that the food cannot be estimated using the first learned model, performs learning using the foods recorded in the image information acquired by the acquisition unit to generate a second learned model.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, there are restaurants in which a plate with sushi is placed on a endless conveyor and circulated around the customer seats. The system described in Patent Document 1 is used in such a restaurant. It captures an image of the plate moving on the conveyor from above, removes the pattern described on the plate, and then recognizes the nigiri. Further, the system measures the perimeter of the recognized nigiri, and outputs a warning when the perimeter is longer than the perimeter in the normal state (when the ingredients are displaced or fallen).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, the perimeter of the nigiri varies depending on the ingredient. For this reason, since the system described in Patent Document 1 does not recognize the type of nigiri, it may not be able to measure the perimeter for each type of nigiri (for each ingredient). Further, although the system described in Patent Document 1 removes the pattern described on the plate, there is a risk that the pattern on the plate cannot be removed when there is a smear or the like on the pattern. In that case, there is a possibility that the perimeter of the nigiri cannot be accurately measured.

[0005] The present disclosure provides an information processing apparatus, an information processing method, and an information processing program that enable estimation of the placement state of food.

Means for Solving the Problems

[0006] An information processing device of one embodiment includes an acquisition unit that acquires image information generated by imaging a container on which multiple types of food are placed, a determination unit that recognizes the food recorded in the image information acquired by the acquisition unit and, when estimating the placement state of the food, determines whether the food can be estimated using a first trained model that has learned the corresponding training target food and the placement state of the training target food, or whether the corresponding training target food has not been used in learning and therefore the food cannot be estimated using the first trained model, and a learning unit that, if the determination unit determines that the food cannot be estimated using the first trained model, performs learning using the food recorded in the image information acquired by the acquisition unit to generate a second trained model. [Effects of the Invention]

[0007] The information processing device, information processing method, and information processing program of the present disclosure can estimate the placement state of food. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an information processing device according to an embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. [Figure 3] 1A and 1B are diagrams for explaining the placement of nigiri as an example of food, in which (A) shows an example of an inappropriate placement of nigiri, and (B) shows an example of an appropriate placement of nigiri. [Figure 4] 1A and 1B are diagrams for explaining the placement state of a warship as an example of food, where (A) shows an example of a case where the placement state of the warship is inappropriate, and (B) shows an example of a case where the placement state of the warship is appropriate. [Figure 5] 1 is a first flowchart illustrating an information processing method according to an embodiment. [Figure 6] 10 is a second flowchart illustrating an information processing method according to an embodiment. [Figure 7]A third flowchart for explaining an information processing method according to an embodiment.

Embodiments for Carrying Out the Invention

[0009] Hereinafter, an embodiment will be described.

[0010] [Overview of Information Processing Apparatus 100] First, an overview of the information processing apparatus 100 according to an embodiment will be described. FIG. 1 is a diagram for explaining the information processing apparatus 100 according to an embodiment.

[0011] The information processing apparatus 100 may be configured as, for example, a determination apparatus that determines whether it is possible to estimate the placement state of the food 20 placed in the container by using a learned model. Further, when the information processing apparatus 100 cannot estimate the placement state of the food 20 placed in the container by using a learned model because it has not learned, for example, it may be configured as a learning apparatus that performs learning of the placement state of the food 20. Further, the information processing apparatus 100 may be configured as, for example, an estimation (determination) apparatus that estimates (determines) the placement state of the food 20 placed in the container. Further, the information processing apparatus 100 may be configured as, for example, a control apparatus (prohibition apparatus) that controls to prohibit providing the food 20 to a customer when the placement state of the food 20 is inappropriate. The information processing apparatus 100 is not limited to the apparatus of the above-described example, and may be configured as various apparatuses. The information processing apparatus 100 may be, for example, a computer such as a server, a desktop, a laptop, a tablet, and a smartphone.

[0012] The information processing apparatus 100 acquires a plurality of pieces of image information generated by imaging a container on which the food 20 is placed. The food 20 may be, for example, sushi placed on a plate (container), side orders placed on a plate (container), and bowls, or various other foods. The sushi may be, for example, nigiri, gunkan, maki, and hosomaki. The side orders may be, for example, fried foods, steamed eggs, and desserts.

[0013] Based on the learned model and the image information, the information processing apparatus 100 estimates the placement state of the food 20 recorded in the image information (the placement state of the food 20 placed on the container). The learned model may be, for example, a model generated by learning the appropriate placement state of the food 20 and the inappropriate placement state of the food 20. The container on which the food 20 is placed may be, for example, a placement member such as a plate, dish, bowl, tray, pack, box, and bento box.

[0014] The information processing apparatus 100 determines whether it is possible to estimate the placement state of the food 20 placed on the container using the first learned model. As an example, since the menu (food 20 (product)) already provided in the restaurant is a learning target, the information processing apparatus 100 determines that it is possible to estimate the placement state of the food 20 using the first learned model. On the other hand, as an example, since the new menu (new food (new product)) provided by a fair or the like is not a learning target, the information processing apparatus 100 determines that it is not possible to estimate the placement state of the new food using the first learned model. Here, when estimating the placement state of the food 20 using the learned model, it is necessary to learn the placement state of the food 20 in advance. However, it is difficult for the restaurant operation to learn the fair menu that is frequently changed each time.

[0015] When recognizing food 20 recorded in image information and estimating the placement state of food 20, information processing device 100 determines whether it is possible to estimate the placement state of food 20 using a first trained model that has learned about food 20 (learning food) and the placement state of that learning food. Here, a case in which it is not possible to estimate the placement state of food 20 using the first trained model may be, for example, a case in which the food 20 (learning food) and the placement state of that food 20 (learning food) have not been learned and therefore cannot be estimated using the first trained model.

[0016] When the information processing device 100 determines that the food 20 cannot be estimated using the first trained model, it performs training using the image information acquired as described above, i.e., the food 20 recorded in the image information and the placement state of the food 20, to generate a second trained model. That is, as an example, the information processing device 100 also learns about new menu items (new foods (new products)) offered at fairs and the like, and estimates the placement state of the new foods using the second trained model. The information processing device 100 automatically learns new menu items without having to learn them in advance each time a new menu item is introduced, which is suitable for restaurant operations.

[0017] To summarize the above overview, the information processing device 100, for example, (1) When there is an unlearned food 20 (product) when estimating the placement state of the food 20, (2) A captured image (image information) of the food 20 (product) is linked to the order information (food name (product name)) of the food 20 and stored; (3) The results of linking and storing the data in (2) above are analyzed (learned) as learning targets. (4) The learning target is added to a trained model (AI model) that estimates the placement state of food 20, (5) The food item 20 (product) has been learned. Processing may be performed.

[0018] [Details of the information processing device 100] Next, the information processing apparatus 100 according to an embodiment will be described in detail. FIG. 2 is a block diagram for explaining the information processing apparatus 100 according to an embodiment.

[0019] The information processing apparatus 100 includes, for example, a communication unit 121, a storage unit 122, a display unit 123, an imaging unit 124, a control unit 110, and the like. The control unit 110 includes, for example, an acquisition unit 111, a determination unit 112, a learning unit 113, a first estimation unit 114, a second estimation unit 115, a reception unit 116, a third estimation unit 117, a conveyance control unit 118, and the like. The control unit 110 may be configured by, for example, an arithmetic processing unit of the information processing apparatus 100. The control unit 110 (for example, an arithmetic processing unit or the like) may realize the functions of each unit by appropriately reading and executing various programs and the like stored in the storage unit 122 and the like. That is, the functions of each unit may be realized by computer implementation.

[0020] The communication unit 121 is, for example, a communication interface capable of transmitting and receiving various information to and from a device (external device) outside the information processing apparatus 100. The external device may be, for example, a conveyance lane 200 (a motor (not shown) for driving the conveyance lane 200) or the like.

[0021] The storage unit 122 may store, for example, various information and programs. An example of the storage unit 122 may be a memory, a solid state drive, a hard disk drive, or the like. Note that the storage unit 122 may be, for example, a storage area and a server on the cloud or the like.

[0022] The display unit 123 is, for example, a display capable of displaying various characters, symbols, images, and the like.

[0023] (Learning process) First, the function for performing the learning process will be described here.

[0024] The acquisition unit 111 acquires the image information generated by imaging the container on which the food 20 is placed. There may be a plurality of containers. Also, there may be a plurality of types of the food 20 (products). As an example, the acquisition unit 111 may acquire the image information from the imaging unit 124 that images the food 20. Also, as an example, when the acquisition unit 111 obtains the image information from the second estimation unit 115 and the third estimation unit 117 described later, or when the image information is recorded in the storage unit 122 by the processing of the second estimation unit 115 and the third estimation unit 117, the acquisition unit 111 may acquire the image information of the food 20 recorded in the storage unit 122. In this case, when acquiring the image information, the acquisition unit 111 may acquire the information on the type (for example, product name) of the food 20 that is specified in the display unit 123 described later and associated with the image information, or the information on the type (product name) of the food 20 included in the order information described later.

[0025] The food 20 (the container on which the food 20 is placed) is placed on a transport lane 200 (or a transport conveyor or the like) arranged from the kitchen to the customer seats and provided to the customer. The transport lane 200 may be linear, endless in a circular shape, or various other shapes. The imaging unit 124 is arranged above the transport lane 200 and generates image information by imaging the food 20 (the container on which the food 20 is placed) on the transport lane 200. The imaging unit 124 may be, for example, a camera or the like.

[0026] When the determination unit 112 recognizes the food 20 recorded in the image information acquired by the acquisition unit 111 and estimates the placement state of the food 20, it determines whether the food 20 can be estimated using the first learned model that has learned the learning target food corresponding to the food 20 and the placement state of the learning target food, or whether the food 20 cannot be estimated by the first learned model because the learning target food corresponding to the food 20 has not been used for learning. The food 20 may be recognized by, for example, using pattern recognition such as image recognition to recognize the food 20 and the type (product name) of the food 20. The food 20 may be associated with the type of food 20 in advance. After recognizing the food 20, the determination unit 112 can estimate the placement state of the food 20 based on the recognition result (the recognized food 20) and the first trained model. The first trained model may be a model generated by learning the food 20 (learning food) and the placement state of the food 20 (learning food). The first trained model may also be a model generated by learning the type (product name) of the food 20 (learning food), the food 20 (learning food), and the placement state of the food 20 (learning food).

[0027] 3A and 3B are diagrams for explaining the placement state of a nigiri as an example of food 20. Fig. 3A shows an example of a case where the nigiri is not placed properly, and Fig. 3B shows an example of a case where the nigiri is placed properly. Figure 4 is a diagram for explaining the placement state of a warship as an example of food 20. Figure 4(A) shows an example of a case where the placement state of the warship is inappropriate, and Figure 4(B) shows an example of a case where the placement state of the warship is appropriate.

[0028] The placement state of food 20 (learning target food) used for learning may be, for example, a state in which food 20 (sushi 22) is placed appropriately on plate 21 (container), a state in which food 20 (sushi 22) is placed improperly on plate 21 (container), etc. Note that food 20 is not limited to sushi 22 such as nigiri, gunkan, hosomaki, and tsutsumi, which will be described later, but may be various other dishes.

[0029] A state where the placement state of the food 20 is inappropriate may be, for example, in the case of nigiri (sushi 22), a state where the topping has shifted from the rice (see Fig. 3(A)) and a state where it has fallen, and in the case of gunkan (sushi 22), a state where the nori seaweed has peeled off and a state where it has fallen (see Fig. 4(A)). Furthermore, in the case of hosomaki (food), a state where the nori seaweed has peeled off and a state where the hosomaki has rolled and is placed on a plate (container), etc., may be a state where the food 20 (sushi 22) is not properly placed on the plate 21 (container) and cannot be provided to the customer.

[0030] A state where the placement state of the food 20 is appropriate may be, for example, in the case of nigiri (sushi 22), a state where the topping is properly placed on the rice and a state where the nigiri has not fallen (see Fig. 3(B)), and in the case of gunkan (sushi 22), a state where the nori seaweed has not peeled off and a state where it has not fallen (see Fig. 4(B)). Furthermore, in the case of hosomaki (food), a state where the nori seaweed is properly rolled and a state where the hosomaki is properly placed on the plate (container) without rolling, etc., may be a state where the food 20 (sushi 22) is properly placed on the plate 21 (container) and can be provided to the customer.

[0031] In addition, in the above-described embodiment, an example of performing estimation using the first pre-trained model after "recognition of food" has been described. However, in this embodiment, "recognition of food" does not necessarily have to be performed. In this case, the first pre-trained model may be a model generated by learning an image of the food 20 (food to be learned) and the placement state of the food 20 (food to be learned). Alternatively, the first pre-trained model may be a model generated by learning an image of the food 20 (food to be learned), the type (product name) of the food 20 (food to be learned), and the placement state of the food 20 (food to be learned). That is, when the determination unit 112 estimates the placement state of the food 20 recorded in the image information acquired by the acquisition unit 111, it determines whether the food 20 is a food that can be estimated using a first learned model that has learned the image of the learning target food corresponding to the food 20, the type (product name) of the learning target food, and the placement state of the learning target food, or a food that cannot be estimated by the first learned model because the learning target food corresponding to the food 20 has not been used for learning.

[0032] Note that the estimation units (the first estimation unit 114, the second estimation unit 115, and the third estimation unit 117) described later cannot estimate the placement state of the food 20 that was not a learning target when generating the first learned model using the first learned model. In this case, the determination unit 112 may determine that the food 20 recorded in the image information is a food for which the placement state cannot be estimated using the first learned model for foods that were not learning targets when generating the first learned model (unlearned foods). On the other hand, the determination unit 112 may determine that the food 20 recorded in the image information is a food for which the placement state can be estimated using the first learned model for foods that were learning targets when generating the first learned model (learning target foods).

[0033] When the learning unit 113 determines that the food 20 cannot be estimated using the first learned model by the determination unit 112, it performs learning using the food 20 recorded in the image information acquired by the acquisition unit 111 to generate a second learned model. In this case, the learning unit 113 may generate a second learned model by learning the food 20 recorded in the image information acquired by the acquisition unit 111 and the placement state of the food 20. That is, for the food 20 determined by the determination unit 112 as described above to be unable to have its placement state estimated using the first learned model, the learning unit 113 newly performs learning with that food 20 (the image of the food 20) as the learning target food (the image of the learning target food). The image of the food 20 may be, for example, an image of the food based on the image information acquired by the acquisition unit 111.

[0034] In this case, the learning unit 113 labels whether the placement state of the food 20 (the placement state of the food 20 recorded in the image) is appropriate or not, and learns the type of the food 20, the image of the food 20, and the placement state of the food 20 after labeling (the placement state of the food 20 recorded in the image) to generate a second learned model. The type of the food 20 may be set based on the type (for example, the product name) of the food 20 (product) for which designation is received in the display unit 123 described later, or may be set based on the type (product name) of the food 20 (product) included in the order information described later.

[0035] Here, for example, as the placement state of the food 20, the learning unit 113 may analyze the structure of the food 20 recorded in the image, that is, as an example, in the case of a nigiri, the structure of the shari and the neta, to classify whether it is an appropriate placement state or an inappropriate placement state. For example, as the analysis of the structure of the shari and the neta in the case of a nigiri, the learning unit 113 may consider that the laminated structure in which the neta is placed on the shari is an appropriate placement state, and the structure in which the neta is not placed on the shari is an inappropriate placement state. Also, for example, as the analysis of the structure of the shari and the neta in the case of a nigiri, the learning unit 113 may consider that the structure in which the neta is adjacent to the side of the shari (when the nigiri is falling over) is an inappropriate placement state. That is, when the learning unit 113 determines that the food 20 cannot be estimated using the first learned model by the determination unit 112, and determines that the food 20 is a nigiri, the learning unit 113 estimates the placement state of the nigiri (food 20) based on the structure of the shari and the neta recorded in the image information acquired by the acquisition unit 111, and learns the type of the nigiri (food 20), the image of the nigiri (food 20), and the estimated placement state of the nigiri (food 20) to generate a second learned model.

[0036] The second learned model may be the same model as the first learned model described above, that is, a model in which the first learned model and the second learned model are combined into one, or may be a model different from the first learned model (a model not combined into one).

[0037] When the acquisition unit 111 newly acquires image information, the first estimation unit 114 estimates the placement state of the food 20 based on the food 20 recorded in the image information and the second learned model generated by the learning unit 113. For the food 20 that was not a learning target in the first learned model described above, when the second learned model is generated later, the placement state can be estimated using the second learned model. As described above, the second learned model may be the same model (a single integrated model) as the first learned model. Therefore, for the food 20 that was a learning target when generating the first learned model and the food 20 that was a learning target when generating the second learned model, the first estimation unit 114 can estimate the placement state of the food 20 recorded in the newly acquired image information based on the image information newly acquired by the acquisition unit 111 and the first and second learned models.

[0038] Note that the first estimation unit 114, the second estimation unit 115 described later, and the third estimation unit 117 described later may function as a single "estimation unit" or may have different functions.

[0039] (Preprocessing) Next, the function of performing the process (preprocessing) in the stage before performing the above-described learning process will be described. In the preprocessing, the process of transporting the food 20 from the kitchen to the customer seat using the transport lane 200 (or a transport conveyor, etc.) arranged in a store such as a restaurant will be mainly described. Hereinafter, "transport conveyor" is included in the concept of "transport lane", and "transport lane (or a transport conveyor, etc.)" will be described as "transport lane".

[0040] As illustrated in FIG. 1, the conveyance lane 200 is arranged across from the kitchen to the customer seats. The conveyance lane 200 may be linear, endless in a circular shape, or various other shapes. The food 20 (the container on which the food 20 is placed) is placed on the conveyance lane 200 and provided to the customer. That is, the conveyance lane 200 conveys the container on which the food 20 is placed to the customer seats. Note that the food 20 is provided according to the customer's order. As an example, the customer uses an order terminal (not shown) arranged at the customer seat to input an order for the food 20. The order terminal receives the order from the customer and transmits it to the information processing device 100 as order information. Note that the order information may record information on the number (table number) of the customer seat where the customer is seated. The reception unit 116 of the information processing device 100 (for example, the control unit 110, etc.) receives, via the communication unit 121, the order information generated in response to the order of the food 20 by the customer in the customer seat. The staff of the restaurant performs cooking according to the order information (the customer's order) and provides the cooked food 20 (product) to the customer. At this time, the conveyance lane 200 conveys the container on which the food 20 cooked based on the order information is placed to the customer seats. The order information is managed for each conveyance lane 200.

[0041] The imaging unit 124 (for example, a camera, etc.) illustrated in FIG. 1 is arranged above the conveyance lane 200, generates image information by imaging the food 20 (the container on which the food 20 is placed) on the conveyance lane 200, and transmits the image information to the information processing device 100. The imaging unit 124 may be able to image a plurality of conveyance lanes 200 that convey the food 20, and images the containers arranged on each of the plurality of conveyance lanes 200 to generate image information. The imaging unit 124 performs imaging, for example, by still images or moving images, and generates image information. The imaging unit 124 may be arranged for each group (conveyance group) that conveniently combines a plurality of conveyance lanes 200 into one. That is, when there are a plurality of conveyance groups, the imaging unit 124 may be arranged at least one for each conveyance group. The imaging unit 124 may be capable of imaging, for example, a plurality of transport lanes 200 arranged in parallel (for example, two left and right transport lanes 200A, 200B, etc.). As an example, the imaging unit 124 may be capable of imaging the base end portion 201 side (kitchen side) of the linear transport lane 200. The imaging unit 124 may be capable of imaging one or a plurality of foods 20 arranged on the transport lane 200 for each transport lane 200. That is, the imaging unit 124 may be capable of imaging a plurality of foods 20 arranged on a plurality of lanes.

[0042] (First example of preprocessing) In the first example of preprocessing, when transporting the food 20 using the transport lane 200, first, after accepting the designation of the food 20 to be transported (product name such as the type of the food 20), an example in which the food 20 is placed on the transport lane 200 will be described.

[0043] The type (product name) of the food 20 may be the type of the product (food 20) provided to the customer, including, for example, maguro nigiri, salmon nigiri, sweet shrimp nigiri, maguro tataki warship, and takuan maki (thin roll), etc. That is, the type of the food 20 may be the product (product name) ordered by the customer, etc.

[0044] The display unit 123 accepts the designation of the type (product name) of the food 20 to be placed in the container to be placed on the transport lane 200. That is, the food 20 cooked in the kitchen (the food 20 placed in the container) is transported using the transport lane 200 when provided to the customer. At this time, when accepting the selection of the product name (the type of the food 20 based on the order information) displayed on the display unit 123, the transport lane 200 can transport the product (food 20) to the seat corresponding to the table number based on the order information (table number information) associated with the product (food 20). When the selection operation is performed by the employee, the display unit 123 accepts the selection of the product name (the type of the food 20) to be transported using the transport lane 200. After that, the employee places the product (the food 20 placed in the container) on the transport lane 200.

[0045] The second estimation unit 115 acquires image information generated in response to the imaging unit 124 imaging a product (food 20 placed in a container) placed on the transport lane 200. The acquisition of the image information may be performed via the acquisition unit 111 described above. The second estimation unit 115 estimates the placement state of the food 20 recorded in the image information based on the image information acquired by the acquisition unit 111 and the first learned model. The first learned model here may be a model that does not include the second learned model described above. That is, the second estimation unit 115 may estimate the placement state of the food 20 based on the food 20 (type of food 20), the image recording the food 20, and the first learned model, assuming that the food 20 is placed on the transport lane 200 in response to receiving a designation of the type (product name) of the food 20 by the display unit 123.

[0046] Note that when the product (food 20) placed on the transport lane 200, that is, the type (product name) of the food 20 received a designation by the display unit 123, that is, the food 20 is a food that was not a learning target when generating the first learned model, the second estimation unit 115 does not estimate the placement state of the food 20 using the first learned model. In other words, when the product (food 20) placed on the transport lane 200, that is, the type (product name) of the food 20 received a designation by the display unit 123, that is, the food 20 is a food that was a learning target when generating the first learned model, the second estimation unit 115 estimates the placement state of the food 20 using the first learned model.

[0047] The transport lane 200 can transport a plurality of products (food 20 placed in a plurality of containers respectively) to the same passenger seat at once. In this case, the display unit 123 receives a designation of the type (a plurality of product names) of the food 20 placed in each of the plurality of containers respectively. In this case, when all of the types (multiple product names) of the plurality of foods 20 received by the display unit 123 are foods that were learning targets when generating the first learned model, the second estimation unit 115 uses the first learned model to estimate the placement states of all of the foods 20. On the other hand, when some or all of the types (multiple product names) of the plurality of foods 20 received by the display unit 123 are foods that were not learning targets when generating the first learned model, the second estimation unit 115 does not use the first learned model to estimate the placement states of all of the foods 20. That is, when all of the types of the foods 20 placed in the container on the transport lane 200 are already learned in the first learned model when the container (food 20) is placed on the transport lane 200 after the second estimation unit 115 receives the designation of the type of the food 20 by the display unit 123, the second estimation unit 115 may use the first learned model to estimate the placement state of the food 20. On the other hand, when some or all of the types of the foods 20 placed in the container on the transport lane 200 are not already learned in the first learned model when the container (food 20) is placed on the transport lane 200 after the second estimation unit 115 receives the designation of the type of the food 20 by the display unit 123, the second estimation unit 115 may not use the first learned model to estimate the placement state of the food 20.

[0048] When some or all of the types (multiple product names) of the plurality of foods 20 received by the display unit 123 are foods 20 that were not learning targets when generating the first learned model, the control unit 110 (for example, the second estimation unit 115, etc.) may input the image information recording all of the foods 20 (products) to the acquisition unit 111, determination unit 112, and learning unit 113 described above, or may store it in the storage unit 122. In this case, the control unit 110 (for example, the second estimation unit 115, etc.) may associate the information on the type (product name) of the food 20 received by the display unit 123 with the image information.

[0049] When the second estimation unit 115 estimates that the placement state of the food 20 is appropriate or when the second estimation unit 115 does not perform an estimation, the conveyance control unit 118 may control the conveyance lane 200 to convey the container on which the food 20 is placed to the passenger seat. That is, when the second estimation unit 115 estimates that the placement state of the food 20 is appropriate by using the first learned model, the conveyance control unit 118 controls the conveyance lane 200 to convey the food 20 (product) to the passenger seat. The conveyance control unit 118 controls, via the communication unit 121, to drive the motor that operates the conveyance lane 200. On the other hand, when the second estimation unit 115 estimates that the placement state of the food 20 is not appropriate by using the first learned model, the conveyance control unit 118 controls the conveyance lane 200 so as not to convey the food 20 (product) to the passenger seat. That is, the conveyance control unit 118 controls so as not to drive the motor that operates the conveyance lane 200. In addition, when the product (food 20 placed in the container) placed on the conveyance lane 200 is a food 20 that was not a learning target when generating the first learned model, and the second estimation unit 115 does not estimate the placement state of the food 20 by using the first learned model, the conveyance control unit 118 controls the conveyance lane 200 to convey the food 20 (product) to the passenger seat. That is, the conveyance control unit 118 controls, via the communication unit 121, to drive the motor that operates the conveyance lane 200.

[0050] (Second example of preprocessing) Next, in the second example of preprocessing, when conveying the food 20 by using the conveyance lane 200, an example will be described in which after the food 20 (product) is placed on the conveyance lane 200, a designation of the type (product name) of the food 20 (food 20 placed in the container) is received. The type (product name) of the food 20 may be the same as in the first example described above.

[0051] First, an employee places the merchandise (the food 20 placed on the container) on the transport lane 200. The transport lane 200 can transport a plurality of merchandise (the food 20 placed on each of the plurality of containers) to the same passenger seat at once. Therefore, the employee may place a plurality of merchandise (the food 20 placed on each of the plurality of containers) on the transport lane 200.

[0052] Next, the third estimation unit 117 acquires the image information generated in response to the merchandise (the food 20 placed on the container) placed on the transport lane 200 being imaged by the imaging unit 124. The acquisition of the image information may be performed via the acquisition unit 111 described above. When the third estimation unit 117 acquires the image information, it refers to one or a plurality of order information (or all order information excluding those already provided) received by the reception unit 116, and if some or all of the merchandise (the type of food 20 (the food 20)) recorded in the order information is food that was not a learning target when generating the first learned model, it does not estimate the placement state of the food 20 using the first learned model. In other words, the third estimation unit 117 refers to one or a plurality of order information (or all order information excluding those already provided) received by the reception unit 116, and if all of the merchandise (the type of food 20 (the food 20)) recorded in the order information is food that was a learning target when generating the first learned model, it estimates the placement state of the food 20 using the first learned model. The third estimation unit 117 estimates the placement state of the food 20 recorded in the image information based on the acquired image information and the first learned model. The first learned model here may be a model that does not include the second learned model described above.

[0053] That is, the third estimation unit 117 may estimate the placement state of the food 20 based on the first learned model that learned the type (product name) of the food 20 recorded in the order information, the image recording the food 20, and the placement state of the food 20, assuming that the type (product name) of the food 20 recorded in the order information is placed on the transport lane 200.

[0054] In addition, when the third estimation unit 117 associates the conveyance lane 200 with the seats (table numbers) where the food 20 (product) can be provided on the conveyance lane 200, it may refer to the order information (or all order information excluding those already provided) transmitted from all the seats (all order terminals) where the food 20 placed on the conveyance lane 200 can be provided, and determine whether the placement state of the food 20 (product) placed on the conveyance lane 200 can be estimated using the first learned model.

[0055] That is, when a container is placed on the conveyance lane 200, the third estimation unit 117 refers to the types of the food 20 recorded in the order information. When all types of the food 20 are already learned in the first learned model, it estimates the placement state of the food 20 using the first learned model. When some or all types of the food 20 are not already learned in the first learned model, it may not estimate the placement state of the food 20 using the first learned model.

[0056] When some or all of the types (multiple product names) of the multiple foods 20 recorded in the order information are foods 20 that were not learning targets when generating the first learned model, the control unit 110 (for example, the third estimation unit 117, etc.) may input the image information recording all of the foods 20 (products) to the acquisition unit 111, the determination unit 112, and the learning unit 113 described above, or may store it in the storage unit 122. In this case, the control unit 110 (for example, the third estimation unit 117, etc.) may associate the information on the types (product names) of the food 20 based on the order information with the image information.

[0057] After that, the employee selects one or more types (product names) of the plurality of foods 20 displayed on the display unit 123 among the types (plural product names) of the foods 20 placed on the conveyance lane 200. In this case, the display unit 123 receives the designation of the type of the food 20 placed in the container placed on the conveyance lane 200. That is, when a selection operation is performed by the employee, the display unit 123 receives the selection of the food 20 (product) to be conveyed using the conveyance lane 200. In other words, since the type (product name) of the food 20 and the seat (table number) that provides the food 20 (product) are associated based on the order information, the display unit 123 can identify the seat (table number) that provides the food 20 (product) in response to receiving the selection of the type (product name) of the food 20.

[0058] When the placement state of the food 20 is estimated to be appropriate by the third estimation unit 117 or when the third estimation unit 117 does not perform the estimation, the conveyance control unit 118 may control the conveyance lane 200 to convey the container on which the food 20 is placed to the seat when receiving the designation of the type of the food 20 by the display unit 123. That is, when the placement state of the food 20 is estimated to be appropriate by the third estimation unit 117 using the first learned model, the conveyance control unit 118 controls the conveyance lane 200 to convey the food 20 (product) to the seat. The conveyance control unit 118 controls to drive the motor that operates the conveyance lane 200 via the communication unit 121. On the other hand, when the third estimation unit 117 estimates that the placement state of the food 20 is not appropriate using the first learned model, the conveyance control unit 118 controls the conveyance lane 200 so as not to convey the food 20 (product) to the seat. That is, the conveyance control unit 118 controls not to drive the motor that operates the conveyance lane 200. In addition, when the food product 20 (food 20 placed in a container) placed on the conveyance lane 200 is not a food product 20 that was a learning target when generating the first learned model, if the second estimator 115 does not estimate the placement state of the food product 20 using the first learned model, the conveyance control unit 118 controls the conveyance lane 200 to convey the food product 20 (product) to the passenger seat. That is, the conveyance control unit 118 controls, via the communication unit 121, to drive a motor that operates the conveyance lane 200.

[0059] [Information Processing Method] Next, an information processing method according to an embodiment will be described.

[0060] (Learning Process) First, an example of the learning process in the information processing method will be described. FIG. 5 is a first flowchart for explaining an information processing method according to an embodiment.

[0061] In step ST101, the acquisition unit 111 acquires image information generated by imaging a container on which a plurality of types of food products 20 are placed.

[0062] In step ST102, when estimating the placement state of the food product 20 recognized from the image information acquired in step ST101, the determination unit 112 determines whether the food product 20 can be estimated using a first learned model that has learned a learning target food product corresponding to the food product 20 and the placement state of the learning target food product, or whether the food product 20 cannot be estimated using the first learned model because the learning target food product corresponding to the food product 20 has not been used for learning. If it is determined in step ST102 that the placement state cannot be estimated (No), the process proceeds to step ST103. If it is determined in step ST102 that the placement state can be estimated (Yes), the process proceeds to step ST104.

[0063] In step ST103, when the learning unit 113 determines that the food 20 cannot be estimated using the first learned model in step ST102 (when the answer in step ST102 is No), the learning unit 113 performs learning using the food 20 recorded in the image information acquired in step ST101 to generate a second learned model. In this case, the learning unit 113 may learn the food 20 recorded in the image information acquired in step ST101 and the placement state of the food 20 to generate a second learned model. Further, the learning unit 113 may learn the food 20 recorded in the image information acquired in step ST101, the type (product name) of the food 20 associated with the image information, and the placement state of the food 20 recorded in the image information to generate a second learned model.

[0064] In step ST104, when the first estimation unit 114 newly acquires image information by the acquisition unit 111, the first estimation unit 114 estimates the placement state of the food 20 based on the food 20 recorded in the image information and the second learned model generated in step ST103. Further, when the first estimation unit 114 newly acquires image information by the acquisition unit 111, the first estimation unit 114 may estimate the placement state of the food 20 based on the food 20 recorded in the image information and the first learned model.

[0065] (First example of preprocessing) Next, a first example of preprocessing in the information processing method will be described. FIG. 6 is a second flowchart for explaining an information processing method according to an embodiment.

[0066] In step ST201, the display unit 123 receives a designation of the type (product name) of the food 20 to be placed in a container to be placed on the transport lane 200.

[0067] In step ST202, when the determination unit 112 is imaged by the imaging unit 124 when the container (food 20) is placed on the transport lane 200 after receiving the designation of the type of food 20 in step ST201, based on the type of food 20 received in step ST201, it is determined whether the food 20 is a food that can be estimated using the first learned model that has learned the learning target food (image) corresponding to the food 20, the type (product name) of the learning target product, and the placement state of the learning target food, or whether the food 20 is a food that cannot be estimated by the first learned model because the learning target food corresponding to the food 20 has not been used for learning. In step ST202, when it is determined that the placement state can be estimated (Yes), the process proceeds to step ST203. In step ST202, when it is determined that the placement state cannot be estimated (No), the process proceeds to step ST204.

[0068] In step ST203, when all types of the food 20 placed in the container placed on the transport lane 200 have already been learned in the first learned model when the container (food 20) is placed on the transport lane 200 after receiving the designation of the type of food 20 in step ST201 (when Yes in step ST202), the placement state of the food 20 is estimated based on the image information of the food 20 and the first learned model.

[0069] In step ST204, when some or all types of the food 20 placed in the container placed on the transport lane 200 have not already been learned in the first learned model when the container (food 20) is placed on the transport lane 200 after receiving the designation of the type of food 20 in step ST201 (when No in step ST202), it is not necessary to estimate the placement state of the food 20 using the first learned model.

[0070] In step ST205, when the conveyance control unit 118 estimates that the placement state of the food 20 is appropriate in step ST203, and when no estimation is performed in step ST204 and the conveyance start button is further operated, the conveyance control unit 118 controls the conveyance lane 200 to convey the container on which the food 20 is placed to the passenger seat.

[0071] (Second Example of Pretreatment) First, a second example of pretreatment in the information processing method will be described. FIG. 7 is a third flowchart for explaining an information processing method according to an embodiment.

[0072] In step ST301, the reception unit 116 receives order information generated in response to an order for the food 20 by a customer sitting in the passenger seat.

[0073] In step ST302, after receiving the order information in step ST301, when the food 20 corresponding to the order information (order) is imaged by the imaging unit 124 by being placed on the conveyance lane 200, based on the type (product name) of the food 20 corresponding to the order information (order) received in step ST301, it is determined whether the food 20 can be estimated using a first learned model that has learned the learning target food (image) corresponding to the food 20, the type (product name) of the learning target product, and the placement state of the learning target food, or whether the food 20 cannot be estimated by the first learned model because the learning target food corresponding to the food 20 has not been used for learning. In step ST302, when it is determined that the placement state can be estimated (Yes), the process proceeds to step ST303. In step ST302, when it is determined that the placement state cannot be estimated (No), the process proceeds to step ST304.

[0074] In step ST303, when a container is placed on the transport lane 200, the third estimation unit 117 refers to the type of food 20 recorded in the order information received in step ST301. When all types of the food 20 are already learned in the first learned model (Yes in step ST302), based on the image information of the food 20 and the first learned model, the placement state of the food 20 is estimated.

[0075] In step ST304, when a container is placed on the transport lane 200, the third estimation unit 117 refers to the type of food 20 recorded in the order information received in step ST301. When some or all types of the food 20 are not already learned in the first learned model (No in step ST302), it may not be necessary to estimate the placement state of the food 20 using the first learned model.

[0076] In step ST305, the display unit 123 receives a designation of the type (product name) of the food 20 placed in the container on the transport lane 200. Since the type (product name) of the food 20 and the seat (table number) that provides the food 20 (product) are associated based on the order information received in step ST301, when the display unit 123 receives a selection of the type (product name) of the food 20, it is possible to identify the seat (table number) that provides the product (food 20).

[0077] In step ST306, when it is estimated in step ST303 that the placement state of the food 20 is appropriate, and when the transport start button is further operated without performing the estimation in step ST304, when the display unit 123 receives a designation of the type (product name) of the food 20 in step ST305, the transport control unit 118 controls the transport lane 200 to transport the container on which the food 20 is placed to the seat (table number).

[0078] [Regarding Functions and Circuits] Next, the functions and circuits of the information processing apparatus 100 described above will be explained. Each part of the information processing apparatus 100 may be realized as a function of a computer's arithmetic processing unit or the like. That is, the acquisition unit 111, determination unit 112, learning unit 113, first estimation unit 114, second estimation unit 115, reception unit 116, third estimation unit 117, and conveyance control unit 118 (control unit 110) of the information processing apparatus 100 may be realized as an acquisition function, determination function, learning function, first estimation function, second estimation function, reception function, third estimation function, and conveyance control function (control function) by a computer's arithmetic processing unit or the like, respectively. When the first estimation unit 114, second estimation unit 115, and third estimation unit 117 are grouped together as one "estimation unit", it may also be realized as an estimation function by a computer's arithmetic processing unit or the like. The information processing program can cause a computer to realize each of the above-described functions. The information processing program may be recorded on a non-transitory computer-readable storage medium such as a memory, solid state drive, hard disk drive, or optical disk. The storage medium may be, for example, rephrased as a non-transitory computer-readable medium that stores the information processing program. Also, the information processing program may be transmitted online. Also, as described above, each part of the information processing apparatus 100 may be realized by a computer's arithmetic processing unit or the like. The arithmetic processing unit or the like is composed of, for example, an integrated circuit or the like. Therefore, each part of the information processing apparatus 100 may be realized as a circuit that constitutes the arithmetic processing unit or the like. That is, the acquisition unit 111, determination unit 112, learning unit 113, first estimation unit 114, second estimation unit 115, reception unit 116, third estimation unit 117, and conveyance control unit 118 (control unit 110) of the information processing apparatus 100 may be realized as an acquisition circuit, determination circuit, learning circuit, first estimation circuit, second estimation circuit, reception circuit, third estimation circuit, and conveyance control circuit (control circuit) that constitute a computer's arithmetic processing unit or the like. When the first estimation unit 114, second estimation unit 115, and third estimation unit 117 are grouped together as one "estimation unit", it may also be realized as an estimation circuit that constitutes a computer's arithmetic processing unit or the like. The imaging unit 124, the communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as, for example, an imaging function including the functions of an arithmetic processing device, as well as a communication function, a storage function, and a display function (output function). The imaging unit 124, the communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as, for example, an imaging circuit, a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured using integrated circuits, etc. The imaging unit 124, the communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as, for example, an imaging device, a communication device, a storage device, and a display device (output device) by being configured using a plurality of devices.

[0079] The information processing device 100 can combine one or any combination of the above-mentioned multiple units. In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[0080] [Aspects and Effects of the Present Embodiment] Next, one aspect of this embodiment and the effects of each aspect will be described. Note that each aspect described below is an example at the time of filing, and this embodiment is not limited to the aspects described below. In other words, this embodiment is not limited to the aspects described below, and may be realized by appropriately combining the above-mentioned parts. Furthermore, a lower-level aspect may be able to cite any of the higher-level aspects. The effects of the present embodiment described below are merely examples, and the effects of each aspect are not limited to those described below. Each aspect may, for example, achieve at least one of the effects described below.

[0081] (Aspect 1) An information processing apparatus according to one aspect includes an acquisition unit that acquires image information generated by imaging a container on which a plurality of types of foods are placed, and recognizes the foods recorded in the image information acquired by the acquisition unit, and when estimating the placement state of the foods, determines whether the food is a food that can be estimated using a first learned model obtained by learning a learning target food corresponding to the food and the placement state of the learning target food, or a food that cannot be estimated by the first learned model because the learning target food corresponding to the food has not been used for learning, and a learning unit that generates a second learned model by performing learning using the foods recorded in the image information acquired by the acquisition unit when it is determined by the determination unit that the food cannot be estimated using the first learned model. Thereby, for foods that were not learning targets when generating the first learned model, the information processing apparatus can perform learning on those foods, enabling estimation of the placement state of the foods.

[0082] (Aspect 2) An information processing apparatus according to one aspect may include a first estimation unit that estimates the placement state of a food based on the food recorded in the image information and the second learned model generated by the learning unit when the acquisition unit newly acquires image information. Thereby, the information processing apparatus can also estimate the placement state of newly learned foods using the learned model.

[0083] (Aspect 3) In an information processing apparatus according to one aspect, the learning unit may generate a second learned model by learning the foods recorded in the image information acquired by the acquisition unit and the placement state of the foods. Thereby, the information processing apparatus can generate a learned model using image information and the like that record foods that were not learning targets in the first learned model.

[0084] (Aspect 4) An information processing apparatus according to one aspect includes: a conveyance lane that conveys a container on which food is placed to a passenger seat; a display unit that receives a designation of the type of food to be placed on the container to be placed in the conveyance lane; and a second estimation unit that, when the container is placed in the conveyance lane after receiving the designation of the type of food by the display unit, estimates the placement state of the food placed on the container using a first learned model if all types of food placed on the container to be placed in the conveyance lane have already been learned by the first learned model, and does not estimate the placement state of the food using the first learned model if some or all types of food placed on the container to be placed in the conveyance lane have not already been learned by the first learned model; and a conveyance control unit that controls the conveyance lane to convey the container on which food is placed to the passenger seat when the second estimation unit estimates that the placement state of the food is appropriate and when the second estimation unit does not perform the estimation. Accordingly, the information processing apparatus can identify the type of food to be placed on the conveyance lane by receiving a selection via the display unit. Further, the information processing apparatus can estimate the placement state of the food placed on the container based on the identified type of food, the food recorded in the image information, and the first learned model. On the other hand, the information processing apparatus can determine that it cannot estimate the placement state of the food placed on the container when the identified type of food is food that was not a learning target when generating the first learned model. When the information processing apparatus estimates that the placement state of the food (product) placed on the conveyance lane is appropriate using the first learned model, the information processing apparatus can convey the food (product) to the passenger seat.

[0085] (Aspect 5) An information processing apparatus according to one aspect includes: a reception unit that receives order information generated in response to an order for food by a customer sitting in a seat; a conveyance lane that conveys a container on which food prepared based on the order information is placed to the seat; a display unit that receives a designation of the type of food placed on the container placed in the conveyance lane; and a third estimation unit that, when the container is placed in the conveyance lane, refers to the type of food recorded in the order information, and estimates the placement state of the food using the first learned model if all types of the food have already been learned by the first learned model, and does not estimate the placement state of the food using the first learned model if some or all types of the food have not already been learned by the first learned model. When the third estimation unit estimates that the placement state of the food is appropriate and when the third estimation unit does not perform the estimation, a conveyance control unit that controls the conveyance lane to convey the container on which the food is placed to the seat when the display unit receives a designation of the type of food. Thereby, the information processing apparatus can specify the type of food placed on the conveyance lane based on the order information. Further, the information processing apparatus can estimate the placement state of the food placed on the container based on the specified type of food, the food recorded in the image information, and the first learned model. On the other hand, the information processing apparatus can determine that it cannot estimate the placement state of the food placed on the container when the specified type of food is a food that was not a learning target when generating the first learned model. When the information processing apparatus estimates that the placement state of the food (product) placed on the conveyance lane is appropriate using the first learned model, the information processing apparatus can convey the food (product) to the seat.

[0086] (Aspect 6) In a method of information processing according to one aspect, a computer executes an acquisition step of acquiring image information generated by imaging a container on which a plurality of types of foods are placed, and when recognizing the foods recorded in the image information acquired in the acquisition step and estimating the placement states of the foods, determines whether the foods can be estimated using a first learned model obtained by learning a learning target food corresponding to each food and the placement state of the learning target food, or whether the foods cannot be estimated using the first learned model because the learning target foods corresponding to the foods have not been used in learning. When it is determined in the determination step that the foods cannot be estimated using the first learned model, a learning step of performing learning using the foods recorded in the image information acquired in the acquisition step to generate a second learned model is executed. As a result, the information processing method can achieve the same effects as the information processing apparatus according to the above-described one aspect.

[0087] (Aspect 7) An information processing program according to one aspect causes a computer to realize an acquisition function of acquiring image information generated by imaging a container on which a plurality of types of foods are placed, a determination function of determining whether the foods can be estimated using a first learned model obtained by learning a learning target food corresponding to each food and the placement state of the learning target food, or whether the foods cannot be estimated using the first learned model because the learning target foods corresponding to the foods have not been used in learning, when recognizing the foods recorded in the image information acquired by the acquisition function and estimating the placement states of the foods, and a learning function of performing learning using the foods recorded in the image information acquired by the acquisition function to generate a second learned model when it is determined in the determination function that the foods cannot be estimated using the first learned model. As a result, the information processing program can achieve the same effects as the information processing apparatus according to the above-described one aspect.

Description of Reference Numerals

[0088] 100 Information processing apparatus 110 Control unit 111 Acquisition unit 112 Determination unit 113 Learning unit 114 First estimation unit (estimation unit) 115 Second estimation unit (estimation unit) 116 Reception unit 117 Third estimation unit (estimation unit) 118 Conveyor control unit 121 Communication unit 122 Memory unit 123 Display unit 124 Imaging unit 200(200A,200B) Conveyor lane 201 Base end (kitchen side) 20 Food 21 Dish 22 Sushi

Claims

1. An acquisition unit that acquires image information generated by imaging a container on which a plurality of types of food are placed; When recognizing the food recorded in the image information acquired by the acquisition unit and estimating the placement state of the food, it is possible to use a first learned model that has learned the learning target food corresponding to the food and the placement state of the learning target food to estimate whether it is a food that can be estimated, or a food that cannot be estimated by the first learned model because the learning target food corresponding to the food has not been used for learning. A determination unit for determining; When it is determined by the determination unit that the food cannot be estimated using the first learned model, a learning unit that performs learning using the food recorded in the image information acquired by the acquisition unit to generate a second learned model; An information processing apparatus comprising:

2. When newly acquiring image information by the acquisition unit, it includes a first estimation unit that estimates the placement state of the food based on the food recorded in the image information and the second learned model generated by the learning unit. The information processing apparatus according to claim 1.

3. The learning unit learns the food recorded in the image information acquired by the acquisition unit and the placement state of the food to generate a second learned model. The information processing apparatus according to claim 1.

4. A transport lane that transports a container on which food is placed to a passenger seat; A display unit that receives a designation of the type of food to be placed on a container to be placed on the transport lane; When a container is placed on the transport lane after receiving a designation of the type of food by the display unit, if all types of food placed on the container placed on the transport lane have already been learned in the first learned model, the first learned model is used to estimate the placement state of the food. If some or all of the types of food placed on the container placed on the transport lane have not already been learned in the first learned model, a second estimation unit that does not estimate the placement state of the food using the first learned model; A transport control unit that controls the transport lane to transport the container on which food is placed to the passenger seat when the second estimation unit estimates that the placement state of the food is appropriate and when the second estimation unit does not perform the estimation; The information processing apparatus according to any one of claims 1 to 3, comprising:

5. A reception unit that receives order information generated in response to an order for food by a customer sitting in a passenger seat; A conveyance lane that conveys a container on which food prepared based on the order information is placed to a passenger seat; A display unit that receives a designation of the type of food placed in the container placed on the conveyance lane; When a container is placed on the conveyance lane, referring to the types of food recorded in the order information, if all types of the food have already been learned by the first learned model, using the first learned model to estimate the placement state of the food, and if some or all types of the food have not already been learned by the first learned model, a third estimation unit that does not estimate the placement state of the food using the first learned model; When the third estimation unit estimates that the placement state of the food is appropriate and when the third estimation unit does not perform an estimation, when the display unit receives a designation of the type of food, a conveyance control unit that controls the conveyance lane to convey the container on which the food is placed to the passenger seat; The information processing apparatus according to any one of claims 1 to 3, comprising:

6. A computer: An acquisition step of acquiring image information generated by imaging a container on which a plurality of types of food are placed; When recognizing the food recorded in the image information acquired in the acquisition step and estimating the placement state of the food, determining whether the food is a food that can be estimated using a first learned model that has learned the learning target food corresponding to the food and the placement state of the learning target food, or a food that cannot be estimated by the first learned model because the learning target food corresponding to the food has not been used for learning; When it is determined in the determination step that the food cannot be estimated using the first learned model, a learning step of generating a second learned model by performing learning using the food recorded in the image information acquired in the acquisition step; An information processing method for executing.

7. In a computer: An acquisition function of acquiring image information generated by imaging a container on which a plurality of types of food are placed; When recognizing the food recorded in the image information acquired by the acquisition function and estimating the placement state of the food, determining whether the food is a food that can be estimated using a first learned model that has learned the learning target food corresponding to the food and the placement state of the learning target food, or a food that cannot be estimated by the first learned model because the learning target food corresponding to the food has not been used for learning; When it is determined by the determination function that a food item cannot be estimated using the first learned model, a learning function that generates a second learned model by performing learning using the food item recorded in the image information acquired by the acquisition function, and An information processing program for realizing this.

Images