System for operating a food serving system

A system with weighing and imaging sensors at food serving points trains machine learning algorithms to classify food content accurately, overcoming the limitations of existing systems by providing detailed nutritional information.

JP2024543949APending Publication Date: 2024-11-26UNIVERSITY OF TURKU
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024532287
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-01
Filing Date
2022-11-30
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing food serving systems can accurately determine the weight and energy content of consumed food but lack the capability to provide detailed nutritional information, limiting the value of the data for users and service providers.

Method used

Implement a system with multiple food serving points equipped with weighing devices and sensors that collect imaging data, linked to user identifiers, to train machine learning algorithms for accurate classification of food content based on sensor data and weighing results.

Benefits of technology

Enables detailed nutritional analysis and classification of food content, providing comprehensive information about user intake and meal composition using a trained model, reducing the need for extensive data collection at each point.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543949000001_ABST
    Figure 2024543949000001_ABST
Patent Text Reader

Abstract

According to one aspect, a system for managing a food serving system is provided, where food collected by a user is weighed and sensor data, e.g., image data, is obtained about the food collected by the user, which may be used as training data for a machine learning algorithm to build a model that enables subsequent classification of the food based at least on the sensor data associated with the food.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates generally to the field of data processing, and to solutions for operating and managing food serving systems, as well as machine learning solutions for interpreting and analyzing acquired and collected data. [Background technology]

[0002] Solutions exist for tracking the amount of food consumed by people having lunch, for example, and linking the consumed food to an identified user. For example, a lunch serving line may be provided with a weighing device for the served dishes and a reader for reading a user associated identifier, for example from a smart card. At a particular food collecting point, a user may first be identified by a smart card, after which the user may take a desired amount of a dish or food. The amount taken by the user is weighed and the weight information is associated with the user. By identifying a user at multiple food collecting points and associating food weight information with the user, it is possible to determine, for example, the energy content of the food selected by the user.

[0003] However, even if the weight and energy content can be accurately determined based on measurements, this information is only available if the food collected by the user and the amount of food collected were measured at the time of collecting the food. Furthermore, detailed information regarding nutritional value cannot be reasonably and accurately determined by weight alone, which requires additional parameters and analyses to give additional information to the end user and service provider. Summary of the Invention [Problem to be solved by the invention]

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] The objective of the present disclosure is to provide a technical solution for classifying food contents based on sensor data, e.g. imaging data, stored about the food. [Means for solving the problem]

[0006] The above object is achieved by means of the features of the attached independent claims. Further embodiments and examples are evident from the dependent claims, the detailed description and the accompanying drawings.

[0007] According to a first aspect, a method for providing a food service using a food collection session includes: a plurality of food serving points configured to serve food, each food serving point configured to serve a predetermined dish, each food serving point associated with a weighing device configured to weigh an amount of food collected from the food serving point to provide a weighing result, and a reader configured to read an identifier associated with a food collection session of a user; and at least one detection point, each detection point comprising at least one sensor configured to provide sensor data regarding food collected by a user, and a reader configured to read an identifier associated with a food collection session of a user; storing data relating to the food served at each of the plurality of food serving points; obtaining the weighing result and the identifier from each food serving point; and storing the data relating to the food served at each of the plurality of food serving points. and correlating the weighing results with the identifiers with each other; and a training system configured to acquire sensor data originating from at least one detection point, the sensor data being associated with an identifier; acquire from the management system the weighing results associated with the identifiers and data related to food served at the multiple food serving points; and build a model using at least the acquired sensor data associated with the identifiers, the weighing results associated with the identifiers, and the data related to food served at the multiple food serving points as training data for a machine learning algorithm, the model enabling subsequent classification of food based at least on the sensor data associated with the food.

[0008] In one implementation of the first aspect, the system further comprises a control unit configured to receive a trigger event and trigger storage of images of the food captured by the user by the at least one camera.

[0009] In one implementation of the first aspect, the control unit is configured to receive a trigger event from the reader.

[0010] In one implementation of the first aspect, the system further includes a control unit and a detection point at each of the plurality of food serving points, wherein the control unit at the food serving point is configured to receive a trigger event and trigger storage of sensor data associated with the food collected by a user by at least one sensor at the detection point.

[0011] In one implementation of the first aspect, the control unit is configured to receive a trigger event from a reader associated with the food serving point.

[0012] In one implementation of the first aspect, the control unit is configured to receive a trigger event from a metering device associated with the food serving point.

[0013] In one implementation of the first aspect, the data relating to the foods served at the multiple food serving points includes at least one of the foods served by each of the multiple food serving points and nutrient content information associated with each food served by each of the multiple food serving points.

[0014] In one implementation of the first aspect, the data system is configured to create a session having a session identifier when obtaining the identifier for the first time and determining that no active sessions exist, associate a session start time with the session, and link the identifier with the session having the session identifier.

[0015] In one implementation of the first aspect, the system further includes a user identifier reader, and the data system is configured to obtain a user identifier from the user identifier reader and associate the user identifier with the session.

[0016] In one implementation of the first aspect, the identifier includes a radio frequency identifier, a near field communication identifier, a bar code, a QR code, or a visually recognizable identifier associated with the tray used by the user.

[0017] In one implementation of the first aspect, the identifier includes an identifier associated with a user, a radio frequency identifier, a near field communication identifier, a smart wearable identifier, a smart ring identifier, a fingerprint, a biometric identifier, and a visually recognizable identifier.

[0018] In one implementation of the first aspect, the system further comprises a waste collection point comprising a weighing device configured to weigh an amount of biowaste left by a user to provide a waste weighing result, a reader configured to read a food collection session identifier associated with the user's food collection session, and at least one sensor configured to provide sensor data regarding the food left by the user, and the training system is configured to obtain the waste weighing result, the food collection session identifier associated with the user's food collection session, and the sensor data regarding the food left by the user, and provide additional information regarding the food left by the user based on the obtained waste weighing result, the food collection session identifier associated with the user's food collection session, and the sensor data regarding the food left by the user.

[0019] In one implementation of the first aspect, the at least one sensor includes at least one of a camera, a stereo camera, a depth camera, a multispectral camera, an infrared camera, an RGB camera, a spectroscopic sensor, a near-infrared sensor, a spectroscopic sensor, a photogrammetry sensor, a lidar sensor, a 3D scanner, and a photodetector.

[0020] In one implementation of the first aspect, the training system is configured to acquire additional sensor data and manually labeled data associated with the additional sensor data, and to complement the model using the acquired additional sensor data and the manually labeled data as training data for the machine learning algorithm.

[0021] According to a second aspect, there is provided a computer-implemented method comprising: acquiring sensor data relating to food collected by a user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of a food collected from a food serving point; acquiring data relating to food served at the plurality of food serving points; and building a model using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm to enable subsequent classification of foods based at least on the sensor data associated with the foods.

[0022] In one implementation of the second aspect, the method further includes acquiring additional sensor data and manually labeled data associated with the additional sensor data, and complementing the model using the acquired additional sensor data and the manually labeled data as training data for the machine learning algorithm.

[0023] In one implementation of the second aspect, the method further includes obtaining waste weighing results, a food collection session identifier associated with the user's food collection session, and sensor data regarding food left by the user, and building a model using the obtained waste weighing results, the food collection session identifier associated with the user's food collection session, and the sensor data regarding food left by the user as training data for a machine learning algorithm to enable subsequent classification of the food based at least on the sensor data associated with the food.

[0024] In one implementation of the second aspect, the data relating to foods served at the multiple food serving points includes at least one of foods served by each of the multiple food serving points and nutritional composition information associated with each food served by each of the multiple food serving points.

[0025] In one implementation of the second aspect, the method further includes receiving sensor data associated with the food products collected by the user and applying the model to classify the food products collected by the user based on the sensor data.

[0026] In one implementation of the second aspect, the method further includes receiving a weighing result associated with the food collected by the user and applying a model to classify the food collected by the user based on the sensor data and the weighing result.

[0027] According to a third aspect there is provided a computer program comprising instructions for causing an apparatus to carry out the method of the second aspect.

[0028] According to a fourth aspect, there is provided an apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured, in conjunction with the at least one processor, to cause the apparatus to at least acquire sensor data relating to food collected by a user from a plurality of food serving points, where the sensor data is associated with a food collection session identifier; acquire weighing results associated with the food collection session identifier, where each weighing result provides a weight of the food collected from the food serving point; acquire data relating to the food served at the plurality of food serving points; and use at least the sensor data, the weighing results, and the data relating to the food served at the plurality of food serving points as training data for a machine learning algorithm to build a model that enables subsequent classification of the food based at least on the sensor data associated with the food.

[0029] In one implementation of the fourth aspect, at least one memory and computer program code configured, together with at least one processor, to cause the device to at least acquire additional sensor data and manually labeled data associated with the additional sensor data, and complement the model using the acquired additional sensor data and the manually labeled data as training data for a machine learning algorithm.

[0030] In one implementation of the fourth aspect, at least one memory and computer program code configured, together with at least one processor, to cause the device to obtain at least waste weighing results, a food collection session identifier associated with the user's food collection session, and sensor data regarding food left by the user, and to build a model using at least the obtained waste weighing results, the food collection session identifier associated with the user's food collection session, and the sensor data regarding food left by the user as training data for a machine learning algorithm, the model enabling subsequent classification of food based at least on the sensor data associated with the food.

[0031] In one implementation of the fourth aspect, the data relating to foods served at the multiple food serving points includes at least one of foods served by each of the multiple food serving points and nutritional composition information associated with each food served by each of the multiple food serving points.

[0032] In one implementation of the fourth aspect, at least one memory and computer program code configured, together with at least one processor, to cause the device to at least receive sensor data associated with food collected by a user, and apply a model to classify the food collected by the user based on the sensor data.

[0033] In one implementation of the fourth aspect, at least one memory and computer program code configured, together with at least one processor, to cause the device to at least receive weighing results associated with food collected by a user, and apply a model to classify the food collected by the user based on the sensor data and the weighing results.

[0034] According to a fifth aspect, there is provided an apparatus comprising means for acquiring sensor data relating to food collected by a user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of a food collected from a food serving point; acquiring data relating to food served at the plurality of food serving points; and building a model using at least the sensor data, the weighing results and said data as training data for a machine learning algorithm to enable subsequent classification of foods based at least on the sensor data associated with the foods.

[0035] According to a sixth aspect, there is provided a method comprising receiving sensor data associated with food collected by a user; and applying a trained machine learning model to classify the food collected by the user based on the sensor data, the trained machine learning model being obtained by: acquiring sensor data regarding food collected by the user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of the food collected from the food serving point; acquiring data relating to food served at the plurality of food serving points; and building a model using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm, the model enabling subsequent classification of the food based at least on the sensor data associated with the food.

[0036] In one implementation of the sixth aspect, the method further includes receiving a weighing result associated with a food item collected by the user, and applying a model to classify the food item collected by the user based on the sensor data and the weighing result.

[0037] According to a seventh aspect, there is provided an apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured, in conjunction with the at least one processor, to cause the apparatus to at least receive sensor data associated with food collected by a user; and apply a trained machine learning model to classify the food collected by the user based on the sensor data, the trained machine learning model being obtained by: acquiring sensor data regarding food collected by the user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of the food collected from the food serving point; acquiring data relating to food served at the plurality of food serving points; and using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm to build a model that enables subsequent classification of the food based at least on the sensor data associated with the food.

[0038] In one implementation of the seventh aspect, at least one memory and computer program code configured, together with at least one processor, to cause the device to at least receive weighing results associated with food collected by a user, and apply the trained model to classify the food collected by the user based on the sensor data and the weighing results.

[0039] According to an eighth aspect there is provided a computer program comprising instructions for causing an apparatus to carry out the method of the sixth aspect.

[0040] According to a ninth aspect, an apparatus is provided, comprising means for receiving sensor data associated with food collected by a user; and applying a trained machine learning model to classify the food collected by the user based on the sensor data, the trained machine learning model being obtained by: acquiring sensor data regarding food collected by the user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of the food collected from the food serving point; acquiring data relating to food served at the plurality of food serving points; and building a model using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm, the model enabling subsequent classification of the food based at least on the sensor data associated with the food.

[0041] According to a tenth aspect, a system is provided comprising a detection point comprising at least one sensor configured to provide sensor data relating to food collected by a user, a control unit configured to control the detection point, and an analysis unit configured to apply a trained model to classify food collected by the user based at least on the sensor data to provide an estimation and / or classification of the food taken by the user.

[0042] In one implementation of the tenth aspect, the system may further comprise a weighing device configured to weigh an amount of food collected by the user, the weighing device being controlled by the control unit, and the analysis unit configured to apply the trained model to classify the food collected by the user based at least on the sensor data and the weighing results to provide an estimation and / or classification of the food taken by the user.

[0043] According to an eleventh aspect, a system is provided comprising a detection point having at least one sensor configured to provide sensor data regarding food left by a user, a control unit configured to control the detection point, and an analysis unit configured to apply a trained model to classify food left by the user based at least on the sensor data to provide an estimation and / or classification of the food left by the user.

[0044] In one implementation form of the eleventh aspect, the system may further comprise a weighing device configured to weigh an amount of food left by the user, the weighing device being controlled by the control unit, and the analysis unit configured to apply the trained model to classify the food left by the user based at least on the sensor data and the weighing results to provide an estimation and / or classification of the food left by the user.

[0045] Other features and advantages of the present invention will be apparent from a reading of the following detailed description and a review of the accompanying drawings.

[0046] The principles of the present invention will now be described with reference to the accompanying drawings. [Brief description of the drawings]

[0047] [Figure 1A] FIG. 1 illustrates a system according to an exemplary embodiment. [Figure 1B] FIG. 2 illustrates a system according to another exemplary embodiment. [Figure 1C] FIG. 2 illustrates a system according to another exemplary embodiment. [Figure 1D] FIG. 2 illustrates a system according to another exemplary embodiment. [Diagram 2] FIG. 4 is a signaling diagram of a method according to an exemplary embodiment. [Diagram 3] FIG. 4 is a signaling diagram of a method according to another exemplary embodiment. [Figure 4A] FIG. 1 illustrates a system according to an exemplary embodiment. [Figure 4B] FIG. 1 illustrates a system according to an exemplary embodiment. [Diagram 5] FIG. 1 illustrates an apparatus that may include various optional hardware and software components, according to one example embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0048] In the following description, reference is made to the accompanying drawings, which form a part of this disclosure, and in which are shown by way of example specific aspects, embodiments and examples in which the disclosure may be placed. It is to be understood that other aspects may be utilized and structural or logical changes may be made without departing from the scope of the disclosure. Therefore, the following detailed description should not be taken in a limiting sense, as the scope of the disclosure is defined by the appended claims. Moreover, the disclosure may be embodied in many other forms and should not be construed as limited to the specific structure or function disclosed in the following description.

[0049] According to the detailed description, it will be apparent to those skilled in the art that the scope of the present disclosure covers any embodiment of the present invention disclosed herein, regardless of whether the embodiment is implemented independently or in cooperation with other embodiments of the present disclosure. For example, the system disclosed herein can actually be implemented by using any number of the embodiments provided herein. Furthermore, it should be understood that any embodiment of the present disclosure can be implemented using one or more of the elements presented in the appended claims.

[0050] The term "food serving point" as used herein may refer to any physical location where food is served. Furthermore, one or more food dishes may be served at a single food serving point. In one exemplary embodiment, each dish may be served at a separate food serving point. For example, a food serving point may include a location where served dishes are placed and from which a user may take a user-selected amount of the dish onto his or her plate. Multiple food serving points may be located at a single location. A food serving point may have an associated "food collection point" where a user may place a tray and one or more dishes, e.g., plates carried by the tray. A food serving point or food collection point may also include a weighing device configured to weigh a tray located at the food serving point of the food collection point.

[0051] The term "tray" as used herein may refer to an object that may be used to carry one or more other items, e.g., plates, glasses, etc. The tray may take any suitable form, e.g., its shape may be rectangular, rounded rectangular, circular, etc. Each tray may be associated with an identifier that may uniquely identify a tray among all trays used at the food serving point. The identifier may be provided, for example, by a wirelessly readable tag, e.g., a near field communication (NFC) tag, a radio frequency identification (RFID) tag, or a visually readable tag. The identifier may also be provided, for example, by a visually readable code or identifier, e.g., a QR code, a bar code, etc. In one exemplary embodiment, the identifier may be a property of the tray itself. For example, a tag comprising the identifier may be fixedly attached to the tray, e.g., glued onto a bottom surface of the tray or incorporated into the tray. In another exemplary embodiment, the visually readable code or identifier may be, for example, printed on the tray or attached to the tray, e.g., as a sticker. In another exemplary embodiment, the identifier may be provided by a separate device or tag that may be associated with the user or with the user's device. For example, a mobile device placed on the tray may provide the identifier. Thus, the identifier may be provided, for example, via wireless transmission, for example, by using Bluetooth, RFID, NFC, Wi-Fi, LoRa, ZigBee, LTE / 4G / 5G / 6G, etc. Thus, in one exemplary embodiment, the identifier may be associated with the user or with the user's mobile or wearable device, instead of the tray. Thus, although the exemplary embodiment described below may use an identifier as being associated with the tray, the identifier may alternatively be associated with the user or with the user's mobile device, for example (e.g., a fingerprint or a biometric identifier (e.g., voice, face, heartbeat, etc.)).Additionally, the identifier may be unique among identifiers used within a particular food service system, or the identifier may be globally unique among all identifiers. The identifier may also be unique among all identifiers used by a particular device or device type. The identifier may be, for example, a serial number or code associated with a device or component, an electromagnetic substance capable of transmitting information, or any other identifier that may be used for identification purposes.

[0052] Furthermore, the term "reader" as used herein may refer to any type of reader capable of reading or recognizing an identifier. The reader may, for example, apply near field communication (NFC), radio frequency identification (RFID) or visual recognition. In another exemplary embodiment, the reader may be a fingerprint reader, a biometric reader, a face recognition based reader, etc. Visual recognition may, for example, refer to a solution in which the reader is capable of reading a visually readable code or identifier, for example, a QR code, a bar code, etc. Alternatively, visual recognition may refer to a solution based on recognizing information from one or more images from a camera. Furthermore, the reader may be configured to read the identifier continuously or at preset intervals. The reader may also be configured to read the identifier after receiving an instruction to do so.

[0053] Furthermore, the terms "trained model" and "machine learning training" as used herein may refer to a model that is trained or has already been trained earlier and is only then applied. The training or learning may be performed locally at the location with the food serving point. Alternatively or additionally, the training or learning may be performed externally, e.g., by edge computing, or centrally, e.g., as a cloud-based solution. Similarly, when applying an earlier trained model, the model may be at the site with the food serving point or externally or centrally, e.g., in the cloud. In one exemplary embodiment, the training or learning may be implemented automatically and / or independently at the location, e.g., by a neuromorphic chip or an artificial chip. In another exemplary embodiment, the training or learning may start from scratch. Alternatively, the training or learning may start from an existing algorithm and an existing data set.

[0054] FIG. 1A illustrates a system according to an example embodiment.

[0055] The system comprises at least one food serving point 102, from which a user can take a desired amount of food. The food serving point 102 may comprise a control unit 104, a weighing device 108 connected to the control unit 104 and configured to weigh a tray of the user, and a reader 106 connected to the control unit 104 and configured to read an identifier associated with the tray. The food serving point 102 may also comprise a display 110 configured to display information associated with the food serving point 102. The control unit 104 associated with the food serving point 102 may be configured to initiate a weighing event by the weighing device 108 associated with the food serving point 102 when it detects a change in weight. In response to the initiation, the control unit 104 may be configured to associate an identifier associated with the tray read by the reader 106 with the weighing event. Further, the control unit 104 may be configured to generate at least one weighing result by the weighing device 108 and to stop the weighing event by the weighing device 108 when it does not detect a weight. In addition to the food serving point 102, the system may comprise one or more additional food serving points not associated with a weighing device, such as, for example, a grill point, a dessert point, a beverage point, a soup point, a salad point, a bread point, etc. The weighing device 108 may be configured to measure the decrease in weight of a container from which a user has collected food. Alternatively or additionally, the weighing device 108 may be configured to measure the increase in weight of a tray carrying a plate or bowl on which a user has placed the collected food. In other words, the weighing measurement may be implemented by the tray (by means of a built-in weighing device) at the location where the user has placed his / her tray near or at the food serving point (measuring the decrease in weight of the food collected by the user).

[0056] In one exemplary embodiment, the reader 106 may be configured to read the identifier, for example, continuously or at preset intervals. In another exemplary embodiment, the reader 106 may be configured to read the identifier after receiving an instruction to read the identifier. Furthermore, when the control unit 104 associates the identifier read by the reader 106 with a metering event, the control unit 104 may use the last read identifier as the identifier to be associated with the metering event. Alternatively, when the control unit 104 initiates a metering event, the control unit 104 may be configured to instruct the reader 106 to read the identifier and to associate the read identifier with the metering event. Furthermore, in one exemplary embodiment, if the control unit 104 receives a read identifier from the reader 106 after the metering event has already initiated, the control unit 104 may set this identifier as the identifier associated with the metering event.

[0057] The food serving point 102 may also include a detection point 122 associated with the food serving point 102 or a separate detection point 122. The detection point 122 may include at least one sensor configured to provide sensor data regarding the food collected by the user. The at least one sensor may include at least one of a camera, a stereo camera, a depth camera, a multispectral camera, a hyperspectral camera, an infrared camera, an RGB camera, an ultraviolet camera, a spectroscopic sensor, a near-infrared sensor, a spectroscopic sensor, a photogrammetry sensor, a lidar sensor, a 3D scanner, and a photodetector. Different sensors may provide different levels of accuracy depending on the type of food, and thus, two or more sensors may be used to provide more accurate sensor data regarding the food collected by the user. One or more of the sensors may be configured to detect at wavelengths of 200-1700 nm, 280-1550 nm, 315-1380 nm, 380-1000 nm, 390-900 nm, or 400-800 nm. The sensing points 122 may be positioned such that at least one sensor is capable of providing sensor data, e.g., image data, regarding the food collected by the user from the food serving point 102. One possible location for the at least one sensor may be on a counter where a tray is placed when the user collects the food from the food serving point 102. Thus, the at least one sensor is capable of storing the sensor data regarding the food collected by the user. The provided data provided by the at least one sensor, associated with the identifier, may be sent to the data system 100. Alternatively or additionally, the data may be sent directly to the training system 124.

[0058] In one exemplary embodiment, the system may also include at least one identification point 112 comprising a first reader 116 configured to read an identifier associated with the tray. In one exemplary embodiment, the identification point 112 may also comprise a second reader 118 configured to read a user identifier. The identification point 112 may also comprise a control unit 114 configured to control the operation of the identification point 112. The identification point 112 may also comprise a display 120 configured to provide information, for example, regarding the food collected by the user. In one exemplary embodiment, the second reader 118 may comprise a QR code reader or a short-range wireless communication reader. In another exemplary embodiment, the second reader 118 may also be located at one food serving point 102.

[0059] The system may comprise a data system 100 and a training system 124. The data system 100 may be configured to store data related to the food served at each of the plurality of food serving points 102. The data may include, for example, the food served by each of the plurality of food serving points 102. Furthermore, the data related to the food served at the plurality of food serving points may include detailed information about the food, for example, ingredient data, nutrition content data (e.g., relating to fat, carbohydrates, protein, etc.), energy content data, measurement data, etc. In other words, the data system 100 may be aware of the recipe content of the food served at the food serving points 102.

[0060] The data system 100 may also be configured to obtain weighing results and an identifier, e.g., a tray identifier, from each food serving point 102 and associate weighing results having the same identifier with each other. The data system 100 may be configured to initiate a new session when it first detects an identifier obtained by the reader 106. The data system 100 may be configured to create a session with a session identifier when it first obtains an identifier (e.g., a tray identifier) ​​and determines that there is no active session associated with that identifier. The data system 100 may be configured to associate a session start time with the session and link the identifier with the session with the session identifier. After initiation, subsequent measurements at the food serving point 102 may be linked to the session based on the obtained identifier. In one exemplary embodiment, the data system 100 may also be configured to receive sensor data from at least one sensor and associate the sensor data with the corresponding weighing results in the session. Alternatively, the sensor data associated with the identifier acquired by the reader 106 may be transmitted directly to the training system 124. The session may also include a user identifier. Since a single user identification is sufficient, it may be sufficient to read the user identifier once at a location during a session.

[0061] The training system 124 may be configured to acquire sensor data originating from at least one detection point 122, where the sensor data is associated with an identifier, acquire from the management system weighing results associated with the identifier and data related to food served at the multiple food serving points from the food serving points, and use at least the acquired sensor data associated with the identifier, the weighing results associated with the identifier, and the data related to food served at the multiple food serving points as training data for the machine learning algorithm to build a model that allows subsequent classification of the food based at least on the sensor data associated with the food. Any applicable machine learning algorithm for building the model may be used. For example, if the detection points 122 are located at each food serving point, sensor data may be acquired from each food serving point separately. This leads to an incremental sensor data series where an increase in weight at a food serving point corresponds to an added element in the sensor data (i.e., food taken from the food serving point). For example, the data system 100 may know that a certain food serving point serves rice. The recipe in this case is quite simple, and the data system 100 is aware of the nutritional composition of rice. The weight of the rice taken by the user is obtained from a weighing device. At the detection point, sensor data (e.g., image data) is captured, for example, including the contents of the tray, and based on the sensor data, it is known that the most recently added item (i.e., rice) has a certain weight.

[0062] When training data is repeatedly obtained and used as training data for the machine learning model, the model evolves and becomes more accurate. This then enables a solution where weighing and sensing events at the food serving point are no longer required, since the trained model is capable of providing a subsequent classification of the food based at least on the sensor data associated with the food.

[0063] The solution shown in FIG. 1A includes detailed information about the food served at multiple food serving points. In other words, before the user collects food from the food serving points, the system is aware of the food that may be collected by the user. Then, when the user starts to collect food from the food collection points, the amount of food collected by the user at each food serving point is weighed. At the same time, the detection point provides sensor data about the collected food by at least one sensor. Based on the sensor data, it may be possible to calculate the volume of the food. For example, it may also be possible to separate fat, protein, carbohydrates and moisture (for example, vegetables, meat, etc. in the collected food may be detected by a spectral camera). When all this information is combined and used as training data for a machine learning algorithm, a model may be obtained that allows a subsequent classification of the food based at least on the sensor data associated with the food. In other words, when the model is trained, less information (for example, only sensor data obtained from one or more sensors) is sufficient to provide an estimate of the content of the food collected by the user. Furthermore, the presented solution may also enable calculation of the user's intake (weight / mass) and dietary content.

[0064] FIG. 1B illustrates a system according to an exemplary embodiment. The embodiment illustrated in FIG. 1B is similar to the embodiment illustrated in FIG. 1A, except that a detection point 126 is now at the identification point 112. In other words, a separate detection point is not placed at each food serving point 102. Instead, a single detection point 126 is applied at the identification point 112, for example, at a caching point. The sensor data associated with the identifier acquired by the reader 116 may be transmitted to the data system 100. Alternatively or additionally, the sensor data associated with the identifier acquired by the reader 116 may be transmitted directly to the training system 124.

[0065] 1C illustrates a system according to an exemplary embodiment. The embodiment illustrated in FIG. 1C is similar to the embodiment illustrated in FIG. 1A, except that the system further includes at least one waste collection point 138.

[0066] At least one waste collection point 138 may comprise a control unit 128 connected to a reader 130 configured to read an identifier associated with the tray when the tray is returned. The waste collection point 138 may further comprise at least one weighing device 132 connected to the control unit 128 and configured to weigh an amount of bio-waste left at the waste collection point 138 by a user. The management unit 100 may be configured to receive from the control unit 128 the identifier associated with the tray and the weighed weight of the bio-waste. The waste collection point 138 may also comprise a display 134 connected to the control unit 128. The display 134 may, for example, display the weight of the bio-waste left by the user. At least one waste collection point 138 may further comprise a detection point 136 comprising at least one sensor configured to provide sensor data regarding food left by the user. The at least one sensor may comprise at least one of a camera, a stereo camera, a depth camera, a multispectral camera, an infrared camera, an RGB camera, a spectroscopic sensor, a near infrared sensor, a spectroscopic sensor, a photogrammetry sensor, a lidar sensor, a 3D scanner, and a photodetector. One or more of the sensors may be configured to detect at wavelengths of 200-1700 nm, 280-1550 nm, 315-1380 nm, 380-1000 nm, 390-900 nm, or 400-800 nm. The detection point 136 may be positioned such that the at least one sensor is capable of providing sensor data, e.g., image data, regarding food left by a user. The waste collection point 138 may be connected to the data system 100. Additionally, the sensor data associated with the identifier acquired by the reader 130 may be directly transmitted to the training system 124.

[0067] In one exemplary embodiment, the training system 124 may be configured to obtain waste weighing results, a food collection session identifier associated with the user's food collection session, and sensor data regarding food left by the user, and provide additional information regarding food left by the user based on the obtained waste weighing results, a food collection session identifier associated with the user's food collection session, and sensor data regarding food left by the user. The additional information may include, for example, food items left by the user and / or the amount of different food items in the waste left by the user. In one exemplary embodiment, the training system 124 may use a previously constructed model to provide the additional information. In another exemplary embodiment, a separate model may be constructed for food waste based on food left by the user at the waste collection point 138. Because the data system 100 is aware of the food served at the food serving point 102 and the amount of food taken by the user, the training system 124 may also use this information when providing the additional information regarding food left by the user. For example, when a user proceeds to waste collection point 138 and the identifier is read, it is already known that the user has collected, for example, 150g of rice and 150g of chicken sauce from food serving point 102. This then means that for this particular user, the food left by the user includes rice and chicken sauce, thus significantly limiting the alternatives, what the user may leave as waste food at waste collection point 138, and thus facilitating sorting of the waste food.

[0068] In one exemplary embodiment, the data system 100 may be configured to terminate an existing session associated with an identifier when it fails to register the identifier 144 at the waste collection point 138 within a predetermined time period, e.g., 45 minutes. Thus, the term "session" may define a time period during which the same user uses the same identifier, e.g., by controlling the tray from a first food serving point, until returning the tray at a waste collection point or a separate tray return point. If there is a separate tray return point, it may include a reader configured to read the identifier when the tray is returned. This allows for termination of the session even if the tray return was not registered in the normal manner, e.g., the user left the tray on the table.

[0069] The solution shown in FIG. 1C includes detailed information about the food served at multiple food serving points. In other words, before the user collects food from the food serving points, the system is aware of the food that may be collected by the user. Then, when the user starts to collect food from the food collection points, the amount of food collected by the user at each food serving point is weighed. At the same time, the detection point provides sensor data about the collected food by at least one sensor. Based on the sensor data, it may be possible to separate, for example, fat, protein, carbohydrates and moisture (e.g., vegetables, meat, etc. in the collected food). This knowledge may also be utilized at the waste collection point 138. By using the data collected at the time of food collection, the sensor data provided by the detection point 136, and one or more weighing results from the weighing device 132, it is possible to determine, for example, the nutritional content consumed by the user.

[0070] 1D illustrates a system according to one exemplary embodiment. The embodiment illustrated in FIG. 1D is similar to the embodiment illustrated in FIG. 1C, except that the detection point 126 is now at the discrimination point 112.

[0071] In an example embodiment of any of FIGS. 1A-1D , the system may include an administrative interface connected to a training system 124. The training system 124 may be configured to obtain additional sensor data and manually labeled data associated with the additional sensor data from the administrative interface, and to complement the model using the obtained additional image data and manually labeled data as training data for the machine learning algorithm. In an example embodiment, the training system 124 may also be configured to obtain metric data associated with a new food item or dish. This may be useful, for example, if a new food item or dish is introduced in the system and the training system was not provided with sensor data (and possibly metric data) regarding this item or dish earlier. Without the earlier data, the model trained by the training system is not capable of classifying the new food item or dish. However, via the administrative interface, the new food item or dish may be learned and included in the trained model, and after a sufficient amount of training, the model is capable of classifying the new food item or dish when it is introduced at the food serving point 122.

[0072] 2 shows a signaling diagram of a method according to an exemplary embodiment. The method may be, for example, a computer-implemented method performed by an apparatus included in the training system 124.

[0073] At 200, sensor data about food collected by a user from a plurality of food serving points is obtained, and the sensor data is associated with a food collection session identifier. The data may be obtained from data system 100 or directly from the plurality of food serving points.

[0074] At 202, weighing results associated with the food collection session identifier are obtained, with each weighing result providing a weight of food collected from a food serving point. The data may be obtained from data system 100 or directly from multiple food serving points.

[0075] At 204, data related to food served at a plurality of food serving points is obtained. The data may be obtained from data system 100.

[0076] At 206, at least the sensor data, the metric results, and the above data are used as training data for a machine learning algorithm to build a model that allows for subsequent classification of the food based at least on the sensor data associated with the food. In an exemplary embodiment, the data used as training data may include sensor data or image data obtained from other sources, e.g., the Internet or image banks, representing a similar collection of food. The data related to the food served at the multiple food serving points may include detailed information about the food, e.g., ingredient data, nutritional content data (e.g., relating to fat, carbohydrates, protein, etc.), energy content data, metric data, etc. One or more of the sensors may be configured to detect at wavelengths of 200-1700 nm, 280-1550 nm, 315-1380 nm, 380-1000 nN, 390-900 nm, or 400-800 nm.

[0077] In one exemplary embodiment, sensor data associated with food collected by a user may be received, and a model may be applied to classify the food collected by the user based on the sensor data. In a further exemplary embodiment, weighing results associated with food collected by a user may be received, and a model may be applied to classify the food collected by the user based on the sensor data and the weighing results. In other words, when the model is built, it is possible to classify the food collected by the user based only on a limited amount of data (sensor data, or sensor data and weighing results). The sensor data and weighing results may be collected at a single point, for example, at a cashier, when the user collects the food. Thus, a food collection specific sensing point and weighing device is no longer required to obtain data regarding the food collected by the user.

[0078] 3 shows a signaling diagram of a method according to another exemplary embodiment. The method may be, for example, a computer-implemented method performed by an apparatus including the machine learning model to be trained.

[0079] At 300, sensor data associated with a food item collected by a user is received.

[0080] At 302, a trained machine learning model is applied to classify food collected by the user based on the sensor data obtained by obtaining sensor data related to food collected by the user from a plurality of food serving points, the sensor data being associated with a food collection session identifier, obtaining weighing results associated with the food collection session identifier, each weighing result providing a weight of the food collected from the food serving point, obtaining data related to food served at the plurality of food serving points, and constructing a model using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm to enable subsequent classification of the food based at least on the sensor data associated with the food. In one exemplary embodiment, the method may further include receiving weighing results associated with the food collected by the user and applying the trained model to classify the food collected by the user based on the sensor data and the weighing results.

[0081] FIG. 4A illustrates a system according to an example embodiment.

[0082] The system 400 may comprise a reader 406 configured to read the identifier. The identifier may be associated, for example, with a user or with a mobile device of the user. The system 400 may also comprise a control unit 402 configured to control the operation of the identification point 400. The system 400 may also comprise a display 410 configured to provide information, for example, regarding the food collected by the user. The reader 406 may be, for example, a QR code reader, a short-range wireless communication reader, or a camera. The system 400 may further comprise a weighing device 408 configured to weigh the amount of food collected by the user. The control unit 402 may use some approximate or predefined information regarding the weight of other items on the tray, for example, plates, glasses, cutlery, etc., to determine the actual weight of the food.

[0083] The system 400 may further include a detection point 404. The detection point 404 may include at least one sensor configured to provide sensor data about the food collected by the user. The at least one sensor may include at least one of a camera, a stereo camera, a depth camera, a multispectral camera, an infrared camera, an RGB camera, a spectroscopic sensor, a near infrared sensor, a spectroscopic sensor, a photogrammetry sensor, a lidar sensor, a 3D scanner, and a photodetector. One or more of the sensors may be configured to detect at wavelengths of 200-1700 nm, 280-1550 nm, 315-1380 nm, 380-1000 nm, 390-900 nm, or 400-800 nm. The detection point 404 may be positioned such that the at least one sensor is capable of providing sensor data, e.g., image data, about the food collected by the user. One possible location for the at least one sensor may be on a counter where the tray is placed when the user is about to pay for the food. Thus, the at least one sensor may store sensor data about the food collected by the user. The provided data provided by the at least one sensor, associated with the identifier, may be provided to the analysis unit 412.

[0084] The system 400 may further comprise an analysis unit 412 and a trained machine learning model 414. The trained machine learning model 414 may have been trained earlier by the systems and functionality shown in more detail in any of FIGS. 1A-1D and the related description. The analysis unit 412 may be configured to apply the trained model 414 to classify food collected by a user based on the sensor data or based on the sensor data and the weighing results provided by the weighing device 408. In an exemplary embodiment, the analysis unit 412 may have access to detailed information regarding the food served at the location comprising the system 400, such as the food served at the location, the nutritional content of the food supply, etc. In an exemplary embodiment, the functionality related to the control unit 402, the analysis unit 412 and the trained model 414 may be provided by a single entity comprising a memory for storing the trained model.

[0085] Based on the sensor data, the metric results and the trained model, the analysis unit 412 may be able to provide an estimation or classification of the food taken by the user. For example, the estimation or classification result may be that the user took 150g of rice and 150g of chicken sauce. If the user is identified by the system 400, the estimation or classification result may be associated with the user.

[0086] FIG. 4B illustrates a system 416 according to one example embodiment.

[0087] The system 416 may be implemented with a waste collection point 418. The system 416 may comprise a control unit 424 configured to control an operation of the waste collection point 418. The system 416 may further comprise a weighing device 428 configured to weigh an amount of food left by the user. The system 416 may further comprise a detection point 426. The detection point 426 may comprise at least one sensor configured to provide sensor data regarding the food left by the user. The at least one sensor may comprise at least one of a camera, a stereo camera, a depth camera, a multispectral camera, an infrared camera, an RGB camera, a spectroscopic sensor, a near-infrared sensor, a spectroscopic sensor, a photogrammetry sensor, a lidar sensor, a 3D scanner, and a light detector. One or more of the sensors may be configured to detect at wavelengths of 200-1700 nm, 280-1550 nm, 315-1380 nm, 380-1000 nm, 390-900 nm, or 400-800 nm. The detection points 426 may be positioned such that at least one sensor is capable of providing sensor data, e.g., image data, regarding food left by a user. One possible location for the at least one sensor may be on a counter where a tray is placed when a user is about to enter the waste collection point. Thus, the at least one sensor is capable of storing sensor data regarding food left by a user. The sensor data provided by the at least one sensor may be provided to the analysis unit 420.

[0088] The system 416 may further comprise an analysis unit 420 and a trained machine learning model 422. The trained machine learning model 422 may have been trained earlier by the system and functionality shown in more detail in any of FIG. 1A-1D and the related description. In another exemplary embodiment, the system 416 may use a general machine learning model as the trained model 422. The analysis unit 420 may be configured to apply the trained model 422 to classify the food collected by the user based on the sensor data or based on the sensor data and the weighing results provided by the weighing device 428. In one exemplary embodiment, the analysis unit 420 may have access to detailed information about the food served at the location comprising the system 416, such as the food served at the location, the nutritional content of the food supply, etc. In another exemplary embodiment, the analysis unit 420 does not have pre-defined information available. In one exemplary embodiment, the functionality related to the control unit 424, the analysis unit 420 and the trained model 422 may be provided by a single entity comprising a memory for storing the trained model.

[0089] Based on the sensor data and the trained model 422, and possibly also based on the weighing results from the weighing device 428, the analysis unit 420 may be able to provide an estimation or classification of the food left by the user. The analysis unit 420 may be able to separate the food content of the food left by the user, for example, meat, sauce, salad, bread, as well as the nutritional content associated with the food left by the user.

[0090] One or more of the above described examples and embodiments may enable a solution for estimating food content based on a limited amount of sensor data, e.g., image data, by using a trained model. Furthermore, one or more of the above described examples and embodiments may enable a solution for providing detailed information on the overall food consumption and the amount and type of food waste by a user. Furthermore, one or more of the above described examples and embodiments may enable a solution for providing a user-based historical food content analysis when the user is identified during the food collection process. Furthermore, one or more of the above described examples and embodiments may enable a solution in which an end user can utilize the data provided by the illustrated system, for example, for personal purposes, e.g., for monitoring health and / or diet. Furthermore, one or more of the above described examples and embodiments may enable a solution for planning applicable meals for a person. This may also take into account, for example, existing medical issues (e.g., medications, diseases, etc.) of the person.

[0091] 5 illustrates an apparatus 500 that may include various optional hardware and software components. The apparatus 500 may include one or more controllers or processors 502 (e.g., signal processors, microprocessors, ASICs, or other control and processing logic) for performing tasks such as signal coding, data processing, input / output processing, power control, and / or other functions, and a network interface 508 to enable wireless and / or wired data communication.

[0092] The device 500 may also include one or more memories 504. The memory 504 may include non-removable and / or removable memory. The non-removable memory may include RAM, ROM, flash memory, a hard disk, or other well-known memory storage technologies. The removable memory may include flash memory or other well-known memory storage technologies. The memory 504 may be used to store data and / or code for running an operating system 506 and / or one or more applications.

[0093] The device 500 may be configured to partially or fully implement various features, examples and embodiments, for example, as illustrated in Figures 1A-1D, 2, 3 and 4A-4B. The functionality described herein may be implemented, at least in part, by one or more computer program product components, such as software components. A system or device may comprise a single device or multiple devices, which may provide cloud-based services that are accessible via a data communications network, for example, the Internet.

[0094] According to an exemplary embodiment, the processor 502 may be configured with program code that, when executed, performs examples and embodiments of the operations and functionality described. Alternatively, or in addition, the functionality described herein may be implemented, at least in part, by one or more hardware logic components. For example, but not limited to, exemplary types of hardware logic components that may be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), and Graphics Processing Units (GPUs). The system or device may further include components and elements not disclosed in FIG. 5, such as an input / output interface, a receiver, a transmitter, a transceiver, an input / output port, a display, and the like.

[0095] Any combination of the illustrated components disclosed in FIG. 5, for example, at least one of the processor 502 and the memory 504, may form a means for performing any of the illustrated functionality herein.

[0096] Those skilled in the art will appreciate that each step or operation, or any combination of steps or operations, described above, may be implemented by various means, such as hardware, firmware, and / or software. As an example, one or more of the steps or operations described above may be implemented by computer or processor executable instructions, data structures, program modules, and other suitable data representations. Furthermore, computer executable instructions for implementing the steps or operations described above may be stored on a corresponding data carrier and executed by at least one processor, such as a processor included in a device. The data carrier may be implemented as any computer readable storage medium configured to be readable by the at least one processor to execute the computer executable instructions. Such computer readable storage media may include both volatile and non-volatile media, both removable and non-removable media. By way of example and not limitation, computer readable media comprises media implemented in any method or technology suitable for storing information. More specifically, practical examples of computer readable media include, but are not limited to, information delivery media, RAM, ROM, EEPROM, flash memory or other memory technologies (e.g., solid state drives (SSD) or NVM Express (NVMe)), CD-ROM, digital versatile discs (DVDs), holographic media or other optical disc storage, magnetic tape, magnetic cassettes, magnetic disc storage, and other magnetic storage devices.

[0097] Although exemplary embodiments of the present invention are disclosed herein, it should be noted that any various changes and modifications may be made in the embodiments of the present invention without departing from the scope of legal protection defined by the appended claims. In the appended claims, the reference of an element in the singular does not exclude the presence of a plurality of such elements, unless expressly stated otherwise.

Claims

1. a plurality of food serving points (102) configured to serve food, each food serving point configured to serve a predetermined dish, each food serving point (102) associated with a weighing device (108) configured to weigh an amount of said food collected from said food serving point (102) to provide a weighing result, and a reader (106) configured to read an identifier associated with a user's food collection session; at least one detection point (122), each detection point (122) comprising at least one sensor configured to provide sensor data regarding the food collected by the user and a reader (108, 116) configured to read the identifier associated with the food collection session of the user; storing data relating to food served at each of said plurality of food serving points (102); obtaining said weighing results and said identifiers from each food serving point (102); Associating the weighing results having the same identifier with each other; A data system (100) configured to: acquiring sensor data originating from the at least one sensing point (122), the sensor data being associated with the identifier; obtaining, from said data system (100), said weighing results associated with said identifiers and said data relating to food served at said plurality of food serving points (120); using the acquired sensor data associated with the identifier, the metric results associated with the identifier, and the data related to food served at the plurality of food serving points as training data for a machine learning algorithm to build a model that enables subsequent classification of food products based at least on the sensor data associated with the food products; A training system (124) configured to perform A system comprising:

2. Receiving a trigger event; triggering storage of sensor data of the n food items collected by the user by the at least one sensor; The system of claim 1 , further comprising a control unit (104) configured to:

3. The system of claim 2 , wherein the control unit (104) is configured to receive the trigger event from the reader (106).

4. The food serving point further comprises a control unit (104) and a detection point (122) at each of the plurality of food serving points (102), the control unit (104) of the food serving point (102) being Receiving a trigger event; triggering storage of sensor data associated with the food product collected by the user by the at least one sensor of the detection point (122); The system of claim 1 configured to:

5. The system of claim 4, wherein the control unit (104) is configured to receive the trigger event from the reader (106) associated with the food serving point (102).

6. The system of claim 4, wherein the control unit (104) is configured to receive the trigger event from the metering device (108) associated with the food serving point (102).

7. the data relating to food served at the plurality of food serving points, the food served by each of the plurality of food serving points (102); Nutritional content information associated with each food item served by each of said plurality of food serving points (102); The system according to any one of claims 1 to 6, comprising at least one of:

8. The data system (100) comprises: creating a session having a session identifier when first obtaining said identifier and determining that no active sessions exist; associating a session start time with said session; linking said identifier with said session having said session identifier; A system according to any one of claims 1 to 7, configured to:

9. and a user identifier reader (118), said data system (100) further comprising: obtaining a user identifier from said user identifier reader (118); Associating the user identifier with the session; The system of claim 8 configured to:

10. 10. The system of claim 1, wherein the identifier comprises a radio frequency identifier, a near field communication identifier, a bar code, a QR code, or a visually recognizable identifier associated with a tray used by the user.

11. 9. The system of claim 1, wherein the identifier includes an identifier associated with the user, a radio frequency identifier, a near field communication identifier, a smart wearable identifier, a smart ring identifier, a fingerprint, a biometric identifier, and a visually recognizable identifier.

12. a weighing device (132) configured to weigh the amount of bio-waste left by the user to provide a waste metering result; a reader (130) configured to read a food collection session identifier associated with the food collection session of the user; at least one sensor configured to provide sensor data regarding the food item left by the user; a waste collection point (138) comprising: The training system (124), obtaining the waste weighing results, the food collection session identifier associated with the food collection session of the user, and the sensor data regarding the food left by the user; providing additional information about the food left by the user based on the obtained waste weighing results, the food collection session identifier associated with the food collection session of the user, and the sensor data about the food left by the user; 12. A system according to any preceding claim, configured to:

13. 13. The system of claim 1, wherein the at least one sensor comprises at least one of a camera, a stereo camera, a depth camera, a multispectral camera, a hyperspectral camera, an infrared camera, an RGB camera, an ultraviolet camera, a spectroscopic sensor, a near-infrared sensor, a spectroscopic sensor, a photogrammetry sensor, a lidar sensor, a 3D scanner, and a photodetector.

14. The training system (124), acquiring additional sensor data and manually labeled data associated with the additional sensor data; Complementing the model using the acquired additional sensor data and manually labeled data as training data for the machine learning algorithm.

14. A system according to any preceding claim, configured to:

15. acquiring sensor data relating to food collected by a user from a plurality of food serving points (102), the sensor data being associated with a food collection session identifier; obtaining weighing results associated with the food collection session identifier, each weighing result providing a weight of food collected from a food serving point (102); obtaining data relating to food served at said plurality of food serving points (102); using at least the sensor data, the weighing results, and the data related to food served at the plurality of food serving points (102) as training data for a machine learning algorithm to build a model that enables subsequent classification of food products based at least on the sensor data associated with the food products; 4. A computer-implemented method comprising:

16. acquiring additional sensor data and manually labeled data associated with the additional sensor data; Complementing the model using the acquired additional sensor data and manually labeled data as training data for the machine learning algorithm. The computer-implemented method of claim 15 further comprising:

17. obtaining waste weighing results, the food collection session identifier associated with the user's food collection session, and sensor data regarding the food left by the user; providing additional information about the food left by the user based on the obtained waste weighing results, the food collection session identifier associated with the food collection session of the user, and the sensor data about the food left by the user; 17. The computer-implemented method of claim 15 or 16, further comprising:

18. the data relating to food served at the plurality of food serving points, the food served by each of the plurality of food serving points (102); Nutritional content information associated with each food item served by each of said plurality of food serving points (102); 18. The computer-implemented method of claim 15, further comprising at least one of:

19. receiving sensor data associated with a food item collected by a user; applying the model to classify the food items collected by the user based on the sensor data; 19. The computer-implemented method of claim 15, further comprising:

20. receiving a weighing result associated with the food product collected by the user; applying the model to classify the food items collected by the user based on the sensor data and the weighing results; 20. The computer-implemented method of claim 19, further comprising:

21. A computer program comprising instructions for causing an apparatus (500) to carry out the method according to any of claims 15 to 20.

22. At least one processor (500); At least one memory (504) containing computer program code; An apparatus (500) comprising: The at least one memory (504) and the computer program code, together with the at least one processor (502), provide the device (500) with at least: acquiring sensor data relating to food collected by a user from a plurality of food serving points (102), the sensor data being associated with a food collection session identifier; obtaining weighing results associated with the food collection session identifier, each weighing result providing a weight of food collected from a food serving point (102); obtaining data relating to food served at said plurality of food serving points; using at least the sensor data, the weighing results, and the data related to food served at the plurality of food serving points (102) as training data for a machine learning algorithm to build a model that enables subsequent classification of food products based at least on the sensor data associated with the food products; An apparatus (500) configured to cause

23. The at least one memory (504) and the computer program code, together with the at least one processor (502), provide the device (500) with at least: acquiring additional sensor data and manually labeled data associated with the additional sensor data; Complementing the model using the acquired additional sensor data and manually labeled data as training data for the machine learning algorithm.

23. The apparatus (500) of claim 22 configured to:

24. The at least one memory (504) and the computer program code, together with the at least one processor (502), provide the device (500) with at least: obtaining waste weighing results, the food collection session identifier associated with the user's food collection session, and sensor data regarding the food left by the user; using the captured waste weighing results, the food collection session identifier associated with the food collection session of the user, and the sensor data regarding the food left by the user as training data for the machine learning algorithm to build a model that enables subsequent classification of food products based at least on the sensor data associated with the food products; 24. The apparatus (500) of claim 22 or 23, configured to:

25. the data relating to food served at the plurality of food serving points, the food served by each of the plurality of food serving points (102); Nutritional content information associated with each food item served by each of said plurality of food serving points (102); 25. The apparatus (500) of any of claims 22 to 24, comprising at least one of:

26. The at least one memory (504) and the computer program code, together with the at least one processor (502), provide the device (500) with at least: receiving sensor data associated with a food item collected by a user; applying the model to classify the food items collected by the user based on the sensor data; 26. An apparatus (500) according to any of claims 22 to 25, configured to:

27. The at least one memory (504) and the computer program code, together with the at least one processor (502), provide the device (500) with at least: receiving a weighing result associated with the food product collected by the user; applying the model to classify the food items collected by the user based on the sensor data and the weighing results; 27. The apparatus (500) of any of claims 22 to 26, configured to:

28. receiving sensor data associated with a food item collected by a user; applying a trained machine learning model to classify the food collected by the user based on the sensor data, the trained machine learning model being obtained by: acquiring sensor data regarding food collected by the user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of a food collected from a food serving point; acquiring data relating to food served at the plurality of food serving points; and using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm to build a model that enables subsequent classification of food based at least on the sensor data associated with the food; 4. A computer-implemented method comprising:

29. receiving a weighing result associated with the food product collected by the user; applying the trained model to classify the food items collected by the user based on the sensor data and the weighing results; 30. The computer-implemented method of claim 28, further comprising:

30. At least one processor (502); At least one memory (504) containing computer program code; The at least one memory (502) and the computer program code, together with the at least one processor (502), provide the apparatus (500) with at least: receiving sensor data associated with a food item collected by a user; applying a trained machine learning model to classify the food collected by the user based on the sensor data, the trained machine learning model being obtained by: acquiring sensor data regarding food collected by the user from a plurality of food serving points, the sensor data being associated with a food collection session identifier; acquiring weighing results associated with the food collection session identifier, each weighing result providing a weight of a food collected from a food serving point; acquiring data relating to food served at the plurality of food serving points; and using at least the sensor data, the weighing results, and the data as training data for a machine learning algorithm to build a model that enables subsequent classification of food based at least on the sensor data associated with the food; An apparatus (500) configured to cause

31. The at least one memory (504) and the computer program code, together with the at least one processor (502), provide the device (500) with at least: receiving a weighing result associated with the food product collected by the user; applying the trained model to classify the food items collected by the user based on the sensor data and the weighing results; 31. The apparatus (500) of claim 30 configured to:

32. A computer program comprising instructions for causing an apparatus (500) to perform the method according to claim 28 or 29.

33. a detection point (404) comprising at least one sensor configured to provide sensor data relating to the food product collected by a user; a control unit (402) configured to control said detection points (404); an analysis unit (412) configured to apply a trained model (414) to classify the food collected by the user based at least on the sensor data to provide an estimation and / or classification of the food taken by the user; A system (400) comprising:

34. a weighing device (408) configured to weigh an amount of the food collected by the user, the weighing device (408) being controlled by the control unit (402); Further equipped with 34. The system (400) of claim 33, wherein the analysis unit (412) is configured to apply the trained model (414) to classify the food collected by the user based at least on the sensor data and metric results to provide an estimation and / or classification of the food taken by the user.

35. a detection point (426) comprising at least one sensor configured to provide sensor data relating to food left by a user; a control unit (424) configured to control said detection points (426); an analysis unit (420) configured to apply a trained model (422) to classify the food left by the user based at least on the sensor data to provide an estimation and / or classification of the food left by the user; The system (416).

36. a weighing device (428) configured to weigh an amount of the food left by the user, the weighing device (428) being controlled by the control unit (424); Further equipped with 36. The system (416) of claim 35, wherein the analysis unit (420) is configured to apply the trained model (422) to classify the food left by the user based at least on the sensor data and metric results to provide an estimation and / or classification of the food left by the user.