A few-sample dish classification method

Abstract

The application discloses a few-sample dish classification method, which is based on an improved Vision Transformer model, namely a Food-VIT model, for few-sample dish classification. The method comprises the following steps: inputting a dish image to be classified into a network, and then giving a classification result by a classification network Food-VIT. The method uses a Transformer-based dish image classification, only needs a small amount of data set for training, reduces the collection and labeling cost of dishes, reduces the labor cost, and shortens the training period.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to a method for classifying dishes with a small number of samples. Background Technology

[0002] Currently, automated billing systems in restaurants and canteens mainly employ two approaches: traditional RFID tag-based identification and billing schemes, and deep learning-based schemes using convolutional neural networks to automatically identify dishes. The former offers advantages such as high accuracy and the ability to use the same set of RFID tags across all restaurants, making it easy to implement. However, it is costly and the RFID chips are susceptible to damage in high-temperature environments. The latter approach eliminates the need to modify existing tableware and can directly identify dish names, facilitating subsequent backend analysis of sales and health data. However, it requires collecting numerous images of dishes for each restaurant and manually labeling all dishes in the images to create labeled data for the deep learning model. This time-consuming and expensive process significantly hinders the large-scale adoption of deep learning-based solutions.

[0003] To address the problems of the above solutions, this invention proposes a food classification method based on an improved VisionTransformer, namely the Food-VIT model. The core of this method is its focus on classifying food based on a small number of samples. It eliminates the need for collecting large datasets of food samples; only a small number of food images are required for each category. Therefore, the classification method based on the Food-VIT model proposed in this invention can significantly reduce labor costs and enable large-scale application. Summary of the Invention

[0004] The purpose of this invention is to propose a method for classifying dishes with a small number of samples, based on the Food-VIT model, which can adapt to various actual restaurant environments.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A few-sample food classification method, utilizing the Food-VIT model to identify food, includes the following steps:

[0007] S1. Obtain dish images and label them to build a dish sample library. p , sample library S p Reconstruct the categorized sample library S according to categories. c ;

[0008] S2, the reconstructed classification sample library S c Used to train the Food-VIT model, resulting in a trained classification network;

[0009] S3. The images of the dishes to be classified are passed through the Food-VIT classification network to obtain the recognition results.

[0010] The trained Food-VIT model performs three image transformations on the input sample image and fuses the category features obtained from the four images to obtain category features.

[0011] The sample image undergoes three image transformations, including a "global + local" image, a "local" image, and a "missing global" image.

[0012] The trained Food-VIT model for recognizing food images includes the following steps:

[0013] P1. Input a dish image and perform three image transformations on the dish image;

[0014] P2. Enter the Patch Embendding layer and convert the four dish images from two-dimensional images to one-dimensional images;

[0015] P3. The image information is encoded through the Transformer Encoder layer, and feature fusion is performed to identify image features.

[0016] P4. After passing through the MLP Head layer, the final category is obtained based on the category features.

[0017] Compared with existing technologies, the present invention provides a method for classifying dishes with a small sample size, which has the following beneficial effects:

[0018] This invention proposes an improved model based on VIT, namely the Food-VIT model, and applies this model to the classification of dishes with few samples. It only requires a small amount of dataset for training, which reduces the cost of collecting and labeling dishes, reduces labor costs, and shortens the training cycle. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a few-sample food classification method based on the Food-VIT model according to an embodiment of the present invention;

[0020] Figure 2 The food sample library S in this embodiment of the invention p and classification sample library S c The establishment of the Food-VIT model and the training flowchart;

[0021] Figure 3 This is a flowchart of the dish recognition process according to an embodiment of the present invention;

[0022] Figure 4This is a structural diagram of the Vision Transformer model according to an embodiment of the present invention;

[0023] Figure 5 This is a structural diagram of the Food-VIT model according to an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0025] Example:

[0026] This invention provides a method for detecting and identifying dishes with a small sample size, which utilizes a classification network for dish classification.

[0027] First, a small number of food images were collected and labeled using labeling software. Then, the labeled food images were used to build a food classification sample library S based on food categories. c This is used as the training set to train the Food-VIT model. The trained Food-VIT model can be used to determine the category of the dish in the image of the dish to be detected, and obtain the final classification result.

[0028] Includes the following steps:

[0029] S1. Obtain food images and build a food sample library. p , sample library S p Reconstruct the categorized sample library S according to categories. c ;

[0030] S2, the reconstructed classification sample library S c Used to train the Food-VIT model, resulting in a trained classification network;

[0031] S3. The images of the dishes to be classified are passed through the Food-VIT classification network to obtain the recognition results.

[0032] Furthermore, the trained Food-VIT model recognizes food images by including the following steps:

[0033] P1. Input a dish image and perform three image transformations on the dish image;

[0034] P2. Enter the Patch Embendding layer and convert the four dish images from two-dimensional images to one-dimensional images;

[0035] P3. The image information is encoded through the Transformer Encoder layer, and feature fusion is performed to identify image features.

[0036] P4. After passing through the MLP Head layer, the final category is obtained based on the category features.

[0037] Furthermore, the trained Food-VIT model performs three image transformations on the input sample image and fuses the category features obtained from the four images to obtain category features.

[0038] Furthermore, the three image transformations of the sample image include a "global + local" image, a "local" image, and a "missing global" image.

[0039] Figure 1 The diagram shown is a schematic representation of the entire process of an embodiment of this application. The specific implementation is as follows:

[0040] Step S110: Collect and label sample images of dishes from actual restaurants to establish a dish sample library S p According to the dish sample database S p Reconstruct a classification sample library S c And use this classification sample library S c Used as a training set to train the Food-VIT model;

[0041] Step S120: Obtain the category of each dish by passing the images of each dish to be classified through the Food-VIT classification network;

[0042] Step S130: Obtain the final classification result of the images of the dishes to be classified.

[0043] The following provides a detailed description of several key steps in the embodiments of this application:

[0044] Figure 2 This describes the creation of the food sample dataset and the classification dataset, and the training of the classification model Food-VIT, where:

[0045] The collected food sample dataset includes images of various dishes, with only a small number of samples for each dish category;

[0046] Step S210: Label the collected dish image data to establish a dish sample library S p ;

[0047] Step S220, add the dish sample library S p Reconstructing a sample database for food category S c ;

[0048] Step S230: Input the samples from the reconstructed classification sample library into the Food-VIT network;

[0049] Step S240: Perform three image transformations on the input sample image, and then fuse the category features obtained from these four images to obtain the final category features;

[0050] Step S250: Obtain the category result of the input sample based on the final category features.

[0051] Figure 3 This describes the food identification process, where:

[0052] Step S310: Extract images of each dish from the collected dish images;

[0053] Step S320: Extract the images of each dish and use the Food-VIT classification network to obtain the category of each dish;

[0054] Step S330 yields the final classification result.

[0055] Figure 4 The structure diagram of the Vision Transformer model includes:

[0056] Step S410, Input image;

[0057] Step S420, passing through the Patch Embedding layer;

[0058] Step S430, passing through the Transformer Encoder layer;

[0059] Step S440: Obtain the category features corresponding to the input image;

[0060] In step S450, after passing through the MLP Head layer, the final category is obtained based on the category features.

[0061] Figure 5 The structure diagram of the Food-VIT model includes:

[0062] Step S510: Input an image and perform three image transformations on it;

[0063] Step S520, passing through the Patch Embedding layer;

[0064] Step S530, passing through the Transformer Encoder layer;

[0065] Step S540: The category features obtained from the four images are fused to obtain the final category features;

[0066] In step S550, after passing through the MLP Head layer, the final category is obtained based on the category features.

[0067] The three types of sample images in this invention are "global + local" images, "local" images, and "missing global" images.

[0068] The "global + local" image approach simultaneously considers both global and local features; the "local" image focuses on the fine texture features of the food image; the "missing global" image, because other food items are included in some food images, can interfere with the classification process. This type of image transformation reduces the interference. In this invention, the three image transformations increase sample diversity, and experiments have shown that performing these three image transformations simultaneously yields the best classification results. The category features obtained from these four images are then fused using additive features to obtain the final category features.

[0069] This invention compares the accuracy of experimental results under the VIT model and the Food-VIT model when there are 10, 5 and 1 samples of each type of dish. The results are shown in Table 1.

[0070] Table 1

[0071]

[0072] Based on the above embodiments, this invention proposes an improved VIT-based model, namely the Food-VIT model, and applies this model to the classification of dishes with few samples. It only requires a small amount of data for training, which reduces the cost of collecting and labeling dishes, reduces labor costs, and shortens the training cycle.

[0073] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for classifying dishes using a small number of samples, employing the Food-VIT model to identify food, characterized in that, Includes the following steps: S1. Obtain dish images and label them to build a dish sample library. , sample library Reconstruct the categorized sample library by category. ; S2. Reconstruct the classification sample library Used to train the Food-VIT model, resulting in a trained classification network; The Food-VIT model identifies food images through the following steps: P1. Input a dish image and perform three image transformations on it; the three image transformations include "global + local" image, "local" image, and "missing global" image. P2. Enter the Patch Embendding layer and convert the four dish images from two-dimensional images into one-dimensional images; P3. The image information is encoded through the Transformer Encoder layer, and feature fusion is performed to identify image features. P4. After passing through the MLP Head layer, the final category is obtained based on the category features; S3. The images of the dishes to be classified are passed through the Food-VIT classification network to obtain the recognition results.

2. The method for classifying dishes with a small sample size according to claim 1, characterized in that: The trained Food-VIT model performs three image transformations on the input sample image and fuses the category features obtained from the four images to obtain category features.

Images