Food waste condition detection method and device
By collecting video data through cameras and using the Laplace variance method and SwiftFormer neural network, the real-time and accuracy issues of food waste detection in cafeteria buffets were solved, and efficient food waste classification and timely intervention were achieved.
Patent Information
- Application Number
- CN202510657106.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-21
- Publication Date
- 2025-09-19
AI Technical Summary
The existing canteen buffet food waste detection methods have problems such as poor real-time performance, low accuracy, and long time consumption, making it difficult to accurately classify food waste and intervene in a timely manner.
A camera is used to collect video data in real time, the Laplace variance method is used to filter target data, and the SwiftFormer neural network is used to detect food waste, including classification into small, medium, large and paper towel obstruction.
It achieves high-precision and high-speed detection of food waste, and can achieve high-accuracy classification and timely intervention while meeting lightweight requirements.
Smart Images
Figure CN120673301A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of artificial intelligence technology, and in particular relates to a method and device for detecting food waste. Background Art
[0002] With increasing attention paid to food resources and ongoing campaigns to combat food waste, the issue of food waste is receiving increasing attention, and it is no exception in cafeterias. With the increasing number of people dining in cafeterias, buffets have become widely popular due to their convenience and variety, but this has also led to a serious food waste problem. To reduce food waste in cafeteria buffets, two main approaches are currently used: manual monitoring, which not only requires significant manpower but is also subject to subjective factors, making comprehensive and accurate monitoring difficult and ineffective. The other approach involves using measuring tools, such as standard plate scales, which are inefficient and unable to provide real-time, dynamic monitoring of food waste. This lack of timeliness makes it difficult to provide timely feedback on food waste. At the same time, AI technology is rapidly developing across various fields and achieving significant results. AI's image recognition, data analysis, and intelligent decision-making capabilities offer new insights and possibilities for addressing food waste in cafeteria buffets. By applying AI, food waste can be accurately categorized, enabling the development of more targeted and effective food waste prevention strategies. This not only helps improve the utilization of food resources and reduce waste, but also promotes a positive social culture of food conservation, which holds significant practical significance.
[0003] Nonlinear regression problems are often solved through machine learning. Early nonlinear regression methods primarily used polynomial regression to fit nonlinear relationships. In the 1990s, decision trees and ensemble methods were proposed and developed by various researchers as effective tools for solving nonlinear regression problems. These include concepts such as bagging, random forests, gradient boosting, XGBoost, and stacking. However, weak learners, such as decision trees, have low complexity but high bias, are prone to overfitting during training, and are highly dependent on ensemble methods. Bagging, an ensemble learning method based on bootstrap sampling, is robust to noise and outliers in the data, but its effectiveness is limited by high correlation between learners. Boosting, an iterative ensemble learning method, adaptively adjusts weights based on the distribution characteristics of different samples during training. However, it also suffers from noise sensitivity, long training times, and difficulty in parallelization. Stacking, while constructing multiple layers of models for ensemble learning and introducing diverse models to improve model accuracy, can overfit the training data, resulting in poor generalization performance on unseen data. These models have excellent fitting capabilities in datasets with sufficient data and complete features, but they have obvious shortcomings in actual cafeteria buffet scenarios.
[0004] Canteens have relatively concentrated dining times and large numbers of diners, requiring waste prevention methods to be highly real-time and capable of efficiently analyzing and processing massive amounts of data. However, existing methods are slow, making it difficult to intervene in wasteful behavior in a timely manner. The accuracy of these methods in classifying wasteful behavior needs to be improved, with misjudgments and omissions often occurring, making them ineffective in providing effective support for canteens to formulate precise waste prevention measures. Summary of the Invention
[0005] In view of this, the purpose of this application is to provide a method for detecting food waste, which has a high average recognition accuracy and good recognition speed, and can achieve high-accuracy detection and classification of waste while meeting high speed and light weight.
[0006] The present application provides a method for detecting food waste, the method comprising:
[0007] Use cameras to collect video data from the cafeteria recycling area in real time;
[0008] Using the Laplace variance method to filter out target data that can reflect food waste from the video data;
[0009] Based on the target data, the food waste situation is detected using a pre-obtained classification model to obtain a detection result; wherein the food waste situation includes: little, medium, much and paper towel obstruction.
[0010] Furthermore, the method of using the Laplace variance method to filter out target data that can reflect the food waste situation from the video data includes:
[0011] Extracting the video data frame by frame into a plurality of color images;
[0012] Grayscale each color image to obtain the corresponding grayscale image;
[0013] For each grayscale image, the Laplace operator is used to calculate the Laplace transform value of each pixel in the grayscale image to evaluate the clarity of the grayscale image;
[0014] The color image corresponding to the high-definition grayscale image is used as the target data.
[0015] Furthermore, the classification model is obtained in advance by the following method:
[0016] Get historical data of the cafeteria recycling area collected by the camera;
[0017] Screening key frame images with high definition in the historical data and manually annotating food waste situations;
[0018] The SwiftFormer neural network is trained using the labeled key frame images to obtain the classification model.
[0019] Furthermore, the SwiftFormer neural network includes: an embedding layer, a convolutional encoder, and a SwiftFormer encoder;
[0020] The convolution encoder includes: a 3×3 depth convolution layer, a normalization layer and a GeLU activation function layer;
[0021] The SwiftFormer encoder includes: a 1×1 point-by-point convolutional layer, an efficient additive attention layer, and a linear layer.
[0022] Furthermore, the SwiftFormer neural network is trained using the annotated keyframe images, including:
[0023] The embedding layer is used to segment the annotated keyframe image into fixed-size image blocks, then convert the image blocks into feature vectors through linear projection and superimpose position encodings. The annotated category is used as a global label to drive the embedding layer to learn basic semantic features related to food waste.
[0024] The convolutional encoder is used to receive the patch features output by the embedding layer and extract local spatial features through 3×3 depthwise convolution. The labeled category is adjusted by backpropagation of the loss function to enhance the extraction of local features such as edges and textures.
[0025] The SwiftFormer encoder uses labeled categories to drive attention to key areas, generates query vectors containing global semantics, and fuses local features with global features to achieve accurate classification.
[0026] Furthermore, after obtaining the test results, the method further includes:
[0027] When the detection results indicate that there is a lot of food waste and paper towels are blocking the food, the corresponding voice prompt function will be activated.
[0028] The present application also provides a device for detecting food waste, the device comprising:
[0029] The data acquisition module is used to collect video data of the canteen recycling area in real time through the camera;
[0030] a data processing module, configured to filter target data that can reflect food waste from the video data using a Laplace variance method;
[0031] A detection module is used to detect food waste based on the target data using a pre-obtained classification model; wherein the food waste conditions include: little, medium, much and paper towel obstruction.
[0032] The present application also provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the food waste detection method as described above are performed.
[0033] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the food waste detection method as described above are executed.
[0034] The food waste detection method and device provided in this application have high average recognition accuracy and good recognition speed, and can achieve precise detection and classification of waste with high accuracy while meeting the requirements of high speed and light weight. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 A flow chart showing a method for detecting food waste provided in an embodiment of the present application is shown;
[0036] Figure 2 A schematic diagram showing the structure of a food waste detection device provided in an embodiment of the present application is shown;
[0037] Figure 3 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0038] In order to make the purpose, technical solution and advantages of this technical solution more clear, the following technical solution is further described in detail in conjunction with specific implementation methods. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of this technical solution.
[0039] Please refer to Figure 1 Flowchart of the method for detecting food waste as shown. Figure 1 As shown, the detection method includes:
[0040] S101. Collect video data of the canteen recycling area in real time through a camera.
[0041] In this step, wired network transmission is performed, and the camera is connected to the local area network where the data storage server is located using an Ethernet cable; the Internet Protocol IP is used to stably and efficiently transmit the video data recorded by the camera in real time to the server at a higher rate.
[0042] S102: Filter target data that can reflect food waste from the video data using the Laplace variance method.
[0043] In this step, the Laplace variance method is a technique used to assess image clarity. It grayscales the color image using the formula Gray = 0.299R + 0.587G + 0.114B. A 3×3 Laplace operator template is then applied to the grayscale image for filtering. The Laplace transform value of each pixel is calculated, enhancing image edges and details. Finally, the variance of all pixel values in the Laplace transformed image is calculated and compared with a set variance threshold. A larger variance indicates higher image clarity, and vice versa. Frames with variance values less than 50 are considered blurry and are filtered out.
[0044] In specific implementation, the target data that can reflect the food waste situation can be screened out from the video data using the Laplace variance method in the following manner:
[0045] Step 1021: Extract the video data frame by frame into a plurality of color images.
[0046] Step 1022: grayscale each color image to obtain a corresponding grayscale image.
[0047] Step 1023: For each grayscale image, use the Laplace operator to calculate the Laplace transform value of each pixel in the grayscale image to evaluate the clarity of the grayscale image.
[0048] Step 1024: Use the color image corresponding to the high-definition grayscale image as target data.
[0049] S103. Based on the target data, the food waste situation is detected using a pre-obtained classification model to obtain a detection result.
[0050] Among them, the food waste situations include: little, medium, much and covered by paper towels.
[0051] In specific implementation, the classification model can be obtained in advance through the following methods:
[0052] Step 1031: Obtain historical data of the cafeteria recycling area collected by the camera.
[0053] As an example, the embodiment of the present application extracts video data from the recycling area of a cafeteria for 10 working days. Taking into account the particularity of meal times, the data extraction focuses on the peak meal times at noon and in the evening, and the time is set to 45 minutes. During these two time periods, the flow of staff in the cafeteria is frequent, and the food recycling situation is relatively concentrated, which can effectively reflect the actual situation of food waste. The video of the relevant time period is extracted at a frame rate of 25 frames per second, and the cumulative extraction time reaches 15 hours, with a total of 1.35 million image data. These images cover the behavioral information of different dining groups on different dates in the recycling area, providing rich and detailed basic data for the detection of food waste.
[0054] Step 10312: Filter the historical data for key frame images with high definition, and manually label the food waste situation.
[0055] In this step, the Laplace variance method is used to filter out blurred frame images from the historical data. After obtaining the key frame images, they are inspected frame by frame. Images without plates cannot provide effective information directly related to food waste. Removing these images can avoid introducing interference factors in model training, ensuring the accuracy and pertinence of the training results. The images are then divided into four categories: "little," "medium," "lots," and "occluded by paper towels" and manually labeled. Specifically, "little" indicates that there is very little food left on the plate, with almost no obvious waste; "medium" indicates that there is a certain amount of food left on the plate, but it has not yet reached the level of serious waste; "lots" indicates that there is a large amount of food left on the plate, with more obvious waste; and "occluded by paper towels" is mainly used for images where the amount of food left cannot be accurately determined due to paper towel obstruction.
[0056] Step 1033: Use the labeled key frame images to train the SwiftFormer neural network to obtain the classification model.
[0057] The SwiftFormer neural network includes an embedding layer, a convolutional encoder, and a SwiftFormer encoder. The convolutional encoder includes a 3×3 depth convolutional layer, a normalization layer, and a GeLU activation function layer. The SwiftFormer encoder includes a 1×1 point-by-point convolutional layer, an efficient additive attention layer, and a linear layer. The convolutional encoder includes a 3×3 depth convolutional layer, a normalization layer, and a GeLU activation function layer. The SwiftFormer encoder includes a 1×1 point-by-point convolutional layer, an efficient additive attention layer, and a linear layer.
[0058] In this step, traditional machine learning methods are usually limited by their underlying linear models, and it is difficult to accurately capture the nonlinear relationship between complex data. As a deep learning model, the SwiftFormer neural network has a more powerful expression ability through multi-layer neuron connections and layer-by-layer feature extraction, and can more effectively explore and learn nonlinear patterns in data. Therefore, a SwiftFormer neural network is constructed in this embodiment. The neural network has a strong nonlinear mapping ability and is suitable for solving problems with complex internal mechanisms. The embodiment of the present application not only sets up 3×3 deep convolution and 1×1 point-by-point convolution to learn more effective feature representations, but also reduces computational complexity by introducing an efficient additive attention mechanism, while improving the accuracy and speed of the model.
[0059] Specifically, the processing process of the embedding layer includes: dividing the input image into image blocks of fixed size, converting it into a feature vector through linear projection, superimposing sine and cosine position encoding to retain spatial position information, and outputting a feature vector containing semantic and position information. The output is a feature vector whose dimensions include batch size, patch grid size, and feature dimension.
[0060] The processing process of the convolutional encoder includes: taking the embedded layer patch feature as input, extracting local spatial features through 3×3 depth convolution, maintaining the spatial size and number of channels, generating a local feature map containing edges and textures through normalization and GeLU activation, and outputting the same dimension to represent the neighborhood spatial details.
[0061] The SwiftFormer encoder's processing includes: taking the local features of the convolutional encoder as input, generating a query / key matrix through 1×1 point-by-point convolution, modeling the global context through efficient additive attention query-key interaction and element-wise multiplication to capture long-range dependencies, and fusion of local-global features through linear transformation and residual connection output to represent the overall semantics for classification.
[0062] In practice, the SwiftFormer neural network can be trained using the annotated keyframe images in the following way:
[0063] Step 1033a: The embedding layer is used to segment the annotated key frame image into image blocks of fixed size, and then convert the image blocks into feature vectors through linear projection and superimpose position encoding, and use the annotated category as a global label to drive the embedding layer to learn basic semantic features related to food waste.
[0064] Step 1033b: The convolution encoder is used to receive the patch features output by the embedding layer and extract local spatial features through 3×3 depth convolution. The labeled category is adjusted by backpropagation of the loss function to enhance the extraction of local features including edges and textures.
[0065] Step 1033c: The SwiftFormer encoder uses the labeled categories to drive attention to focus on key areas, generates a query vector containing global semantics, and fuses local features with global features to achieve accurate classification.
[0066] In addition, after obtaining the test results, the method further includes:
[0067] When the detection results indicate that there is a lot of food waste and paper towels are blocking the food, the corresponding voice prompt function will be activated.
[0068] Based on the same inventive concept, the embodiment of the present application also provides a food waste detection device corresponding to the food waste detection method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned method in the embodiment of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.
[0069] Please refer to Figure 2 The structural diagram of the detection device for food waste is shown in FIG. Figure 2 As shown, the detection device 200 includes:
[0070] The data acquisition module 201 is used to collect video data of the canteen recycling area in real time through a camera;
[0071] A data processing module 202 is configured to filter target data that can reflect food waste from the video data using a Laplace variance method;
[0072] The detection module 203 is used to detect food waste based on the target data using a pre-obtained classification model; wherein the food waste conditions include: little, medium, much and paper towel obstruction.
[0073] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 3 As shown in FIG, the electronic device 300 includes a processor 301, a memory 302 and a bus 303.
[0074] The memory 302 stores machine-readable instructions executable by the processor 301. When the electronic device 300 is running, the processor 301 communicates with the memory 302 via the bus 303. When the machine-readable instructions are executed by the processor 301, the above-mentioned Figure 1 The steps of the food waste detection method in the method embodiment shown are described in detail in the method embodiment and will not be described in detail here.
[0075] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the food waste detection method in the method embodiment shown are described in detail in the method embodiment and will not be described in detail here.
[0076] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0077] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.
[0078] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0079] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0080] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0081] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.
Claims
1. A method for detecting food waste, characterized in that: The detection method comprises: Use cameras to collect video data from the cafeteria recycling area in real time; Using the Laplace variance method to filter out target data that can reflect food waste from the video data; Based on the target data, the food waste situation is detected using a pre-obtained classification model to obtain a detection result; wherein the food waste situation includes: little, medium, much and paper towel obstruction.
2. The method according to claim 1, wherein The method of using the Laplace variance method to filter out target data that can reflect the food waste situation from the video data includes: Extracting the video data frame by frame into a plurality of color images; Grayscale each color image to obtain the corresponding grayscale image; For each grayscale image, the Laplace operator is used to calculate the Laplace transform value of each pixel in the grayscale image to evaluate the clarity of the grayscale image; The color image corresponding to the high-definition grayscale image is used as the target data.
3. The method according to claim 1, wherein The classification model is obtained in advance by the following method: Get historical data of the cafeteria recycling area collected by the camera; Screening key frame images with high definition in the historical data and manually annotating food waste situations; The SwiftFormer neural network is trained using the labeled key frame images to obtain the classification model.
4. The method according to claim 3, wherein The SwiftFormer neural network includes: an embedding layer, a convolutional encoder, and a SwiftFormer encoder; The convolution encoder includes: a 3×3 depth convolution layer, a normalization layer and a GeLU activation function layer; The SwiftFormer encoder includes: a 1×1 point-by-point convolutional layer, an efficient additive attention layer, and a linear layer.
5. The method according to claim 4, wherein The method of training the SwiftFormer neural network using the annotated keyframe images includes: The embedding layer is used to segment the annotated keyframe image into fixed-size image blocks, then convert the image blocks into feature vectors through linear projection and superimpose position encodings. The annotated category is used as a global label to drive the embedding layer to learn basic semantic features related to food waste. The convolutional encoder is used to receive the patch features output by the embedding layer and extract local spatial features through 3×3 depthwise convolution. The labeled category is adjusted by backpropagation of the loss function to enhance the extraction of local features such as edges and textures. The SwiftFormer encoder uses labeled categories to drive attention to key areas, generates query vectors containing global semantics, and fuses local features with global features to achieve accurate classification.
6. The method according to claim 1, wherein After obtaining the test results, the method further includes: When the detection results indicate that there is a lot of food waste and paper towels are blocking the food, the corresponding voice prompt function will be activated.
7. A device for detecting food waste, characterized in that: The detection device comprises: The data acquisition module is used to collect video data of the canteen recycling area in real time through the camera; a data processing module, configured to filter target data that can reflect food waste from the video data using a Laplace variance method; A detection module is used to detect food waste based on the target data using a pre-obtained classification model; wherein the food waste conditions include: little, medium, much and paper towel obstruction.
8. An electronic device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 6 are performed.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.