Kitchen garbage weight estimation model, establishment method, weight estimation method, equipment and medium

Abstract

The invention provides a kitchen garbage weight estimation model, an establishment method, a weight estimation method, equipment and a medium. The establishment method comprises the following steps: acquiring information and establishing a database; establishing a preliminary deep learning model based on the database; training the model; after training is finished, the recognition accuracy of the model is tested, if the recognition accuracy reaches the standard, the trained deep learning model serves as a weight estimation model, and if the recognition accuracy does not reach the standard, training continues. The weight estimation model is obtained by the method. The method comprises the step of analyzing an image acquired by a user by using the model. The apparatus comprises at least one processor; and the memory stores a program instruction, and the program instruction comprises an instruction for executing the establishment method or the revaluation method. Program instructions are stored in the storage medium. According to the invention, garbage classification identification and weight calculation can be automatically completed, a more convenient and accurate processing mode is provided, and manual operation is reduced.

Description

Technical Field

[0001] The present invention relates to the field of kitchen waste treatment. Specifically, it relates to a kitchen waste weight estimation model, a method for establishing the model, a weight estimation method, equipment, and a medium. Background Art

[0002] Currently, the classified treatment of household kitchen waste has become one of the important issues in urban management and environmental protection. However, traditional treatment methods usually require manual use of simple weighing equipment, which is inefficient and labor-consuming. Summary of the Invention

[0003] The purpose of the present invention is to solve at least one of the above-mentioned deficiencies existing in the prior art. For example, one of the purposes of the present invention is to achieve convenient and accurate weighing of kitchen waste.

[0004] To achieve the above purpose, on the one hand, the present invention provides a method for establishing a kitchen waste weight estimation model.

[0005] The establishment method may include the following steps: collecting standard images, non-standard images, background images, and kitchen waste weight parameters, and establishing a database; wherein, the standard images include images that simultaneously have the mouth of the target kitchen waste bucket and the kitchen waste inside the bucket, and in some of the standard images, there is identification information on the bucket mouth, while in others, there is no identification information; the non-standard images include: images of the target kitchen waste bucket in the closed state, images that include the mouth of the target kitchen waste bucket but do not include the kitchen waste inside the bucket, and images that include the bottom of the target kitchen waste bucket and no kitchen waste; the background images include images of non-target kitchen waste buckets and images of non-kitchen waste; the kitchen waste weight parameter is the weight parameter of the kitchen waste in the target kitchen waste bucket corresponding to the standard image; initially establishing a deep learning model; training the deep learning model using the database; after the training is completed, testing the recognition accuracy of the trained model on the target kitchen waste bucket, the identification information, the kitchen waste inside the target kitchen waste bucket, and the weight parameter of the kitchen waste inside the target kitchen waste bucket. If the accuracy rate meets the standard, then the trained deep learning model is used as the weight estimation model. If the accuracy rate does not meet the standard, then continue training until it meets the standard.

[0006] On the other hand, the present invention provides a kitchen waste weight estimation model.

[0007] The weight estimation model is established by the method as described above.

[0008] Optionally, the model is capable of analyzing images collected by the user. The analysis may include: inferring the image to obtain multi-scale matrix features; inferring the multi-scale feature matrix to obtain multi-scale grid confidence and bounding boxes; using a confidence threshold to filter out grids with low confidence to detect whether there are targets meeting the threshold; using non-maximum suppression to remove redundant detection boxes; in the case of detecting a target meeting the threshold, judging whether there is a target food waste bin in the target according to the classification result of the target, and in the case of having one, outputting the confidence, classification, and bounding box of the target food waste bin; judging whether the target food waste bin meets the shooting specification according to the classification of the target; in the case of meeting the shooting specification, judging whether identification information is detected; in the case of detecting the identification information, performing image interception, restoration, and decoding of the identification information, and outputting the decoded content of the identification information; in the case of meeting the shooting specification, calculating the original image feature matrix; scaling the feature matrices of all scales of the multi-scale matrix features to the same size to obtain a fused multi-scale feature matrix, where the fused multi-scale feature matrix and the original image feature matrix are the same in the feature map dimension; connecting the original image feature matrix and the fused multi-scale feature matrix in the feature channel dimension; splicing the connected feature matrices into a one-dimensional matrix, and performing regression on the one-dimensional feature matrix using an image regression layer, where the image regression layer includes multiple linear modules; outputting a preliminary estimation result by the linear layer; in the case where the training set of the model uses a normalization method, performing denormalization on the preliminary estimation result to obtain an estimated weight and outputting it.

[0009] Further optionally, in the case where it is judged according to the classification result of the target that there is no target food waste bin, the model ends the analysis.

[0010] Further optionally, in the case where it is judged that the shooting specification is not met, the model ends the analysis.

[0011] Further optionally, in the case where the identification information is not detected, the model ends the analysis, or the model still proceeds with the subsequent weight estimation process.

[0012] On the other hand, the present invention provides a method for estimating the weight of food waste.

[0013] The method includes: analyzing the images collected by the user using the food waste weight estimation model as described above.

[0014] On another aspect, the present invention provides a computer device.

[0015] The computer device includes: at least one processor; a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method for establishing the food waste weight estimation model as described above, or instructions for executing the food waste weight estimation method as described above.

[0016] Yet another aspect of the present invention provides a computer-readable storage medium.

[0017] Computer program instructions are stored on the computer-readable storage medium. When the computer program instructions are executed by the processor, the method for establishing the kitchen waste weight estimation model as described above is implemented, or the method for kitchen waste weight estimation as described above is implemented.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] (1) After the customer takes a photo and uploads it to the model, the model can automatically perform image recognition, ID QR code recognition and weight measurement, reducing the need for manual operation and improving processing efficiency and accuracy.

[0020] (2) Customers only need to take a simple photo and input it into the model, and the model can automatically complete the identification and weight calculation of garbage classification, providing a more convenient and accurate processing method.

[0021] (3) The present invention uses ID QR code recognition to determine the legitimacy of the container and assigns a code to the kitchen waste collected by the household for traceability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects and / or features of the present invention will become more apparent from the following description in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram of a weight estimation process of the weight estimation model of the present invention is shown.

[0024] Figure 2 The connection relationship between the linear module and the linear layer is shown. DETAILED DESCRIPTION

[0025] Hereinafter, the kitchen waste weight estimation model and establishment method, weight estimation method, device, and medium of the present invention will be described in detail with reference to exemplary embodiments.

[0026] In order to solve the problems of low efficiency and manpower consumption in the existing manual weighing of kitchen waste, this paper proposes a convenient and efficient technical solution. The core of this technical solution is a kitchen waste weight estimation model. The solution mainly includes: the customer opens the lid of the target kitchen waste bin and takes a photo and uploads it, and the kitchen waste estimation model calculates the weight of the kitchen waste in the bin.

[0027] Exemplary embodiment 1

[0028] This exemplary embodiment provides a method for establishing a kitchen waste weight estimation model.

[0029] The establishment method may include the following steps:

[0030] A1: Collect information and establish a database.

[0031] In this embodiment, the collected information may include standard images, non-standard images, background images, and the weight parameters of kitchen waste.

[0032] Among them, the standard images may include images with both the mouth of the target kitchen waste bucket and the kitchen waste inside the bucket. Some of the standard images have identification information on the bucket mouth, while the other part does not.

[0033] The non-standard images may include: images of the target kitchen waste bucket in the closed state, images that contain the mouth of the target kitchen waste bucket but do not contain the kitchen waste inside the bucket, and images that contain the bottom of the target kitchen waste bucket and have no kitchen waste.

[0034] The background images may include: images of non-target kitchen waste buckets and images of non-kitchen waste.

[0035] The weight parameter of the kitchen waste is the weight parameter of the kitchen waste in the target kitchen waste bucket corresponding to the standard image.

[0036] In this embodiment, the standard images and the weight parameters of the kitchen waste may be the photos taken and weighing information obtained by the applicant through the previous recovery and processing of each bucket of kitchen waste.

[0037] In this embodiment, the target kitchen waste bucket may have a specific shape and structure, and the bucket mouth has identification information.

[0038] In this embodiment, step A1 may further include: removing the incorrect, invalid, and duplicate information in the database.

[0039] In this embodiment, the identification information may include an ID code. Through the identification information, the traceability convenience and accuracy of the kitchen waste transfer statistics can be improved.

[0040] A2: According to the task requirements, initially establish a deep learning model. Optionally, the model may include a target detection sub-model, a weight estimation sub-model, and an identification detection sub-model. Among them, the target detection sub-model may include yolov5 for target kitchen waste bucket detection, the weight estimation sub-model may include an image regression model for mass estimation, the image regression model includes an inceptionv4 network, a C3 convolution module + convolution module, an image regression layer, and a linear layer as the output layer, and the identification detection sub-model may include a super-resolution model for restoring identification information (such as ID code images).

[0041] A3: Train the initially established deep learning model.

[0042] Specifically, for the object detection task, a large number of target kitchen waste bins (including images taken in a standard or non-standard manner) and background images (such as images of non-kitchen waste bins, non-kitchen waste images) are used for training. For the weight estimation task, the parameters of the object detection model are frozen for training. For the identification information (such as ID code) recognition task, the original image of the identification information (such as the original ID code image) and the blurred image are used to train the super-resolution model.

[0043] A4: After the training is completed, the recognition accuracy of the deep learning model for the target kitchen waste bin, the kitchen waste in the target kitchen waste bin, and the weight parameters of the kitchen waste in the target kitchen waste bin is tested. If the accuracy meets the standard, the trained deep learning model is used as the weight estimation model. If the accuracy does not meet the standard, continue training until it meets the standard.

[0044] Exemplary Embodiment 2

[0045] This Exemplary Embodiment 2 provides a kitchen waste weight estimation model, which is established by the method described in Exemplary Embodiment 1.

[0046] In this embodiment, the model may include a yolov5 network, an inceptionv4 network, a C3 convolution module, a convolution module, multiple linear modules, a final output layer linear layer, and a super-resolution model. Each linear module includes a linear layer, a batchnorml layer, and a sigmoid activation function. The yolov5 network is widely used in the industrial community for its high efficiency in specific tasks and can bring a good experience to users.

[0047] In this embodiment, the weight estimation model can analyze the images collected by the user. The following combines Figure 2 to further illustrate the analysis process of the kitchen waste measurement model, which includes:

[0048] S1: The picture uploaded by the user is input into the backbone and neck networks of the yolov5 network for inference, and a multi-scale matrix feature is output. Among them, the picture can be uploaded to the model by the user through a WeChat mini-program, an APP, etc.

[0049] S2: Infer the multi-scale feature matrix to output the multi-scale grid confidence and bounding boxes. Among them, the Detect layer of the yolov5 network performs confidence prediction, object classification, and bounding box regression. In step S1, Yolov5 first outputs a multi-scale feature matrix, and inputs the multi-scale feature matrix into the detect layer to initially obtain the confidence, classification, and bounding boxes in each grid.

[0050] S3: Use a confidence threshold to filter out grids with low confidence. Because this task is specific and the images are relatively simple, a higher confidence threshold is set to filter out targets. Non-maximum suppression removes redundant detection boxes. The IoU (Intersection over Union) is calculated between the bounding box of the same target and the bounding box of the grid with the highest confidence. Detection boxes with large overlaps are considered redundant. Because this task is specific and the images are relatively simple, the IoU threshold is relatively low. A high IoU indicates a high degree of overlap between bounding boxes. Bounding boxes with high IoUs for similar targets are removed. Specifically, bounding boxes with IoUs exceeding the IoU threshold are removed. Detection boxes above the IoU threshold are considered redundant.

[0051] S4: Output the confidence score, classification, and bounding box of the target filtered in step S3. The confidence score will be used in subsequent demonstrations of target detection results. Specific classifications include: images containing both the target kitchen waste bin opening and the food waste inside; images of the target kitchen waste bin in a closed state; images containing the target kitchen waste bin opening but not the food waste inside; and images containing the target kitchen waste bin bottom but no food waste. The bounding box is used in subsequent demonstrations of the model's target detection results. Based on the classification results, determine whether the target kitchen waste bin is present. If so, jump to step S5. If not, the model ends and returns a recognition failure result.

[0052] S5: Based on the target category output in S4 (i.e., classification), determine whether it meets the shooting specifications (i.e., whether it is a standard image). If it meets the shooting specifications, jump to steps S6 and S9. If it does not meet the specifications, the process ends and the model returns the result of the shooting not meeting the specifications.

[0053] S6: Determine whether the ID code is detected based on the target category of S4. If it is detected, jump to step S7. If not, jump to step S9, that is, continue to perform subsequent weight estimation and output the estimation result.

[0054] S7: According to the bounding box of the ID code output by S4, the ID code image is cropped, the blurred image is restored with super-resolution, and the QR code decoding algorithm is called for decoding.

[0055] S8: Output the ID code decoding content.

[0056] S9: To obtain the original image, calculate the feature matrix through the inceptionv4 network. The output feature matrix is the same as S10 in terms of feature map dimension.

[0057] S10: Using the multi-scale feature matrix from step S1 as input, it uses multiple layers of C3 convolutional modules and convolutional modules to scale the feature matrices at all scales to the same size and concatenate them along the feature channel dimension. The scaled feature map dimensions are the same as in S9. Both the C3 convolutional module and the convolutional module directly use the feature extraction module of YoL V5.

[0058] S11: Concatenate the feature matrices of S9 and S10 in the feature channel dimension.

[0059] S12: Concatenate the feature maps into a one-dimensional matrix, and perform regression on the one-dimensional feature matrix using an image regression layer, which includes multiple linear modules, such as Figure 2 the 3 linear modules shown. Each linear module is a linear layer + batchnorml layer + sigmoid activation function. The final output layer is a linear layer. Since the training set for estimating weight normalizes the weight to 0-1, the result of the network is de-normalized to obtain the estimated weight and output.

[0060] Furthermore, the result output by the model can be displayed to the user through a mini-program terminal, APP, electronic device, etc.

[0061] Furthermore, as Figure 1 shown, when the model outputs the result, it can also output the bounding box of the target food waste bin and the bounding box of the ID code, and the bounding box can be displayed to the user through a mini-program terminal, APP, electronic device, etc.

[0062] Exemplary Embodiment 3

[0063] This Exemplary Embodiment 3 provides a food waste weight estimation device. The device includes a weight estimation model established by the method described in Exemplary Embodiment 1.

[0064] Exemplary Embodiment 4

[0065] This Exemplary Embodiment 4 provides a food waste weight estimation method. The weight estimation method in this exemplary embodiment is implemented based on the food waste weight estimation model in Exemplary Embodiment 2.

[0066] In this embodiment, the solution may further include the process of the user collecting images. For example, when the customer submits a household food waste order, the user opens the lid of the target food waste bin and takes a photo and uploads it.

[0067] In this embodiment, the user can upload the collected images to the model through a mini-program, APP, electronic device (such as a computer).

[0068] According to the method for establishing a food waste weight estimation model or the food waste weight estimation method of the present invention, it can be programmed into a computer program and the corresponding program code or instructions can be stored in a computer-readable storage medium. When the program code or instructions are executed by a processor, the processor executes the above method, and the following processor and memory can be included in a computer device.

[0069] Exemplary Embodiment 5

[0070] This exemplary embodiment provides a computer device, including:

[0071] At least one processor;

[0072] A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the establishment method according to Exemplary Embodiment 1 or instructions for executing the weight estimation method according to Exemplary Embodiment 4.

[0073] Exemplary Embodiment 6

[0074] This exemplary embodiment provides a computer-readable storage medium.

[0075] A computer program is stored on the storage medium, and when the computer program instructions are executed by a processor, the establishment method according to Exemplary Embodiment 1 is implemented, or the weight estimation method according to Exemplary Embodiment 4 is implemented.

[0076] The computer-readable storage medium can be any data storage device that stores data that can be read by a computer system. For example, examples of computer-readable storage media may include: read-only memory, random access memory, compact disc read-only memory, magnetic tape, floppy disk, optical data storage device, and carrier waves (such as data transmission via the Internet through wired or wireless transmission paths).

[0077] One of the innovations of the present invention is to combine technologies such as target kitchen waste bins, image recognition, and barcode recognition to achieve automation and intelligence in the determination of household kitchen waste, container determination, weight calculation, and ID QR code recognition. The present invention can continue to perform model training through the data collected later, improving the accuracy of image recognition; at the same time, the present invention reduces the need for manual operations by means of a kitchen waste weight estimation model, improving the processing efficiency.

[0078] Although the present invention has been described above in conjunction with exemplary embodiments, those skilled in the art should clearly understand that various modifications and changes can be made to the exemplary embodiments of the present invention without departing from the spirit and scope defined by the claims.

Claims

1. A method for establishing a weight estimation model for kitchen waste, characterized in that, The method includes the following steps: Collect standard images, non-standard images, background images, and the weight parameters of kitchen waste, and establish a database; among them, the standard images include images with both the mouth of the target kitchen waste bucket and the kitchen waste inside the bucket. Some of the standard images have identification information on the bucket mouth, while others do not; the non-standard images include: images of the target kitchen waste bucket in the closed state, images that include the mouth of the target kitchen waste bucket but do not include the kitchen waste inside the bucket, and images that include the bottom of the target kitchen waste bucket and have no kitchen waste; the background images include images of non-target kitchen waste buckets and images of non-kitchen waste; the weight parameter of the kitchen waste is the weight parameter of the kitchen waste in the target kitchen waste bucket corresponding to the standard image; Preliminarily establish a deep learning model; Use the database to train the deep learning model; After the training is completed, test the recognition accuracy of the trained model for the target kitchen waste bucket, identification information, the kitchen waste in the target kitchen waste bucket, and the weight parameter of the kitchen waste in the target kitchen waste bucket. If the accuracy meets the standard, use the trained deep learning model as the weight estimation model. If the accuracy does not meet the standard, continue training until it meets the standard.

2. The method for establishing a kitchen waste weight estimation model according to claim 1, wherein The method further includes: removing incorrect, invalid, and duplicate information from the database.

3. A kitchen waste weight estimation model, characterized in that, The weight estimation model is established by the method described in claim 1 or 2.

4. The food waste weight estimation model according to claim 3, characterized in that The model includes a yolov5 network, an inceptionv4 network, a C3 convolution module, a convolution module, a linear module, a linear layer as the output layer, and a super-resolution model. Each linear module includes a linear layer, a batchnorml layer, and a sigmoid activation function.

5. The food waste weight estimation model according to claim 3, wherein The model can analyze the images collected by the user, and the analysis includes: Perform inference on the image to obtain multi-scale matrix features; Infer the multi-scale feature matrix to obtain multi-scale grid confidence and bounding boxes; Use the confidence threshold to filter out the grids with lower confidence to detect whether there is a target that meets the threshold; use non-maximum suppression to remove redundant detection boxes; In the case of detecting a target that meets the threshold, judge whether there is a target kitchen waste bucket in the target according to the classification result of the target. If so, output the confidence, classification, and bounding box of the target kitchen waste bucket; Judge whether the target kitchen waste bucket meets the shooting specification according to the classification of the target; In the case of meeting the shooting specification, judge whether the identification information is detected; in the case of detecting the identification information, perform image interception, restoration, and decoding of the identification information, and output the decoded content of the identification information; In the case of meeting the shooting specification, calculate the original image feature matrix; scale the feature matrices of all scales of the multi-scale matrix features to the same size to obtain a fused multi-scale feature matrix. The fused multi-scale feature matrix and the original image feature matrix are the same in the feature map dimension; Connect the original image feature matrix and the fused multi-scale feature matrix in the feature channel dimension; Stitch the connected feature matrices into a one-dimensional matrix, and use the image regression layer to perform regression on the one-dimensional feature matrix. The image regression layer includes multiple linear modules; Output a preliminary estimation result by the linear layer; In the case where a normalization method is used in the training set of the model, the preliminary estimation result is denormalized to obtain the estimated weight and output it.

6. The kitchen waste weight estimation model according to claim 4, wherein The classification includes: an image that simultaneously includes the mouth of the target food waste bin and the food waste inside the bin, an image of the target food waste bin in the closed state, an image that includes the mouth of the target food waste bin but does not include the food waste inside the bin, and an image that includes the bottom of the target food waste bin and has no food waste.

7. The food waste weight estimation model according to claim 4, wherein Use the backbone and neck networks of the yolov5 network to perform inference on the image and output multi-scale matrix features; Use the Detect layer of the yolov5 network to infer the multi-scale feature matrix and output multi-scale grid confidence and bounding boxes; Calculate the original image feature matrix through the inceptionv4 network.

8. A method for estimating the weight of kitchen waste, characterized in that, The method includes: analyzing the image collected by the user using the food waste weight estimation model according to any one of claims 3 to 7.

9. A computer device, characterized in that, Includes: At least one processor; A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to claim 1 or 2, or instructions for executing the method according to claim 8.

10. A computer-readable storage medium, on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, the method according to claim 1 or 2 is implemented, or the method according to claim 8 is implemented.

Images