A detection and recognition method for disposable tableware sets based on deep learning technology

Abstract

The present invention discloses a disposable tableware set detection and recognition method based on deep learning technology, comprising the following steps: step S1: acquiring an image of a disposable tableware set to be processed; step S2: sending the acquired image into a tableware detection and recognition module for data preprocessing and tableware detection and recognition; step S3: sending the tableware position information output by S2 into an out-of-bounds detection module, and judging whether the tableware has crossed the boundary by position comparison; step S4: sending the tableware type information output by S2 into a single chopstick recognition module, and judging whether a single chopstick exists by a threshold value; step S5: sending the tableware type information output by S2 into a missing piece recognition module, and judging whether a piece is missing by combining a threshold value with a color recognition method; the beneficial effect of the present invention is that a tableware detection and recognition model is trained by deep learning technology, and the detection and recognition of missing pieces, out-of-bounds and single chopsticks of disposable tableware are performed in combination with threshold judgment and color recognition technology.

Description

Technical Field

[0001] The present invention relates to the field of target detection and recognition, and in particular to a method for detecting and recognizing disposable tableware kits based on deep learning technology. Background Art

[0002] Against the backdrop of intelligent manufacturing, AI-assisted intelligent quality inspection is becoming a key component of enterprise transformation. Intelligent quality inspection is the most mature application of AI in manufacturing. Leveraging image recognition and deep learning technologies, it addresses the pain points of traditional quality inspection, including high labor costs, the inability to perform continuous operations, limited spot checks, and low defect detection rates. This significantly improves product quality inspection efficiency and defect detection accuracy, reducing labor costs while ensuring a high rate of qualified products leaving the factory.

[0003] Specifically, quality inspection in the disposable tableware industry requires three aspects of quality inspection: First, the completeness of the tableware set must be checked to avoid inconvenience caused by the lack of necessary tableware in the disposable tableware set; second, when packaging the disposable tableware set, it is necessary to check whether the tableware is within the range of the packaging paper bag to avoid damage to the tableware due to the tableware being cut beyond the range of the paper bag. Finally, since there are defective single chopsticks, single chopsticks need to be removed. If there are missing pieces in the disposable tableware set, cuts and damage caused by out-of-bounds tableware, or single chopsticks, they need to be removed to prevent unqualified products from reaching customers and affecting their dining experience. Summary of the Invention

[0004] The purpose of this invention is to solve the above problems and design a disposable tableware kit detection and identification method based on deep learning technology.

[0005] To achieve the above-mentioned purpose, the technical solution of the present invention is a method for detecting and identifying disposable tableware sets based on deep learning technology, comprising the following steps:

[0006] Step S1: Acquire an image of the disposable tableware set to be processed;

[0007] Step S2: sending the image of the disposable tableware set to be processed to the tableware detection and recognition module for data preprocessing and tableware detection and recognition;

[0008] Step S3: sending the tableware position information output by the tableware detection and recognition module to the out-of-bounds detection module, performing tableware out-of-bounds detection by position comparison and outputting the detection results;

[0009] Step S4: sending the tableware type information output by the tableware detection and recognition module to the single chopstick recognition module, determining whether a single chopstick exists through a threshold determination method, and outputting the recognition result;

[0010] Step S5: The tableware type information output by the tableware detection and recognition module is sent to the missing piece recognition module, and the missing piece of the disposable tableware set is recognized by combining threshold determination and color recognition, and the recognition result is output.

[0011] In step S1, the disposable tableware set is placed in a stable light environment, and an image of the disposable tableware set to be processed is generated by photographing with a camera.

[0012] The tableware detection and recognition module in step S2 includes two functions: data preprocessing and tableware detection and recognition.

[0013] The data preprocessing includes adjusting the brightness and contrast of the image to be detected and identified so that its features are clearer and more conducive to detection and identification, and then scaling the image after adjusting the brightness and contrast so that its size meets the detection and identification requirements.

[0014] The tableware detection and recognition: the image to be detected and recognized after data preprocessing is sent to the tableware detection and recognition model, and the position information and type information of the detected and recognized tableware are output.

[0015] Before the tableware detection and recognition model performs tableware position information detection and type information recognition, the method further includes:

[0016] Deep learning tableware detection and recognition model training: collect images of disposable tableware sets of different styles; annotate the image data, and the required annotation information is a rectangular box that can represent the position of the tableware and a label that can represent the type of the rectangular box that can represent the position of the tableware; divide the annotated image data into a training set, a validation set, and a test set; use the divided training set and validation set to train the tableware detection and recognition model, input the test set of the disposable tableware set images to be detected into the tableware detection and recognition model to obtain the position information of the detected tableware and the type information of the corresponding position, perform model evaluation, and finally obtain a deep learning tableware detection and recognition model.

[0017] The out-of-bounds detection module in step S3 compares the tableware position information output by the tableware detection and identification module with the specified tableware range. If the tableware position exceeds the specified tableware range, it is determined that the tableware has crossed the boundary; if the tableware position is within the specified tableware range, it is determined that the tableware has not crossed the boundary; when all the tableware detected has not crossed the boundary, the out-of-bounds detection module outputs a no-out-of-bounds result; when one or more tableware has crossed the boundary, the out-of-bounds detection module outputs a yes-out-of-bounds result.

[0018] The single chopstick threshold determination method in step S4 is as follows: a single chopstick determination threshold is set, and the tableware type confidence information finally output by the tableware detection and recognition module is post-processed, that is, the tableware recognition score information is extracted. When the recognition score of a single chopstick is greater than the previously set single chopstick determination threshold, the single chopstick recognition module outputs a result indicating the presence of a single chopstick; otherwise, the single chopstick recognition module outputs a result indicating the absence of a single chopstick.

[0019] The tableware threshold determination method in step S5 includes: setting a threshold as a determination condition for the tableware recognition result; post-processing the tableware type confidence information output by the tableware detection and recognition module, i.e., extracting the tableware type score information; setting a determination threshold for the tableware type score; when the tableware type score is greater than the corresponding tableware determination threshold, determining that the tableware type is present; otherwise, determining that the tableware type is not present;

[0020] The color recognition method in step S5 includes: performing HSV color transformation on the disposable tableware set image obtained in step S1 to obtain a color-transformed image; setting an HSV color range and a pixel number threshold for each type of tableware; and determining that tableware of that type exists in the color-transformed image if there are pixels within the tableware color range and the number of pixels within the color range is greater than the pixel number threshold for that type of tableware; otherwise, determining that tableware of that type does not exist.

[0021] The results returned by the tableware threshold determination method and the color recognition method are comprehensively used to determine whether the corresponding type of tableware exists. When any one of the two methods determines that the corresponding type of tableware exists, it is finally determined that the corresponding type of tableware exists. When both methods determine that the corresponding type of tableware does not exist, it is determined that the corresponding type of tableware does not exist. When any one of the three types of tableware, namely, a spoon, a pair of chopsticks, and a paper towel, is missing in the tableware picture, the tableware missing piece identification module outputs a recognition result that the missing piece exists; otherwise, it outputs a recognition result that the missing piece does not exist.

[0022] Utilizing the technical solutions of the present invention, a method for detecting and identifying disposable tableware sets based on deep learning technology is developed. This method employs a small sample learning approach to train a tableware detection and recognition model through a deep learning algorithm. This method combines threshold determination with color recognition technology to detect and identify missing pieces, out-of-bounds items, and single chopsticks in disposable tableware. The use of machine recognition methods allows for a close integration of quality inspection with the production process, eliminating the possibility of missed inspections during manual spot checks and ensuring a high yield of disposable tableware sets. This integration of these methods addresses existing problems in this specific field. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for detecting and identifying disposable tableware sets based on deep learning technology according to the present invention; DETAILED DESCRIPTION

[0024] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, a method for detecting and identifying disposable tableware sets based on deep learning technology includes the following steps:

[0025] S1: Acquire an image of a disposable tableware set of a product to be processed;

[0026] It should be noted that the disposable tableware set includes items such as chopsticks, spoons, paper towels and toothpicks, which can be of variable quantity, materials and positions;

[0027] S1.1: Place the disposable tableware set of the product to be processed in a stable light environment;

[0028] It should be noted that: since the actual production line environment is relatively open, in order to avoid the influence of natural light on the image characteristics of the tableware set, a stable light environment is specially designed to obtain the tableware set image;

[0029] S1.2: Generate an image of the disposable tableware set of the product to be processed by capturing it with an industrial camera;

[0030] S2: Send the disposable tableware set image of the product to be processed to the tableware detection and recognition module for data preprocessing and tableware detection and recognition;

[0031] S2.1: The tableware detection and recognition module covers two functions: data preprocessing and tableware detection and recognition;

[0032] S2.2: Data preprocessing in S2.1: adjusting the brightness and contrast of the image of the disposable tableware set of the product to be processed, then performing feature lossless scaling on the image after adjusting the brightness and contrast, and preprocessing the scaled image. The preprocessing method is binarization, noise reduction and other operations, and finally obtaining an image with clearer features and an image size that is conducive to visual processing, so that it meets the detection and recognition requirements.

[0033] S2.3: Tableware detection and recognition in S2.1: feeding the pre-processed image of the disposable tableware set into the tableware detection and recognition model, and outputting the position and type information of the detected tableware;

[0034] Furthermore, before the tableware detection and recognition model performs tableware position information detection and type information recognition, the method further includes:

[0035] Training of the deep learning tableware detection and recognition model: The training of the detection model is mainly achieved by using few-shot learning and transfer learning methods. Collect images of disposable tableware sets of products to be processed in different styles; annotate the image data, and the required annotation information is a rectangular box that can represent the position of the tableware and a label that can represent the type of the rectangular box of the tableware position; divide the annotated image data into a training set, a validation set and a test set; use the divided training set and validation set to train the tableware detection and recognition model, input the test set of the disposable tableware set images of the products to be processed into the tableware detection and recognition model to obtain the position information of the tableware and the type information of the corresponding position, conduct model evaluation, and finally obtain the deep learning tableware detection and recognition model;

[0036] S3: Send the tableware position information output by the tableware detection and recognition module into the out-of-bounds detection module, and perform tableware out-of-bounds detection by means of position comparison and output the detection result;

[0037] S3.1: The out-of-bounds detection module in S3: Compare the tableware position information output by the tableware detection and recognition module with the specified tableware range, where the starting point information of the detected tableware position is D1 and the ending point information is D2; the starting point information of the specified tableware range threshold is T1, and the ending point information is T2. If the position of a certain tableware exceeds the specified tableware range, that is, when D1 < T1 or D2 > T2, it is determined that the tableware is out of bounds; if the tableware position is within the specified tableware range, that is, when T1 <= D1 < D2 <= T2, it is determined that the tableware is not out of bounds; when all the detected tableware are not out of bounds, the out-of-bounds detection module determines that there is no out-of-bounds situation in the disposable tableware set of this product, and when there is one or more tableware out of bounds, it is determined that there is an out-of-bounds situation in the disposable tableware set of this product;

[0038] S4: Send the tableware type information output by the tableware detection and recognition module into the single chopstick recognition module, and determine whether there is a single chopstick by means of threshold determination and output the recognition result;

[0039] S4.1: The single chopstick threshold determination method in S4: Set the determination threshold of the single chopstick as T, and post-process the confidence information of the tableware type finally output by the tableware detection and recognition module, that is, extract the recognition score information of the tableware and represent it as S. When the recognition score of the single chopstick is greater than the previously set determination threshold of the single chopstick, that is, when S >= T, the single chopstick recognition module determines that there is a single chopstick in the disposable tableware set of this product, otherwise it is determined that there is no single chopstick in the disposable tableware set of this product;

[0040] S5: Send the tableware type information output by the tableware detection and recognition module into the missing part recognition module, and perform missing part recognition of the disposable tableware set by combining threshold determination and color recognition, and output the recognition result;

[0041] S5.1: Tableware threshold determination method in S5: setting a threshold as a determination condition for tableware recognition results; post-processing the tableware type confidence information output by the tableware detection and recognition module, i.e., extracting tableware type score information, denoted as C; setting a determination threshold for the tableware type score to Q; when the tableware type score is greater than the corresponding tableware determination threshold, i.e., C>=Q, determining that tableware of that type is present; otherwise, determining that tableware of that type is not present;

[0042] S5.2: The color recognition method in S5 includes performing an HSV color transform on the image of the disposable tableware set of the product obtained in S1 to obtain a color-transformed image; setting an HSV color range and a pixel count threshold for each type of tableware; if, in the color-transformed image, there are pixels within the color range of a certain type of tableware, and the number of pixels within the color range is greater than the pixel count threshold for the type of tableware, then the type of tableware is determined to be present; otherwise, the type of tableware is determined not to be present;

[0043] S5.3: Combine the results returned by the two methods described in S5.1 and S5.2 to comprehensively determine whether the corresponding type of tableware exists. In the missing part identification module, first input the output result of the tableware detection and identification module into the method of S5.1 to perform missing part determination. When the method of S5.1 determines that all the components in the tableware set are present, then the tableware set is determined to have no missing parts and is a usable qualified product; when the method of S5.1 determines that there are missing parts in the tableware set, the method of S5.2 is used for a second determination. When the method of S5.2 determines that all the components in the tableware set are present, then the tableware set is determined to have no missing parts and is a usable qualified product; otherwise, the tableware set is determined to have missing parts and is an unusable waste product.

[0044] S5.4: When any of the three types of tableware, namely, a spoon, a pair of chopsticks, and a paper towel, is missing from the image of the disposable tableware set of a product, the tableware missing piece identification module determines that there is a missing piece in the disposable tableware set of the product and the product is an unusable waste product. Otherwise, it determines that there is no missing piece in the disposable tableware set of the product and the product is a usable qualified product.

[0045] The above technical solutions only reflect the preferred technical solutions of the technical solutions of the present invention. Any changes that may be made to certain parts thereof by those skilled in the art all reflect the principles of the present invention and fall within the scope of protection of the present invention.

Claims

1. A method for detecting and identifying disposable tableware sets based on deep learning technology, characterized in that: The following steps are involved: Step S1: Acquire an image of the disposable tableware set to be processed; Step S2: sending the image of the disposable tableware set to be processed to the tableware detection and recognition module for data preprocessing and tableware detection and recognition; Step S3: sending the tableware position information output by the tableware detection and recognition module to the out-of-bounds detection module, performing tableware out-of-bounds detection by position comparison and outputting the detection results; Step S4: sending the tableware type information output by the tableware detection and recognition module to the single chopstick recognition module, determining whether a single chopstick exists through a threshold determination method, and outputting the recognition result; Step S5: The tableware type information output by the tableware detection and recognition module is sent to the missing piece recognition module, and the missing piece of the disposable tableware set is recognized by combining threshold determination and color recognition, and the recognition result is output.

2. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: In step S1, the disposable tableware set is placed in a stable light environment, and an image of the disposable tableware set to be processed is generated by photographing with a camera.

3. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: The tableware detection and recognition module in step S2 includes two functions: data preprocessing and tableware detection and recognition.

4. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 3, characterized in that: The data preprocessing includes adjusting the brightness and contrast of the image to be detected and identified so that its features are clearer and more conducive to detection and identification, and then scaling the image after adjusting the brightness and contrast so that its size meets the detection and identification requirements.

5. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 3, characterized in that: The tableware detection and recognition: the image to be detected and recognized after data preprocessing is sent to the tableware detection and recognition model, and the position information and type information of the detected and recognized tableware are output.

6. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: Before the tableware detection and recognition model performs tableware position information detection and type information recognition, the method further includes: Deep learning tableware detection and recognition model training: collect images of disposable tableware sets of different styles; annotate the image data, and the required annotation information is a rectangular box that can represent the position of the tableware and a label that can represent the type of the rectangular box that can represent the position of the tableware; divide the annotated image data into a training set, a validation set, and a test set; use the divided training set and validation set to train the tableware detection and recognition model, input the test set of the disposable tableware set images to be detected into the tableware detection and recognition model to obtain the position information of the detected tableware and the type information of the corresponding position, perform model evaluation, and finally obtain a deep learning tableware detection and recognition model.

7. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: The out-of-bounds detection module in step S3 compares the tableware position information output by the tableware detection and identification module with the specified tableware range. If the tableware position exceeds the specified tableware range, it is determined that the tableware has crossed the boundary; if the tableware position is within the specified tableware range, it is determined that the tableware has not crossed the boundary; when all the tableware detected has not crossed the boundary, the out-of-bounds detection module outputs a no-out-of-bounds result; when one or more tableware has crossed the boundary, the out-of-bounds detection module outputs a yes-out-of-bounds result.

8. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: The single chopstick threshold determination method in step S4 includes setting a single chopstick determination threshold, performing post-processing on the tableware type confidence information finally output by the tableware detection and recognition module, i.e., extracting the tableware recognition score information; when the single chopstick recognition score is greater than the previously set single chopstick determination threshold, the single chopstick recognition module outputs a result indicating the presence of a single chopstick; otherwise, the single chopstick recognition module outputs a result indicating the absence of a single chopstick; The tableware threshold determination method in step S5 is as follows: setting the threshold as the determination condition for the tableware recognition result; The tableware type confidence information output by the tableware detection and recognition module is post-processed, that is, the tableware type score information is extracted; a judgment threshold for the tableware type score is set. When the tableware type score is greater than the judgment threshold for the corresponding tableware, it is determined that this type of tableware exists; otherwise, it is determined that this type of tableware does not exist.

9. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: The color recognition method in step S5 includes: performing HSV color transformation on the disposable tableware set image obtained in step S1 to obtain a color-transformed image; setting an HSV color range and a pixel number threshold for each type of tableware; and determining that tableware of that type exists in the color-transformed image if there are pixels within the tableware color range and the number of pixels within the color range is greater than the pixel number threshold for that type of tableware; otherwise, determining that tableware of that type does not exist.

10. The method for detecting and identifying disposable tableware sets based on deep learning technology according to claim 1, characterized in that: In step S5, the results returned by the tableware threshold determination method and the color recognition method are comprehensively used to determine whether the corresponding type of tableware exists. When any one of the two methods determines that the corresponding type of tableware exists, it is finally determined that the corresponding type of tableware exists. When both methods determine that the corresponding type of tableware does not exist, it is determined that the corresponding type of tableware does not exist. When any one of the three types of tableware, namely, a spoon, a pair of chopsticks, and a paper towel, is missing in the tableware picture, the tableware missing piece recognition module outputs a recognition result that the missing piece exists; otherwise, it outputs a recognition result that the missing piece does not exist.

Images