Visual non-inductive settlement system and method for dishes

Abstract

The invention relates to a dish visual non-inductive settlement system and method. The system comprises a cloud server, a dish picture library, an image feature extraction unit and a visual identification settlement module. And the visual identification settlement module comprises a dish identification unit, an image processing unit, a vector retrieval unit and a dish settlement unit. The dish recognition unit shoots dish pictures, the image processing unit preprocesses the pictures and uploads the pictures to the cloud server, the image feature extraction unit converts the pictures into feature vectors, the vector retrieval unit compares the feature vectors in a dish picture library to recognize dish information, and the dish settlement unit calculates the total price according to the information, displays the total price and deducts fees after confirmation. According to the method, the settlement process of the dishes can be automatically completed by identifying the dishes, intervention of waiters is not needed, non-inductive settlement is achieved, the settlement speed is increased, the settlement time is shortened, the overall dining efficiency of a restaurant is improved, and the smoothness of a dining assembly line is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of dish settlement, and specifically relates to a visually imperceptible dish settlement system and method. Background Art

[0002] With the continuous development of the catering industry and the advancement of intelligent technology, customers' requirements for dining experience are constantly improving. The traditional ordering and payment methods are inefficient, have long waiting times, are prone to errors, and the checkout process requires a lot of manual participation, including the waiter's ordering and payment operations.

[0003] In the related technology, although similar contactless settlement technology for dishes has been introduced, recognition errors are prone to occur during actual use, or some technical solutions are only used to identify the type of bowl containing the dishes to settle the price, which will undoubtedly limit the flexibility and contactless experience in actual use. Summary of the invention

[0004] The purpose of the present invention is to provide a visual and non-contact settlement system and method for dishes to solve the problems in the prior art.

[0005] To this end, the present invention provides a visual non-sensing settlement system and method for dishes, comprising:

[0006] A visual non-sensing settlement system for dishes, comprising: a cloud server;

[0007] The cloud server includes a dish image library and an image feature extraction unit;

[0008] A visual recognition settlement module, which includes a dish recognition unit, an image processing unit, a vector retrieval unit and a dish settlement unit;

[0009] The dish recognition unit is used to photograph the dish to be settled and form a dish picture, the dish picture is preprocessed by the image processing unit, the dish picture after preprocessing is uploaded to the server, the image feature extraction unit converts the picture into a key feature vector, and the vector retrieval unit performs feature comparison and recognition in the dish picture library based on the feature vector to obtain dish information and send it to the dish settlement unit;

[0010] The dish settlement unit automatically calculates the total price of the dish according to the dish information, displays it on the screen, and deducts the fee after confirmation.

[0011] As a further description of the above technical solution, the dish identification and settlement module also includes a plate target detection unit, and the plate target detection unit uses a YoloV8 detector to locate the position and boundary box of the plate in real time.

[0012] As a further description of the above technical solution, the image feature extraction unit converts the image into a vector and uses a convolutional neural network to extract key feature vectors through the dish image information.

[0013] As a further description of the above technical solution, the dish recognition unit is a camera.

[0014] As a further description of the above technical solution, the vector retrieval unit constructs a vector index to retrieve similar images in the dish image library.

[0015] As a further description of the above technical solution, the dish recognition unit collects video streams in real time, and locates the plate and the dish by cooperating with the plate target detection unit.

[0016] As a further description of the above technical solution, the dish image information includes but is not limited to color, shape and texture.

[0017] As a further description of the above technical solution, the method for extracting key feature vectors through dish picture information includes extracting key feature vectors through color features, shape features, texture features of the picture, through multiple convolutional layers, activation functions and pooling layers, and encoding the visual content of the dish picture into a feature vector.

[0018] The present invention also provides a visually imperceptible settlement method for dishes, comprising:

[0019] Select dishes and place them on the plate;

[0020] Place your meal tray in the checkout area;

[0021] The visual recognition settlement module visually recognizes the dishes to obtain dish information and uploads it to the server, performs feature recognition on the dish data and automatically matches dishes and prices;

[0022] The price is automatically calculated to complete the settlement.

[0023] As a further description of the above technical solution, after automatically matching dishes and prices, the visual recognition calculation module stores the current dish image information in the dish image library in the cloud server.

[0024] Beneficial effects:

[0025] 1. The present invention provides a visual non-contact settlement system and method for dishes, which can automatically complete the settlement process of dishes by identifying dishes, without the intervention of waiters, realize non-contact settlement, speed up settlement, reduce settlement time, improve the overall dining efficiency of the restaurant, and increase the fluency of the dining assembly line. Ensure the accuracy and reliability of settlement, avoid erroneous settlement caused by human factors, and improve the accuracy and trust of the settlement process. Enhance data statistics and analysis capabilities, record dish sales data in real time, provide restaurants with accurate sales statistics and analysis reports, and help restaurant managers better understand customer consumption habits and dish sales. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 This is a schematic diagram of the visual and seamless settlement system for dishes provided by the present invention.

[0028] Figure 2 This is a schematic diagram of the visual and seamless settlement method for dishes provided by the present invention.

[0029] In the figure: 1. cloud server; 101. food image library; 102. image feature extraction unit; 2. visual recognition settlement module; 201. food recognition unit; 202. image processing unit; 203. vector retrieval unit; 204. food settlement unit. DETAILED DESCRIPTION

[0030] The content of the present invention can be more easily understood by selecting the following detailed description of the preferred implementation method of the present invention and the embodiments included. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. When there is a conflict, the definition in this specification shall prevail.

[0031] The present invention provides a visual and non-sensing settlement system and method for dishes, which solves the problem that manual settlement in the prior art is not only inefficient but also may result in settlement errors. The present invention uses a hardware and software integrated data acquisition device to collect video streams in real time through a camera on the top of the settlement area, and uses a dynamic target detector to locate plates and dishes. The ResNet model is used to pre-train the model for a data set of massive dish images, extract features such as texture, shape, and color of the image, and obtain the desired feature vector. Finally, Faiss (a neighbor search library developed by Face Book for efficient use) is used to complete feature database comparison and recognition, quickly identify the dish information placed by the consumer in the settlement area, accurately match the dishes, and automatically calculate the price to complete the settlement. It can effectively improve the accuracy of visual recognition, improve the efficiency of settlement, and reduce the error rate.

[0032] like Figure 1 As shown, a visual non-sensing settlement system for dishes includes: a cloud server 1;

[0033] The cloud server 1 includes a dish image library 101 and an image feature extraction unit 102; wherein the dish image library 101 is used to store and enter target dish images and corresponding dish prices, and can classify and maintain dishes. The image feature extraction unit 102 is used to convert images into vectors, using convolutional neural networks in deep learning, such as a pre-trained ResNet model, to extract key feature vectors through multiple convolutional layers, activation functions and pooling layers based on features such as color, shape and texture of the image, and encode the visual content of the image into feature vectors, which can be used for subsequent image retrieval. The above feature vectors can capture important features of the image.

[0034] It also includes a visual recognition settlement module 2, which includes a dish recognition unit 201, an image processing unit 202, a vector retrieval unit 203 and a dish settlement unit 204; wherein the dish recognition unit 201 is a camera, and the dinner plate is placed in a designated recognition area, and the camera above the recognition area will automatically capture the image of the dishes on the dinner plate, and quickly detect the target through the dish recognition technology to identify the type and quantity of the dishes on the dinner plate.

[0035] The image processing unit 202 performs preprocessing operations such as brightness adjustment, contrast adjustment, background noise removal, and normalization on the food pictures taken by the camera, and uploads the preprocessed pictures to the cloud server.

[0036] The vector retrieval unit 203 constructs a vector index to quickly retrieve similar images. In some embodiments, for example, the IVF (Inverted File Structure) algorithm of Faiss is used to create an index in the vector library, and the index is used to perform similarity retrieval, and feature database comparison and recognition are completed, so that the information of the dishes placed in the checkout area by the consumer can be quickly identified, and the dishes can be accurately matched while maintaining a high recall rate of the dishes.

[0037] The dish settlement unit 204, after identifying the type and quantity of each dish, will automatically calculate the total price of the plate according to the preset price information, generate a bill and display it on the screen. After the consumer confirms that it is correct, the deduction will be completed, and the entire settlement process will be easily completed.

[0038] Specifically, the entire recognition process is that the dish recognition unit 201 takes a picture of the dish to be settled and forms a dish picture, the image processing unit 202 pre-processes the dish picture, and the pre-processed dish picture is uploaded to the server, the image feature extraction unit 102 converts the picture into a key feature vector, and the vector retrieval unit 203 performs feature comparison and recognition in the dish picture library 101 based on the feature vector to obtain dish information and send it to the dish settlement unit 204;

[0039] The dish settlement unit 204 automatically calculates the total price of the dish according to the dish information, displays it on the screen, and deducts the fee after confirmation.

[0040] Through the above technical solution, non-contact settlement can be achieved, which effectively saves manpower and reduces the occurrence of settlement errors. At the same time, through the cooperation of multiple algorithms and vector retrieval algorithms, visual accuracy can be effectively improved.

[0041] Optionally, the dish identification and settlement module also includes a plate target detection unit, which uses a YoloV8 detector to locate the position and bounding box of the plate in real time. Even on edge devices without NPU / GPU, real-time video stream detection can be performed to achieve accurate detection of plate targets.

[0042] Optionally, for distance recognition of similar vector retrieval, in n-dimensional space, for two vectors and The distance can be expressed using the following formula:

[0043]

[0044] For example, in a two-dimensional plane, the distance between points (1,1) and (4,5) is:

[0045]

[0046] In summary, through the automatic dish recognition technology and payment management function, users only need to place the dishes on the recognition device, and the system can automatically recognize and generate a bill. It also supports multiple payment methods, such as Alipay, WeChat Pay, etc. This intelligent and contactless checkout method improves the efficiency of the restaurant and reduces waiting time.

[0047] refer to Figure 2 The present invention also provides a visual non-sensing settlement method for dishes, comprising:

[0048] Select dishes and place them on the plate;

[0049] Place your meal tray in the checkout area;

[0050] The visual recognition settlement module 2 performs visual recognition on the dishes to obtain the dish information and uploads it to the server, performs feature recognition on the dish data and automatically matches the dishes and prices;

[0051] The price is automatically calculated to complete the settlement.

[0052] Optionally, after automatically matching dishes and prices, the visual recognition calculation module stores the current dish image information in the dish image library 101 in the cloud server 1, so that the current dish image can be used as one of the data sources so that the entire system can be trained and learned, which can effectively improve the accuracy and speed of dish recognition. In some embodiments, deep learning can automatically identify and distinguish various dishes, and even identify different cooking methods and plating styles.

[0053] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A visual non-sensing settlement system for dishes, characterized in that: Including cloud servers; The cloud server includes a dish image library and an image feature extraction unit; A visual recognition settlement module, which includes a dish recognition unit, an image processing unit, a vector retrieval unit and a dish settlement unit; The dish recognition unit is used to photograph the dish to be settled and form a dish picture, the dish picture is preprocessed by the image processing unit, and the preprocessed dish picture is uploaded to the server, the image feature extraction unit converts the picture into a key feature vector, and the vector retrieval unit performs feature comparison and recognition in the dish picture library based on the feature vector to obtain dish information and send it to the dish settlement unit; The dish settlement unit automatically calculates the total price of the dish according to the dish information, displays it on the screen, and deducts the fee after confirmation.

2. The visual non-sensing settlement system for dishes according to claim 1 is characterized in that: The dish identification and settlement module also includes a plate target detection unit, which uses a YoloV8 detector to locate the position and boundary box of the plate in real time.

3. The visual non-sensing settlement system for dishes according to claim 1 is characterized in that: The image feature extraction unit converts the image into a vector and uses a convolutional neural network to extract key feature vectors through the dish image information.

4. The visual non-sensing settlement system for dishes according to claim 1 is characterized in that: The dish recognition unit is a camera.

5. The visual non-sensing settlement system for dishes according to claim 1 is characterized in that: The vector retrieval unit constructs a vector index to retrieve similar images in the dish image library.

6. The visual non-sensing settlement system for dishes according to claim 2 is characterized in that: The dish recognition unit collects video streams in real time, and locates the dinner plate and dishes by cooperating with the dinner plate target detection unit.

7. The visual non-sensing settlement system for dishes according to claim 3 is characterized in that: The dish image information includes but is not limited to color, shape and texture.

8. The visual non-sensing settlement system for dishes according to claim 1 is characterized in that: The method for extracting key feature vectors through dish picture information includes extracting key feature vectors through color features, shape features, texture features of the picture, through multiple convolutional layers, activation functions and pooling layers, and encoding the visual content of the dish picture into a feature vector.

9. A visually imperceptible settlement method for dishes, characterized in that: include: Select dishes and place them on the plate; Place your meal tray in the checkout area; The visual recognition settlement module visually recognizes the dishes to obtain dish information and uploads it to the server, performs feature recognition on the dish data and automatically matches dishes and prices; The price is automatically calculated to complete the settlement.

10. The visually imperceptible settlement method for dishes according to claim 9, characterized in that: After automatically matching dishes and prices, the visual recognition calculation module stores the current dish image information in the dish image library in the cloud server.

Images