An intelligent hospitality method based on time-series image analysis

Abstract

The present invention discloses an intelligent guest reception method based on time-series image analysis, comprising: obtaining current image data and cropping video images, and cropping out a table area image, sending the table area image into a feature extraction network to obtain four table corner feature points, calculating a perspective transformation matrix, using the perspective transformation matrix to calculate the table area image after perspective transformation, and then sending it into an image segmentation network, and then calculating the proportion of the exposed table surface in the table area based on the segmented result. The present invention adopts target detection methods and image segmentation methods and other technologies, and through automatic monitoring and analysis, determines the proportion of the exposed table surface in the table area and the number of times the proportion is greater than a threshold, determines the location information of customers and service personnel and the occupancy of the table, and combines the time-series information to derive the current guest reception status, manage the restaurant more efficiently, and improve the utilization rate of the table, greatly reducing the labor costs brought about by the restaurant's increased turnover rate and supervision of cleaning behaviors.

Description

Technical Field

[0001] The present invention relates to the technical field of catering services, and in particular to an intelligent guest reception method based on time-series image analysis. Background Art

[0002] The catering industry is a labor-intensive industry. How to improve the work efficiency of service staff and save costs is the primary issue for current catering owners.

[0003] At present, in order to improve the table turnover rate, restaurants will calculate the consumption data of each table recorded by the ordering system. The simple POS consumption data detection method does not take into account the time series changes. Customer dining is a continuous and changing process. From the moment the customer sits down, to the time the food is served and the customer leaves after eating, to the time the waiter comes to clean the table, it is a dynamic and continuous process. There are cases where the same table of guests places orders multiple times or pays at the same time as they place an order, which makes the ordering system unable to count the actual number of table turnovers. In addition, the target detection method is a qualitative method rather than a quantitative method. Target detection is mainly sensitive to features and is not sensitive to regional growth changes unless the quantitative change reaches a qualitative change. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent hospitality method based on time-series image analysis to solve the problem that the existing technology mentioned in the above background technology cannot accurately monitor and provide feedback on the dynamic and continuous dining process.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solution: an intelligent hospitality method based on time-series image analysis, comprising the following steps:

[0006] S1. Obtain restaurant image data I at the current moment t , wherein the video image includes all target detection dining tables;

[0007] S2. Crop the video image and retain the dining area image Roi t ;

[0008] S3, from the dining area image Roi t Cut out the table area image Table t ;

[0009] S4. Scale the remaining image of the dining area to the specified size and send it to the detection network to obtain the detection results.

[0010] S5. Table area image t Send it to the feature extraction network to get four table corner feature points: Corner t={(cx1, cy1), (cx2, cy2), (cx3, cy3), (cx4, cy4)};

[0011] S6. Calculate the perspective transformation matrix based on the feature point coordinates and alignment coordinates in step S5:

[0012] A t =Perspective(Corner t , Align);

[0013] S7, the perspective transformation matrix A obtained according to step S6 t Calculate the table area image PerspectiveTable after perspective transformation t ;

[0014] S8, the PerspectiveTable obtained in step S7 t Send it to the image segmentation network to obtain the segmentation result, and then calculate the proportion of the exposed table surface in the table area based on the segmentation result;

[0015] S9, determining whether the proportion obtained in step S8 is greater than a set threshold, and counting the number of times the proportion is greater than the threshold;

[0016] S10. Determine the current reception status of the table area, where the table area status includes: ready to receive guests, already receiving guests, ready to be cleared, and being cleared.

[0017] Preferably, the method for calculating the proportion of the exposed table surface in the table area in step S8 is to divide the number of pixels of the table surface in the image by the number of pixels in the entire area. The specific formula is as follows:

[0018]

[0019] Preferably, when t>100, the 100 test results before the current time of the restaurant are counted. The number of times there are guests, the number of times there are waiters, and the percentage of table exposure The number of times the proportion is greater than 0.5 and 0.8.

[0020] Preferably, in step S4, the image of the remaining portion of the dining area is scaled to 640×640.

[0021] Preferably, if no guests and no waiters appear in the detection results and the table exposure ratio is greater than 0.8 for more than 90 times, it is considered that the table is in a state where it can receive guests, and the current state is changed to being able to receive guests; if the number of guests appearing in the detection results is greater than 90 times, it is considered that the area is currently in a state where it can receive guests, and the current state is changed to being able to receive guests.

[0022] Preferably, if the guest appears less than 10 times in the detection results, and the number of times the table ratio is greater than 0.5 is 90, and the POS data shows that the bill has been paid, it is considered that the guest has left and the current status is changed to table clearing; if the guest appears less than 10 times in the detection results, and the service staff appears more than 90 times, and the number of times the table ratio is greater than 0.5 is 90, it is considered that the waiter has started to clear the table, and the status is changed to clearing the table.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention adopts target detection method, image segmentation method and other technologies, and through automatic monitoring and analysis, determines the proportion of exposed table surface in the table area and the number of times the proportion is greater than a threshold, determines the location information of customers and service personnel and the occupancy of tables, and combines time series information to obtain the current status of reception, so as to manage the restaurant more efficiently, improve the user's dining experience, and improve the utilization rate of the table, which greatly reduces the restaurant's labor costs caused by increasing the turnover rate and supervising cleaning activities. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a structural diagram of an intelligent guest reception method based on time-series image analysis according to the present invention. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0027] See also Figure 1 The present invention provides a technical solution: an intelligent hospitality method based on time-series image analysis, comprising the following steps:

[0028] S1. Obtain restaurant image data I at the current moment t , wherein the video image includes all target detection tables, and the video image in the restaurant is obtained by using a video acquisition and monitoring system in the restaurant, the video acquisition system includes one or more cameras set at the top of the restaurant or other suitable locations, the cameras are used to look down at the dining tables and transmit the captured video image data to the processing computer;

[0029] S2. Crop the video image and retain the dining area image Roi t ,In this step, the video image takes the dining area as the target object and ,is cropped by the video processing unit in the processing computer;

[0030] S3, from the dining area image Roi t Cut out the table area image Table t In this step, the video image processed in step S2 uses the table area as the target object and is cropped by the video processing unit in the processing computer;

[0031] S4. Scale the remaining image of the dining area to 640×640 and send it to the detection network to obtain the detection results.

[0032] S5, the table area image Table in step S3 t The image is fed into the feature extraction network, and the coordinate image extraction algorithm obtains four feature points at the four corners of the table:

[0033] Corner t ={(cx1,cy1),(cx2,cy2),(cx3,cy3),(cx4,cy4)};

[0034] S6. Calculate the perspective transformation matrix based on the feature point coordinates and alignment coordinates in step S5:

[0035] A t =Perspective(Corner t ,Align);

[0036] S7, the perspective transformation matrix A obtained according to step S6 t Calculate the table area image PerspectiveTable after perspective transformation t ;

[0037] S8, the PerspectiveTable obtained in step S7 t The image is sent to the image segmentation network. Image segmentation is one of the effective means to simplify machine vision algorithms. It divides the image into some meaningful areas so that the feature extraction process can extract the target features based on these areas. The basis of image segmentation is the similarity and jump of pixels, such as grayscale, texture similarity or mutation. The areas obtained by the image segmentation process generally do not overlap with each other. Certain characteristics within each area are the same or similar, while the characteristics between different areas are significantly different. This can obtain more accurate table area recognition results. The segmented results are then used to calculate the proportion of the exposed table surface in the table area. The calculation method is to divide the number of pixels of the statistical table surface in the image by the number of pixels in the entire area. The specific formula is as follows:

[0038]

[0039] S9: Determine whether the proportion obtained in step S8 is greater than the set threshold, and count the number of times the proportion is greater than the threshold: when t>100, count the 100 test results before the current time of the restaurant The number of times there are guests, the number of times there are waiters, and the percentage of table exposure The number of times the proportion is greater than 0.5 and 0.8;

[0040] S10, determining the current reception status of the table area, wherein the table area status includes: ready to receive guests, already receiving guests, ready to be cleared, and being cleared;

[0041] If there are no guests or waiters in the detection results and the table exposure ratio is greater than 0.8 for more than 90 times, the table is considered to be in a state of serving guests and the current state is changed to serving guests;

[0042] If the number of guests appearing in the detection results is greater than 90, the area is considered to be in a guest-ready state and the current state is changed to guest-ready;

[0043] If the guest appears less than 10 times in the detection results, and the table ratio is greater than 0.5 for 90 times, and the POS data shows that the order has been paid, then it is considered that the guest has left, and the current status is changed to clearable, and the POS system is connected to the processing computer;

[0044] If the number of guests appearing in the test results is less than 10, and the number of waiters appearing is greater than 90, and the number of times the table ratio is greater than 0.5 is 90, then it is considered that the waiter has started to clear the table, and the status changes to clearing the table;

[0045] The present invention adopts target detection method, image segmentation method and other technologies, and through automatic monitoring and analysis, determines the proportion of exposed table surface in the table area and the number of times the proportion is greater than a threshold, determines the location information of customers and service personnel and the occupancy of tables, and combines time series information to derive the current status of reception, so as to manage the restaurant more efficiently, improve the user dining experience, and greatly reduce the restaurant's labor costs caused by increasing the turnover rate and supervising cleaning activities.

[0046] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent hospitality method based on time-series image analysis, characterized in that: The following steps are involved: S1. Get the restaurant's current image data , wherein the video image includes all target detection dining tables; S2. Crop the video image and keep the dining area image ; S3. From the dining area image Cut out the table area image ; S4. Scale the remaining image of the dining area to the specified size and send it to the detection network to obtain the detection results. ; S5, the table area image Send it to the feature extraction network to get four table corner feature points: ； S6. Calculate the perspective transformation matrix based on the feature point coordinates and alignment coordinates in step S5: ； S7, the perspective transformation matrix obtained according to step S6 Calculate the table area image after perspective transformation ; S8, the step S7 obtained Send it to the image segmentation network to obtain the segmentation result, and then calculate the proportion of the exposed table surface in the table area based on the segmentation result; S9, determining whether the proportion obtained in step S8 is greater than a set threshold, and counting the number of times the proportion is greater than the threshold; S10, determine the current table area reception status, the table area status includes: ready to receive guests, already receiving guests, ready to clear the table and clearing the table, when Count the 100 test results before the current time of the restaurant The number of times there are guests, the number of times there are waiters, and the percentage of table exposure If the number of times the proportion of guests is greater than 0.5 and 0.8 in the detection results, if there is no guest and no waiter in the test results and the number of times the table exposure proportion is greater than 0.8 is greater than 90 times, it is considered that the table is in a state where it can receive guests, and the current state is changed to ready to receive guests; if the number of times guests appear in the test results is greater than 90 times, it is considered that the area is currently in a state where it has received guests, and the current state is changed to ready to receive guests; if the number of times guests appear in the test results is less than 10 times, and the number of times the proportion of tables is greater than 0.5 is 90, and the POS data shows that the bill has been paid, it is considered that the guest has left, and the current state is changed to ready to clear the table; if the number of times guests appear in the test results is less than 10 times, and the number of times service staff appear is greater than 90 times, and the number of times the proportion of tables is greater than 0.5 is 90, it is considered that the waiter has started to clear the table, and the state is changed to clearing the table.

2. The intelligent hospitality method based on time series image analysis according to claim 1, characterized in that: The method for calculating the percentage of the exposed table surface in the table area in step S8 is to divide the number of pixels of the table surface in the image by the number of pixels in the entire area. The specific formula is as follows: 。 3. The intelligent hospitality method based on time series image analysis according to claim 2, characterized in that: In step S4, the remaining image of the dining area is scaled to 640×640.

Images