Changeless eating and drinking suppression system and visitor management device

Abstract

To suppress eating and drinking without payment.SOLUTION: In the system 1, an entrance camera EC photographs an entrance area image which is an image of an entrance area including an entrance of a restaurant, and first to sixth customer seat cameras TC-1 to TC-6 photograph customer seat area images which are images of customer seat areas including table seats installed in the restaurant. The visitor management device 10 determines, based on the entrance area image and the customer seat area image, a second customer who is a customer who has entered a table seat among first customers who are customers who have entered from the entrance, determines a third customer who is a customer who has completed ordering and serving of food and drink among the second customers, determines a fourth customer who is a customer who has left the table seat among the third customers based on the customer seat area image, and performs guidance for prompting a payment of a price of the food and drink for which the payment has not been completed among the fourth customers.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a system for preventing eating and drinking without paying and a customer management device. [Background technology]

[0002] In recent years, in order to save labor, restaurants are shifting from manually ordering food and drink and paying for it to systems. One example of ordering food and drink using a system is when customers place their orders themselves using a tablet device. Another example of paying for food and drink using a system is when customers pay for it themselves using a self-checkout register without paying cash. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-145470 Summary of the Invention [Problem to be solved by the invention]

[0004] In restaurants where food and drink bills are paid for using a system, the number of cases of customers eating and drinking without paying is increasing because it is becoming more difficult to determine whether or not a meal has been paid due to the diversification of payment methods and the decrease in opportunities for customers to interact with restaurant staff when paying.

[0005] Therefore, this disclosure proposes a technology that can prevent eating and drinking without paying. [Means for solving the problem]

[0006] The disclosed system for preventing non-payment of food and drink includes a first camera that captures a first image of a first area, which is a predetermined area including an entrance and exit of a restaurant; a second camera that captures a second image of a second area, which is a predetermined area including table seats installed within the restaurant; and a customer management device. Based on the first and second images, the customer management device identifies a second customer who is a first customer who entered the restaurant through the entrance and sat at the table, a third customer among the second customers who has ordered food and drink and had it served, and a fourth customer among the third customers who has left the table, based on the second image. The customer management device also provides a fifth customer among the fourth customers who has been served but has not yet paid for the food and drink, with a prompt to pay the fee. [Effects of the Invention]

[0007] According to the present disclosure, eating and drinking without paying can be prevented. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a system for preventing eating and drinking without paying, according to the present disclosure. [Figure 2] FIG. 1 is a diagram showing an example of table layout in a restaurant according to the present disclosure. [Figure 3] FIG. 1 shows an example of installation of an entrance / exit camera according to the present disclosure. [Figure 4] FIG. 1 shows an example of installation of a seating camera according to the present disclosure. [Figure 5] A processing flow diagram showing an example of the operation of the system for preventing eating and drinking without paying, according to the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating an example of a visitor management table according to the present disclosure. [Figure 7] FIG. 1 illustrates an example of a person list according to the present disclosure. [Figure 8] FIG. 10 is a diagram showing a display example of the display of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the following embodiments, the same components or processes will be denoted by the same reference numerals, and redundant description may be omitted.

[0010] [Example] <Configuration of the system to prevent eating and drinking without paying> FIG. 1 illustrates an exemplary configuration of a system for preventing non-payment of food and drink according to the present disclosure. In FIG. 1, the non-payment of food and drink prevention system 1 includes a customer management device 10, a POS (Point of Sales) system 40, an entrance / exit camera EC, a first customer seat camera TC-1, a second customer seat camera TC-2, a third customer seat camera TC-3, a fourth customer seat camera TC-4, a fifth customer seat camera TC-5, and a sixth customer seat camera TC-6. Hereinafter, the first customer seat camera TC-1, the second customer seat camera TC-2, the third customer seat camera TC-3, the fourth customer seat camera TC-4, the fifth customer seat camera TC-5, and the sixth customer seat camera TC-6 may be collectively referred to as the "customer seat camera TC." The customer management device 10, the POS system 40, the entrance / exit camera EC, and the customer seat camera TC are installed inside a restaurant. The entrance / exit camera EC is installed near the customer entrance / exit, and the customer seat camera TC is installed near the customer seats. The visitor management device 10 and the entrance / exit camera EC are connected via a network 50. The visitor management device 10 and the seat camera TC are also connected via the network 50. The visitor management device 10 and the POS server 41 of the POS system 40 are also connected via the network 50. An example of the network 50 is a LAN (Local Area Network).

[0011] <Configuration of visitor management device> In FIG. 1, the visitor management device 10 includes a processor 11, a communication module 12, a storage unit 13, a display 20, and a speaker 30. The communication module 12 is connected to a network 50, and the processor 11 can communicate with an entrance / exit camera EC and a seat camera TC via the network 50 using the communication module 12. Also, the processor 11 can communicate with a POS server 41 of a POS system 40 via the network 50 using the communication module 12. The storage unit 13 stores a learned model M1. The learned model M1 is generated by machine learning. An example of a device having the processor 11, the communication module 12, and the storage unit 13 is a computer. Examples of the processor 11 include a CPU (Central Processing Unit) and a DSP (Digital Signal Processor). Examples of the storage unit 13 include a storage and a memory. An example of the display 20 is an LCD (Liquid Crystal Display).

[0012] <Configuration of the POS System> In FIG. 1, the POS system 40 includes a POS server 41, a first POS terminal 42-1, a second POS terminal 42-2, and a third POS terminal 42-3. For example, the first POS terminal 42-1 is a portable tablet-type terminal. Also, for example, the second POS terminal 42-2 is installed at a manned cash register and used as a manned cash register, and the third POS terminal 42-3 is installed at a self-checkout cash register and used as a self-checkout. Hereinafter, the first POS terminal 42-1, the second POS terminal 42-2, and the third POS terminal 42-3 may be collectively referred to as "POS terminal 42". The POS server 41 and the POS terminal 42 are connected to the network 50, and the POS server 41 and the POS terminal 42 can communicate with each other via the network 50.

[0013] <Entrance / Exit Camera and Seat Camera> FIG. 2 is a diagram showing an example of the table arrangement inside a restaurant of the present disclosure. FIG. 3 is a diagram showing an example of the installation of the entrance / exit camera of the present disclosure. FIG. 4 is a diagram showing an example of the installation of the seat camera of the present disclosure.

[0014] 2, for example, a restaurant is provided with an entrance / exit ET for customers. Also provided inside the restaurant are, for example, a cash register counter RC, a first table T01, a second table T02, a third table T03, a fourth table T04, a fifth table T05, a sixth table T06, a seventh table T07, an eighth table T08, a ninth table T09, a tenth table T10, an eleventh table T11, and a twelfth table T12. The first table seat T01, the fourth table seat T04, the fifth table seat T05, the sixth table seat T06, the seventh table seat T07, the eighth table seat T08, the eleventh table seat T11, and the twelfth table seat T12 are table seats for four people, while the second table seat T02, the third table seat T03, the ninth table seat T09, and the tenth table seat T10 are table seats for two people. At least one of a second POS terminal 42-2 for manned registers and a third POS terminal 42-3 for self-checkouts is installed above the register counter RC. A display 20 and a speaker 30 are installed near the register counter RC.

[0015] In addition, the entrance / exit camera EC is installed near the entrance / exit ET, and as shown in FIG. 2, captures an image (hereinafter sometimes referred to as an "entrance / exit area image") of a predetermined area AR11 including the entrance / exit ET (hereinafter sometimes referred to as the "entrance / exit area"), and transmits the captured entrance / exit area image to the visitor management device 10. Customers of the restaurant enter and leave the restaurant by passing through the entrance / exit area AR11. The processor 11 of the visitor management device 10 receives the entrance / exit area image via the communication module 12.

[0016] In addition, the first customer seat camera TC-1 is installed near the first table seat T01 and the second table seat T02, and as shown in Figure 2, captures an image (hereinafter sometimes referred to as the "first area") of a predetermined area AR21 including the first table seat T01 and the second table seat T02 (hereinafter sometimes referred to as the "first area image"), and transmits the captured first area image to the customer management device 10.

[0017] In addition, the second customer seat camera TC-2 is installed near the third table seat T03 and the fourth table seat T04, and as shown in Figure 2, captures an image (hereinafter sometimes referred to as the "second area image") of a specified area AR22 including the third table seat T03 and the fourth table seat T04 (hereinafter sometimes referred to as the "second area"), and transmits the captured second area image to the customer management device 10.

[0018] In addition, the third customer seat camera TC-3 is installed near the fifth table seat T05 and the sixth table seat T06, and as shown in Figure 2, it captures an image (hereinafter sometimes referred to as the "third area") of a specified area AR23 including the fifth table seat T05 and the sixth table seat T06 (hereinafter sometimes referred to as the "third area image"), and transmits the captured third area image to the visitor management device 10.

[0019] In addition, the fourth customer seat camera TC-4 is installed near the seventh table seat T07 and the eighth table seat T08, and as shown in Figure 2, captures an image (hereinafter sometimes referred to as the "fourth area image") of a specified area AR24 including the seventh table seat T07 and the eighth table seat T08 (hereinafter sometimes referred to as the "fourth area"), and transmits the captured fourth area image to the visitor management device 10.

[0020] In addition, the fifth customer seat camera TC-5 is installed near the ninth table seat T09 and the tenth table seat T10, and as shown in Figure 2, captures an image (hereinafter sometimes referred to as the "fifth area image") of a specified area AR25 including the ninth table seat T09 and the tenth table seat T10 (hereinafter sometimes referred to as the "fifth area"), and transmits the captured fifth area image to the visitor management device 10.

[0021] In addition, the sixth customer seat camera TC-6 is installed near the eleventh table seat T11 and the twelfth table seat T12, and as shown in Figure 2, captures an image (hereinafter sometimes referred to as the "sixth area image") of a specified area AR26 including the eleventh table seat T11 and the twelfth table seat T12. The captured sixth area image is then transmitted to the customer management device 10.

[0022] Hereinafter, the first area image, the second area image, the third area image, the fourth area image, the fifth area image, and the sixth area image may be collectively referred to as “audience area images.” The processor 11 of the visitor management device 10 receives the audience area images via the communication module 12.

[0023] By linking the entrance / exit camera EC and each customer seat camera TC, the processor 11 tracks customers who enter the store through the entrance / exit ET within the store using the entrance / exit area image and the customer seat area image. The processor 11 tracks the same person within the store using existing algorithms such as MOT (Multiple-Object Tracking), SORT (Simple Online and Real Time Tracking), DeepSORT, TLD (Tracking Learning Detection), and KCF (Kernelized Correlation Filters).

[0024] First, as shown in FIG. 3, when a customer enters the store through the entrance / exit ET, the processor 11 recognizes the image of the customer included in the entrance / exit area image and assigns a person ID (e.g., "P031") to the recognized image of the customer.

[0025] Furthermore, as shown in FIG. 4, the processor 11 detects, based on the fifth area image, that a customer with person ID "P031" has sat at the ninth table seat T09.

[0026] <Operation of the system to prevent eating and drinking without paying> Fig. 5 is a process flow diagram showing an example of the operation of the system for preventing eating and drinking without paying according to the present disclosure. Fig. 6 is a diagram showing an example of a visitor management table according to the present disclosure. The visitor management table TB shown in Fig. 6 is stored and managed in the memory unit 13 for each business day of the restaurant.

[0027] 5, in step S100, processor 11 determines whether a customer has visited the store based on the entrance / exit area image. Processor 11 determines that a customer has visited the store when it recognizes an image of a customer entering the store in the entrance / exit area image. The processing of step S100 is repeatedly executed until it is determined that a customer has visited the store (step S100: No). When it is determined that a customer has visited the store (step S100: Yes), the processing proceeds to step S110.

[0028] In step S110, processor 11 assigns a person ID to the image of the customer recognized in step S100. Also in step S110, processor 11 registers the person ID and the time of arrival in visitor management table TB shown in Fig. 6. In the following explanation using Figs. 5 and 6, an example will be described in which processor 11 assigns a person ID of "P031" in step S110 to the image of the customer recognized in step S100.

[0029] Here, the visitor management table TB shown in Figure 6 has the following items to be registered: person ID, arrival time, table ID, seating time, ordering time, serving time, payment time, payment type, leaving time, leaving time, and tracking target flag.

[0030] In the processing from step S110 onwards, processor 11 uses the existing algorithm as described above to track the customer with person ID "P031" within the store.

[0031] Next, in step S120, processor 11 determines, based on the customer seating area image, at which of the 12 tables, from table T01 to table T12, the customer with person ID "P031" has sat. The process of step S120 is repeatedly executed until it is determined that the customer with person ID "P031" has sat down (step S120: No). When it is determined that the customer with person ID "P031" has sat down (step S120: Yes), the process proceeds to step S130.

[0032] In step S130, processor 11 associates the table ID and the seating time with person ID "P031" and registers them in visitor management table TB. Here, as an example, it is assumed that the customer with person ID "P031" sat at table ninth seat T09. Therefore, in step S130, processor 11 associates the table ID "T09" with person ID "P031" and registers it in visitor management table TB.

[0033] Next, in step S140, the processor 11 switches the display on the display 20 for the seat at table ninth T09 from an empty seat display to an occupied seat display.

[0034] Next, in step S150, the POS server 41 determines whether the food and drink order at the ninth table seat T09 has been completed on the first POS terminal 42-1 carried by the store staff member. The process of step S150 is repeatedly executed until it is determined that the food and drink order at the ninth table seat T09 has been completed (step S150: No). When it is determined that the food and drink order at the ninth table seat T09 has been completed (step S150: Yes), the process proceeds to step S160.

[0035] In step S160, the POS server 41 transmits the order information to the visitor management device 10, and the processor 11 receives the order information via the communication module 12. The order information transmitted from the POS server 41 includes the table ID of the table seat where the order was completed (here, "T09") and the time when the order was completed (hereinafter, sometimes referred to as the "order time").

[0036] Next, in step S170, processor 11 registers the order time in customer management table TB in association with table ID "T09."

[0037] Next, in step S180, the processor 11 switches the display on the display 20 for the seat at table ninth seat T09 from a seated display to an ordered display.

[0038] Next, in step S190, the POS server 41 determines whether an operation to complete the serving of food and drink to the ninth table seat T09 has been executed on the first POS terminal 42-1 carried by a store staff member. The process of step S190 is repeatedly executed until it is determined that the serving of food and drink to the ninth table seat T09 has been completed (step S190: No). When it is determined that the serving of food and drink to the ninth table seat T09 has been completed (step S190: Yes), the process proceeds to step S200.

[0039] In step S200, the POS server 41 transmits the food delivery information to the visitor management device 10, and the processor 11 receives the food delivery information via the communication module 12. The food delivery information transmitted from the POS server 41 includes the table ID of the table to which the food delivery has been completed (here, "T09") and the time when the food delivery has been completed (hereinafter, sometimes referred to as the "food delivery time").

[0040] Next, in step S210, the processor 11 registers the serving time in the guest management table TB in association with the table ID "T09."

[0041] Next, in step S220, the processor 11 switches the display 20 for the seat at table 9 T09 from an "ordered" display to a "served" display.

[0042] Next, in step S190, the POS server 41 determines whether a transaction while the customer is still at the table (hereinafter referred to as a "tabletop transaction") has been performed on the first POS terminal 42-1 carried by a store staff member. When a tabletop transaction is performed, the payment for the food and drink is completed. For example, a customer with personal ID "P031" at the ninth tabletop seat T09 can pay by code payment using their smartphone. If a tabletop transaction has been performed (step S230: Yes), the process proceeds to step S240. If a tabletop transaction has not been performed (step S230: No), the process proceeds to step S260.

[0043] In step S240, the POS server 41 transmits information indicating that the payment for the food and drink has been completed (hereinafter, sometimes referred to as "payment information") to the visitor management device 10, and the processor 11 receives the payment information via the communication module 12. The payment information transmitted from the POS server 41 includes the table ID (here, "T09") of the table seat where the payment has been completed, the time when the payment was completed (hereinafter, sometimes referred to as the "payment time"), and the payment type. The payment type is set to one of "table," "manned register," and "self-checkout." For example, when a transaction is made using the first POS terminal 42-1 carried by a store staff member, the payment type is set to "table," when a transaction is made using the second POS terminal 42-2, which is a manned register installed at the checkout counter RC, the payment type is set to "manned register," and when a transaction is made using the third POS terminal 42-3, which is a self-checkout installed at the checkout counter RC, the payment type is set to "self-checkout." Therefore, the payment type in the payment information transmitted from the POS server 41 in step S240 is set to "table."

[0044] Next, in step S250, processor 11 registers the payment time and payment type in customer management table TB in association with table ID "T09."

[0045] Meanwhile, in step S260, the POS server 41 determines whether a transaction (hereinafter sometimes referred to as a "register transaction") has been performed using the second POS terminal 42-2, which is a manned register, or the third POS terminal 42-3, which is a self-checkout register. When a register transaction is performed, the payment for the food and drink is completed. If a register transaction has been performed (step S260: Yes), the process proceeds to step S340; if a register transaction has not been performed (step S260: No), the process returns to step S230.

[0046] Furthermore, in step S270 after step S250, the processor 11 determines, based on the fifth area image, whether the customer with person ID "P031" has left the ninth table seat T09. The process of step S270 is repeatedly executed until it is determined that the customer with person ID "P031" has left the ninth table seat T09 (step S270: No). When it is determined that the customer with person ID "P031" has left the ninth table seat T09 (step S270: Yes), the process proceeds to step S280.

[0047] In step S280, the processor 11 registers the departure time in the visitor management table TB in association with the table ID "T09."

[0048] Next, in step S290, processor 11 determines whether the table-based transaction has been completed by determining whether the payment time and the payment type "table-based" have been registered in the customer management table TB in which table ID "T09" is registered. In this way, processor 11 determines whether the table-based transaction has been completed at the time the customer leaves the table (hereinafter sometimes referred to as the "first time point") (step S270: Yes). Processor 11 determines that the table-based transaction has been completed if the payment time and the payment type "table-based" have been registered in the customer management table TB in which table ID "T09" is registered. On the other hand, processor 11 determines that the table-based transaction has not been completed (i.e., the table-based transaction is incomplete) if the payment time and the payment type are not registered in the customer management table TB in which table ID "T09" is registered. If the tabletop payment has been completed (step S290: Yes), the process proceeds to steps S300, S310, and S410. If the tabletop payment has not been completed (step S290: No), the process proceeds to steps S320, S330, and S360.

[0049] In step S300, the processor 11 switches the display 20 for the ninth table seat T09 from the "Meals Served" display to the "Payment Completed" display.

[0050] Furthermore, in step S310, processor 11 causes a voice expressing gratitude for visiting the store (hereinafter, sometimes referred to as a "first voice") to be output from speaker 30. For example, processor 11 causes speaker 30 to output a voice saying "Thank you for visiting us" as the first voice.

[0051] On the other hand, in step S320, the processor 11 switches the display 20 for the ninth table seat T09 from a display indicating that the food has been served to a display indicating that the bill has not yet been paid.

[0052] Furthermore, in step S330, the processor 11 causes the speaker 30 to output a voice (hereinafter sometimes referred to as a "second voice") representing guidance urging the customer to pay the bill at the second POS terminal 42-2 (manned register) or the third POS terminal 42-3 (self-checkout) installed on the checkout counter RC. For example, the processor 11 causes the speaker 30 to output, as the second voice, a voice saying, "Please pay at the register. If the register is manned, a staff member will come, so please wait a moment." In this way, when the tabletop payment has not been completed at the first point in time (step S270: Yes) (step S290: No), the processor 11 causes the customer to pay the bill at a register installed in the restaurant.

[0053] In step S340, the POS server 41 transmits the payment information to the visitor management device 10, and the processor 11 receives the payment information via the communication module 12. The payment information transmitted from the POS server 41 includes the table ID (here, "T09") of the table seat where the payment was completed, the time of the payment, and the payment type. As described above, the payment type is set to either "table," "manned register," or "self-checkout." Therefore, the payment type in the payment information transmitted from the POS server 41 in step S340 is set to "manned register" or "self-checkout."

[0054] Next, in step S350, processor 11 registers the payment time and payment type in customer management table TB in association with table ID "T09."

[0055] Next, in step S360, processor 11 determines whether the register transaction has been completed by determining whether the payment time and the payment type "manned register" or "self-checkout" are registered in customer management table TB in which table ID "T09" is registered. In this way, processor 11 determines whether the register transaction has been completed at a point in time after the first point in time and before the customer leaves the store through entrance / exit ET (hereinafter sometimes referred to as the "second point in time"). Processor 11 determines that the register transaction has been completed when the payment time and the payment type "manned register" or "self-checkout" are registered in customer management table TB in which table ID "T09" is registered. On the other hand, processor 11 determines that the register transaction has not been completed (i.e., the register transaction is incomplete) when the payment time and the payment type are not registered in customer management table TB in which table ID "T09" is registered. If the cash register transaction has been completed (step S360: Yes), the process proceeds to steps S370, S380, and S410. If the cash register transaction has not been completed (step S360: No), the process proceeds to steps S390, S400, and S410.

[0056] In step S370, the processor 11 switches the display on the display 20 for the ninth table seat T09 from the "Meals Served" display to the "Payment Completed" display.

[0057] Furthermore, in step S380, the processor 11 causes the speaker 30 to output a first voice. For example, the processor 11 causes the speaker 30 to output a voice saying, "Thank you for visiting our store." as the first voice.

[0058] On the other hand, in step S390, the processor 11 switches the display on the display 20 for the ninth table seat T09 from the "served" display to the "unpaid" display.

[0059] Furthermore, in step S400, processor 11 causes speaker 30 to output a voice (hereinafter sometimes referred to as a "third voice") representing guidance urging the customer to pay the bill inside the restaurant. For example, processor 11 causes speaker 30 to output, as the third voice, a voice saying, "Have you finished paying? Please pay inside the restaurant." In this way, when the cash register transaction has not been completed at the second point in time (step S360: No), processor 11 provides guidance urging the customer to pay the bill inside the restaurant.

[0060] In step S410, processor 11 determines, based on the entrance / exit area image, whether the customer with person ID "P031" has left the store. The process of step S410 is repeatedly executed until it is determined that the customer with person ID "P031" has left the store (step S410: No). When it is determined that the customer with person ID "P031" has left the store (step S410: Yes), the process proceeds to step S420.

[0061] In step S420, the processor 11 registers the store exit time in the visitor management table TB in association with the person ID "P031."

[0062] Next, in step S430, the processor 11 switches the display 20 for the seat at table ninth T09 from the "paid" display or "unpaid" display to the "vacant" display.

[0063] Next, in step S440, the processor 11 registers a tracking target flag in the visitor management table TB in association with the person ID "P031." When the processor 11 determines that the customer with the person ID "P031" is not a tracking target (is not a tracking target), the processor 11 sets the tracking target flag in association with the person ID "P031" to "0," and when the processor 11 determines that the customer with the person ID "P031" is a tracking target, the processor 11 sets the tracking target flag in association with the person ID "P031" to "1." For example, in the visitor management table TB in which the person ID "P031" is registered, if both the payment time and the store exit time are registered, the processor 11 sets the tracking target flag to "0," and if the payment time is not registered but the store exit time is registered, the processor 11 sets the tracking target flag to "1."

[0064] <List of people> Fig. 7 is a diagram showing an example of a person list of the present disclosure. The person list PL shown in Fig. 7 is stored in the storage unit 13. In Fig. 7, the person list PL has, as items to be registered, the following items: person ID, registration date and time, characteristic information, warning flag, and caution flag.

[0065] For example, the processor 11 generates a person list PL for a person ID for which the tracking target flag is set to "1" in the visitor management table TB shown in Fig. 6. When the processor 11 sets the tracking target flag in the visitor management table TB to "1" in step S440 shown in Fig. 5, the processor 11 registers the person ID "P031" associated with the tracking target flag of "1" in the visitor management table TB as a person ID in the person list PL, and also registers the date and time when the tracking target flag was set to "1" in the visitor management table TB as the registration date and time of the person list PL.

[0066] Furthermore, for example, when a customer with person ID "P031" is determined to be a suspicious person using the trained model M1 based on an image of the customer included in the customer seating area image, the processor 11 may generate a person list PL having the person ID "P031." Examples of suspicious people include customers who stayed in the store for a period of time equal to or longer than a threshold, customers who slept while seated, customers who engaged in suspicious behavior, and customers who committed violent acts.

[0067] In addition, the processor 11 registers the characteristic information of the customer with the person ID "P031" in the person list PL. Examples of the characteristic information of the customer include the customer's face, the customer's gender, the customer's age, the customer's height, the customer's body type, the customer's clothing, the customer's belongings, etc.

[0068] Furthermore, the processor 11 registers a warning flag and a caution flag in the person list PL. For example, the processor 11 sets a warning flag associated with a person ID for which a tracking target flag is set to "1" in the visitor management table TB shown in Fig. 6 (i.e., the person ID of a customer who has an outstanding payment). Also, for example, the processor 11 sets a caution flag associated with a person ID of a customer who is determined to be a person requiring caution.

[0069] <Display> 8 is a diagram showing an example of the display of the present disclosure. As shown in FIG. 8, display 20 displays an empty seat display, a seated person display, an order has been placed display, a food has been served display, a payment has been made display, and a payment has not yet been made display.

[0070] In FIG. 8, the second table seat T02, the fifth table seat T05, the sixth table seat T06, the eighth table seat T08, the tenth table seat T10, and the eleventh table seat T11 are displayed as vacant seats.

[0071] It also shows that one customer with person ID "P001" is seated at the first table seat T01 and is waiting to order.

[0072] Also, two customers with person IDs "P011" and "P012" are seated at the fourth table seat T04, and it is displayed that they have already placed their orders.

[0073] Also, it is displayed that two customers with person IDs "P051" and "P052" are seated at the third table T03 and that their food has already been served.

[0074] Additionally, four customers with person IDs "P041," "P042," "P043," and "P044" are seated at the twelfth table, T12, and it is displayed that they have already paid.

[0075] Furthermore, the seventh table seat T07 is displayed as having three customers with person IDs "P021," "P022," and "P023" seated there, indicating that the customer has not yet paid. Furthermore, processor 11 adds a star mark to the customer display for person ID "P022," indicating that, among the three customers with person IDs "P021," "P022," and "P023," customer ID "P022" is a customer who is likely to eat or drink without paying. For example, processor 11 extracts characteristic information of customer ID "P022" from the image of customer ID "P022" included in the entrance / exit area image and the fourth area image, searches the person list PL based on the extracted characteristic information, and determines that customer ID "P022" is a customer who is likely to eat or drink without paying when a warning flag is set in the person list PL that has characteristic information similar to the characteristic information of customer ID "P022."

[0076] Furthermore, the display shows that a customer with person ID "P031" is seated at table ninth seat T09 and has not yet paid. Furthermore, processor 11 adds a star mark to the customer display for person ID "P031," indicating that the customer with person ID "P031" is likely to eat or drink without paying. For example, processor 11 extracts characteristic information of the customer with person ID "P031" from the image of the customer with person ID "P031" included in the entrance / exit area image and the fifth area image, searches the person list PL based on the extracted characteristic information, and, if a warning flag is set in the person list PL that has characteristic information similar to the characteristic information of the customer with person ID "P031," determines that the customer with person ID "P031" is likely to eat or drink without paying.

[0077] As described above, the display 20 displays the payment status for each table seat.

[0078] Furthermore, for example, when the processor 11 determines that the customer with person ID "P031" is a suspicious person using the trained model M1 based on the image of the customer with person ID "P031" included in the fifth area image, the processor 11 may add a star mark to the customer display of person ID "P031." Examples of suspicious people include customers who stay in the store for a threshold amount of time or more, customers who sleep while seated, customers who behave suspiciously, and customers who commit violent acts.

[0079] The above is a description of the embodiment.

[0080] As described above, the payment-free eating and drinking prevention system (payment-free eating and drinking prevention system 1 of the embodiment) of the present disclosure includes a first camera (entrance / exit camera EC of the embodiment) that captures a first image (entrance / exit area image of the embodiment), which is an image of a first area that is a predetermined area including an entrance / exit of a restaurant, a second camera (customer seating area image of the embodiment), which is an image of a second area that is a predetermined area including table seats installed in the restaurant, and a customer management device (customer management device 10 of the embodiment). Based on the first and second images, the customer management device determines a second customer who is a first customer who entered the restaurant through the entrance / exit and is seated at a table, determines a third customer among the second customers who has ordered food and drink and has had their food and drink served, and based on the second image, determines a fourth customer among the third customers who has left their table, and provides a fifth customer among the fourth customers who is a customer who has been served but has not yet paid for their food and drink, with a message to encourage them to pay.

[0081] This will encourage customers who have not yet paid for their food and drink to pay, which will help prevent people from eating and drinking without paying. In particular, by providing guidance to encourage customers to pay, it is possible to prevent people from eating and drinking without paying without installing gates at the entrance and exit. [Explanation of symbols]

[0082] 1. System to prevent eating and drinking without paying 10 Visitor management device 40 POS systems EC Entrance / Exit Camera TC-1 First Audience Camera TC-2 Second Audience Camera TC-3 Third Audience Camera TC-4 Fourth Audience Camera TC-5 Fifth Audience Camera TC-6 Sixth Audience Camera 11 processors 20 Display 30 speakers

Claims

1. a first camera that captures a first image, which is an image of a first area that is a predetermined area including an entrance and exit of the restaurant; a second camera configured to capture a second image of a second area, which is a predetermined area including table seats installed in the restaurant; Based on the first image and the second image, a second customer is determined to be a customer who has sat at the table seat among first customers who have entered the store through the entrance; Among the second customers, a third customer is determined as a customer who has completed ordering food and drink and having the food and drink served; determining a fourth customer who has left the table seat among the third customers based on the second image; a visitor management device that provides a fifth customer, who is one of the fourth customers but has not yet paid for the food and drink that has been served, with a prompt to urge the fifth customer to pay the fee; A system to prevent eating and drinking without paying.

2. The visitor management device includes: determining whether the fourth customer is the fifth customer at a first time point when the fourth customer leaves the table seat; When the fourth customer is the fifth customer at the first time point, the guidance is provided to urge the fifth customer to pay the price at a cash register installed in the restaurant. The system for preventing eating and drinking without paying as claimed in claim 1.

3. The visitor management device includes: determining whether the fourth customer is the fifth customer at a second time point that is after the first time point and before the fourth customer leaves the store through the entrance; When the fourth customer is the fifth customer at the second time point, the guidance is provided to urge the fifth customer to pay the bill within the restaurant. The system for preventing eating and drinking without paying as claimed in claim 2.

4. the visitor management device provides the guidance using voice, The system for preventing eating and drinking without paying as claimed in claim 1.

5. The restaurant further includes a display installed in the restaurant that displays the payment status for each table seat. The system for preventing eating and drinking without paying as claimed in claim 1.

6. The visitor management device includes: determining whether the second customer is a sixth customer who is likely to eat or drink without paying; displaying the status of the settlement on the display and displaying a mark indicating that the second customer is the sixth customer on the display; The system for preventing eating and drinking without paying as set forth in claim 5.

7. Based on an image of a first area, which is a predetermined area including an entrance / exit of the restaurant, and a second image, which is an image of a second area, which is a predetermined area including table seats installed in the restaurant, determining a second customer, who is a customer who has sat at the table seat among first customers, who are customers who have entered the restaurant through the entrance / exit; Among the second customers, a third customer is determined as a customer who has completed ordering food and drink and having the food and drink served; determining a fourth customer who has left the table seat among the third customers based on the second image; a processor that provides a fifth customer, who is one of the fourth customers but has not yet paid for the food and drink that has been served, with a message to encourage the fifth customer to pay the price; A visitor management device comprising:

Images