Information processing system, information processing method, and information processing program

Abstract

To provide an information processing system, information processing method, and information processing program, which enable accurate determination of whether a usage target in a facility is vacant or not.SOLUTION: An information processing device 1 comprises an image acquisition processing unit 111 that acquires a captured image of a usage area including a usage target used by a user in a facility, and a vacancy determination processing unit 114 that determines whether a usage target is vacant or not based on the types of persons in the usage area in the captured image acquired by the image acquisition processing unit 111.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and an information processing program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, in facilities such as restaurants, a technique has been proposed for detecting the presence or absence of a customer using a camera, a sensor, or the like, and determining whether a seat at a table is vacant or not (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-133847 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technology, if a seat is occupied, it is determined that the seat is occupied, and if no one is in the seat, it is determined that the seat is vacant, which can cause the following problems. For example, if a user leaves the seat temporarily to pick up food or go to the restroom, the seat is determined to be vacant because no one is in the seat. This causes a problem in which the facility staff guides other users to the table (target table) that is being used by the user.

[0005] An object of the present invention is to provide an information processing system, an information processing method, and an information processing program capable of accurately determining whether a seat for a facility user is vacant. [Means for solving the problem]

[0006] The information processing system of the present invention includes an image acquisition processing unit that acquires an image of an area of ​​use including a target of use used by a user in a facility, a facility acquisition processing unit that acquires facility information including at least one of payment information regarding payment of the facility usage fee, order information regarding orders for products offered by the facility, and reservation information regarding reservations for the facility, and a vacancy determination processing unit that determines whether the target of use is vacant or not based on the image acquired by the image acquisition processing unit and the facility information acquired by the facility acquisition processing unit.

[0007] The information processing method of the present invention is a method executed by one or more processors, which includes an image acquisition step of acquiring an image of an area of ​​a facility including a target of use used by a user; a facility acquisition step of acquiring facility information including at least one of payment information regarding payment of the facility usage fee, order information regarding orders for products offered by the facility, and reservation information regarding reservations for the facility; and a vacancy determination step of determining whether the target of use is vacant or not based on the image acquired by the image acquisition step and the facility information acquired by the facility acquisition step.

[0008] The information processing program of the present invention is a program for causing one or more processors to execute the following steps: an image acquisition step of acquiring an image of an area of ​​a facility including a target of use used by a user; a facility acquisition step of acquiring facility information including at least one of payment information regarding payment of the facility usage fee, order information regarding orders of products offered by the facility, and reservation information regarding reservations for the facility; and a vacancy determination step of determining whether the target of use is vacant or not based on the image acquired by the image acquisition step and the facility information acquired by the facility acquisition step. Effect of the Invention

[0009] According to the present invention, there are provided an information processing system, an information processing method, and an information processing program capable of accurately determining whether a seat for a facility user is vacant or not. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of reservation information used in the information processing system according to the embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing an example of order information used in the information processing system according to the embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of payment information used in the information processing system according to the embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing an example of a captured image of the usage area according to the embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of a captured image of the usage area according to the embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of a captured image of the usage area according to the embodiment of the present invention. [Figure 8] FIG. 8 is a diagram showing an example of vacant seat information used in the information processing system according to the embodiment of the present invention. [Figure 9] FIG. 9 is a flowchart showing an example of a procedure of a vacant seat determination process executed by the information processing device according to the embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing another example of the procedure of the vacant seat determination process executed by the information processing device according to the embodiment of the present invention. [Figure 11] FIG. 11 is a flowchart showing another example of the procedure of the vacant seat determination process executed by the information processing device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings to facilitate understanding of the present invention. Note that the following embodiment is an example of the present invention and does not limit the technical scope of the present invention.

[0012] [Information processing system 100] 1, an information processing system 100 according to an embodiment of the present invention includes an information processing device 1, a facility terminal 2, and an imaging device 3. The information processing device 1, the facility terminal 2, and the imaging device 3 are capable of communicating with each other via a communication network N1 such as the Internet, a LAN, a WAN, or a public telephone line.

[0013] The imaging device 3 is a digital camera that captures an image of a subject and outputs it as digital image data. The imaging device 3 has a communication function that transmits the digital image data to the information processing device 1 through the communication network N1. The imaging device 3 may be one or more. Specifically, the imaging device 3 is installed in a facility and captures a predetermined use area including a table (use object) used by users of the facility. For example, the imaging device 3 is installed on the ceiling of the facility and captures the use area including one or more tables arranged in the facility. The imaging device 3 captures the use area at a predetermined frame rate and transmits the image data to the information processing device 1. The use area is, for example, a predetermined area including one table and multiple chairs attached to the table. The use area may also be, for example, the entire area of ​​the facility. In the present invention, a camera already installed in the facility can be applied to the imaging device 3.

[0014] Facility terminal 2 is placed at the facility and is capable of accepting operations from facility staff. Facility terminal 2 has a reservation management function for managing reservation information related to reservations for the facility, an order management function for managing order information related to product orders by users, and a payment management function for managing payment information related to payments (accounts) for the user's usage fees. Facility terminal 2 transmits facility information including at least one of the reservation information, the order information, and the payment information to information processing device 1. In addition to each of the management functions, facility terminal 2 may also have a reservation function for executing reservation processing, an order function for executing order processing, a payment function for executing payment processing, and the like. Facility terminal 2 may be a POS terminal.

[0015] The information processing device 1 judges whether or not the table to be judged is vacant based on the captured image of the usage area captured by the imaging device 3 and the facility information acquired from the facility terminal 2. The information processing device 1 may have a function as a facility information providing site (e.g., a reservation site) that provides facility information on a plurality of facilities. The facility is, for example, a restaurant. The subject of use of the present invention is not limited to a table, but may be a room (private room), a chair (seat), a counter, etc. Furthermore, the facility of the present invention is not limited to a restaurant, but may be an amusement park, a theater, a karaoke shop, etc. In the following embodiment, a restaurant is taken as an example of a facility for explanation. For example, the information processing device 1 judges whether a specific table in a restaurant is vacant or occupied, and outputs the judgment result (vacant seat status). Also, for example, the information processing device 1 displays the vacant seat status of the restaurant on a reservation site displayed on a user terminal (not shown). The information processing device 1 may have a reservation function that executes a reservation for a restaurant in response to a user request on the reservation site.

[0016] In this embodiment, the information processing device 1 alone corresponds to the information processing system according to the present invention, but the information processing system according to the present invention may include one or more of the components of the information processing device 1, the facility terminal 2, and the imaging device 3. For example, when multiple components of the information processing device 1, the facility terminal 2, and the imaging device 3 cooperate to share and execute various vacant seat determination processes (see FIG. 9, etc.) described below, a system including multiple components that execute the processes can be regarded as the information processing system according to the present invention. For example, the information processing system according to the present invention may be composed of the information processing device 1 and the facility terminal 2.

[0017] [Facility Terminal 2] 1, facility terminal 2 is a server computer including a control unit 21, a storage unit 22, an operation display unit 23, and a communication I / F 24. Note that facility terminal 2 is not limited to a single computer, and may be a computer system in which multiple computers operate in cooperation. Also, various processes executed by facility terminal 2 may be executed in a distributed manner by one or multiple processors.

[0018] The communication I / F 24 is a communication interface for connecting the facility terminal 2 to the communication network N1 via a wired or wireless connection, and for performing data communication in accordance with a predetermined communication protocol between the facility terminal 2 and external devices such as the information processing device 1 and the imaging device 3 via the communication network N1.

[0019] The operation display unit 23 is a user interface that includes a display unit, such as a liquid crystal display or an organic EL display, that displays various information, and an operation unit, such as a mouse, a keyboard, or a touch panel, that accepts operations.

[0020] The storage unit 22 is a non-volatile storage unit including a semiconductor memory, a hard disk drive (HDD), or a solid state drive (SSD) that stores various information. Specifically, the storage unit 22 stores data such as reservation information D1 relating to reservations at restaurants, order information D2 relating to orders for products (food, drinks, etc.) provided by restaurants, and payment information D3 relating to payments made by users at restaurants. Fig. 2 is a diagram showing an example of the reservation information D1, Fig. 3 is a diagram showing an example of the order information D2, and Fig. 4 is a diagram showing an example of the payment information D3.

[0021] The reservation information D1 is information on the reservation status of the restaurant, and is stored in the storage unit 12 in association with each restaurant. FIG. 2 shows the reservation information D1 corresponding to a specific restaurant. As shown in FIG. 2, the reservation information D1 includes information such as "reservation person name", "number of people", "visit date", "scheduled visit time", "usage time", and "table number". The "reservation person name" is the name of the user (reservation person) who made the reservation. The "number of people" is the number of people who are scheduled to visit the restaurant. The "visit date" is the day when the user is scheduled to visit the restaurant. The "scheduled visit time" is the time when the user is scheduled to visit the restaurant. The "usage time" is the time when the user can use the restaurant (table). The "table number" is information for identifying the object of use such as a table used by the user, and the number of a table seat, a private room, a counter seat, etc. is registered. For example, the number of the table assigned in the reservation process or seating arrangement process executed in the facility terminal 2 is registered in the "table number".

[0022] Here, the "scheduled visit time" and "usage time" represent the available time of the corresponding table. For example, the available time of table T1 with table number "1" is 2 hours from 18:00 on April 30, 2020 (i.e., 18:00 to 20:00), the available time of table T2 with table number "2" is 2 hours from 18:30 on April 30, 2020 (i.e., 18:30 to 20:30), and the available time of table T3 with table number "3" is 2 hours from 19:00 on April 30, 2020 (i.e., 19:00 to 21:00). The end time of the available time represents the limit time (available end time) by which a user can use the table. For example, the available end time of table T1 is "20:00", the available end time of table T2 is "20:30", and the available end time of table T3 is "21:00".

[0023] The reservation information D1 may include the details of the product reserved by the user (such as a meal course). The reservation information D1 is registered or changed every time a new reservation or a reservation change is accepted from a user terminal (not shown).

[0024] The order information D2 is information related to an order for a product received from a user. As shown in FIG. 3, the order information D2 includes information such as a table number, the number of users, an order number, an ordered product, and a serving status. The table number is identification information for identifying the table for which an order has been received. The number of users is the number of users using the table. The information on the number of users is obtained, for example, from an order terminal (not shown) placed at the table, an operation terminal (not shown) operated by a staff member of the restaurant, reservation information D1 (see FIG. 2), or the like. The order number is identification information for identifying an order by a user. The control unit 21 registers a number (order number) in the order information D2, for example, in the order in which the order is received from the user, that is, in the order in which the order is received from the order terminal. The ordered product is information indicating the contents of the order. The ordered product is registered with the product name of a product (food, drink, etc.) selected by the user from among products that the restaurant can provide. The order number is associated with each ordered product.

[0025] The serving status is information for identifying whether or not the product ordered by the user has been served to the user. For example, the control unit 21 outputs information on the ordered product registered in the order information D2 to a kitchen terminal and instructs the kitchen staff to cook the product. When the kitchen staff completes cooking the ordered product, the waiter staff serves the ordered product to the table of the user. When the waiter staff serves the ordered product to the table and performs a completion operation, the control unit 21 registers information indicating that the serving of the ordered product has been completed ("Complete") in the serving status. On the other hand, when the order has been received from the user but the serving of the ordered product has not yet been completed, the control unit 21 registers information indicating that the serving of the ordered product has not yet been completed ("Incomplete") in the serving status. In the example shown in FIG. 3, the table T1 indicates that the serving of dishes M1 to M3 has been completed, the table T2 indicates that the serving of dishes M4 and M5 has not yet been completed, and the table T3 indicates that the serving of dish M8 has not yet been completed. That is, the table T1 indicates that the serving of the product has been completed, and the tables T2 and T3 indicate that the serving of the product has not yet been completed. Each piece of information in the order information D2 is updated as appropriate by operations by users, staff, etc.

[0026] The payment information D3 is information on the amount used by the user for each table. As shown in FIG. 4, the payment information D3 includes information such as the table number, the amount used, and the payment status. The table number is identification information that identifies the table used by the user. The amount used indicates the fee (billed amount) charged to the user for the table. The payment status is information that identifies whether or not the payment for the amount used for the table has been completed. For example, when a user of table T1 finishes his / her meal and performs the payment operation, the control unit 21 registers "Completed" as the payment status for table T1. When a user has not yet performed the payment operation for tables T2 and T3, the control unit 21 registers "Incomplete" as the payment status for tables T2 and T3. Each piece of information in the payment information D3 is updated as appropriate according to the payment operation by the user.

[0027] As another embodiment of the facility terminal 2, some or all of the reservation information D1, order information D2, and payment information D3 may be stored in another server accessible from the facility terminal 2 via the communication network N1. Also, some or all of the reservation information D1, order information D2, and payment information D3 may be stored in the information processing device 1.

[0028] Furthermore, the storage unit 22 stores a control program for causing the control unit 21 to execute various processes. For example, the control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a reading device (not shown) such as a CD drive or a DVD drive provided in the facility terminal 2 and stored in the storage unit 22. The control program may also be downloaded from an external device via the communication network N1 and stored in the storage unit 22.

[0029] The control unit 21 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The control unit 21 controls the facility terminal 2 by executing, by the CPU, various control programs stored in advance in the ROM or the storage unit 22.

[0030] Specifically, as shown in Fig. 1, the control unit 21 includes various processing units such as a reservation processing unit 211, an order processing unit 212, and a payment processing unit 213. The control unit 21 functions as the various processing units by executing various processes in accordance with the control program using the CPU. Some or all of the processing units included in the control unit 21 may be configured with electronic circuits. The control program may be a program for causing a plurality of processors to function as the various processing units.

[0031] The reservation processing unit 211 executes reservation processing for a restaurant based on a reservation request from a user. Specifically, when the reservation processing unit 211 acquires search conditions from a user, it presents the search results of restaurants that satisfy the search conditions to the user, and when it accepts a reservation request for a restaurant from the user, it executes reservation processing for the restaurant. For example, when a user inputs a visit date, number of people, and usage time (scheduled visit time) on a reservation site, the reservation processing unit 211 searches for restaurants that satisfy these search conditions, and when it accepts a reservation request for a restaurant from the user in the search results, it completes a reservation for the restaurant. In addition, the reservation processing unit 211 assigns a table to a user (reservation person) based on, for example, seating arrangement operations by the restaurant staff. The reservation processing unit 211 registers reservation information D1 (see FIG. 2) every time reservation processing is completed.

[0032] The order processing unit 212 receives orders for products (food, drinks, etc.) from users. For example, the order processing unit 212 receives an operation by the user to select a product on an order screen displayed on an order terminal (not shown) placed at a table. When the order processing unit 212 receives an order for a product from a user, it registers the order number and information on the ordered product in order information D2 (see FIG. 3) in association with the table number of the table used by the user.

[0033] The order processing unit 212 also outputs information on orders received from users (order information) to the kitchen terminal. Specifically, the order processing unit 212 outputs order information including the order details of the products ordered by the user, the table number of the table used by the user, and the order number to the kitchen terminal. The order information may also include the order date and time (date and time of receipt). The order processing unit 212 also causes an order slip including the order details, the table number, and the order number to be output (printed or displayed) from the kitchen terminal.

[0034] In addition, the order processing unit 212 determines the provision status of the ordered product and registers it in the order information D2. Specifically, when the ordered product has been provided to the user, the order processing unit 212 registers "Completed" as the provision status of the order information D2, and when the ordered product has not been provided to the user, the order processing unit 212 registers "Incomplete" as the provision status of the order information D2 (see FIG. 3).

[0035] When accepting a payment operation from a user, the payment processing unit 213 performs a payment process for the amount used. For example, the payment processing unit 213 transmits the user's payment authentication information (authentication information required to make an electronic payment using a payment server), the amount used (amount billed), and identification information of the restaurant to a payment server (not shown), and performs the payment process for the amount billed.

[0036] The payment processing unit 213 also determines the payment status of the usage amount and registers it in the payment information D3. Specifically, when the payment process of the usage amount is completed, the payment processing unit 213 registers "Completed" as the payment status in the payment information D3, and when the payment process of the usage amount is not completed, the payment processing unit 213 registers "Incomplete" as the payment status in the payment information D3 (see FIG. 4).

[0037] [Information processing device 1] As shown in Fig. 1, the information processing device 1 is a server computer including a control unit 11, a storage unit 12, an operation display unit 13, and a communication I / F 14. The information processing device 1 is not limited to a single computer, and may be a computer system in which multiple computers work in cooperation. Various processes executed by the information processing device 1 may be executed in a distributed manner by one or multiple processors. The information processing device 1 is an example of an information processing system of the present invention.

[0038] The communication I / F 14 is a communication interface for connecting the information processing device 1 to the communication network N1 via a wired or wireless connection, and for performing data communication in accordance with a predetermined communication protocol between the information processing device 1 and external devices such as the facility terminal 2 and the imaging device 3 via the communication network N1.

[0039] The operation display unit 13 is a user interface that includes a display unit, such as a liquid crystal display or an organic EL display, that displays various information, and an operation unit, such as a mouse, a keyboard, or a touch panel, that accepts operations.

[0040] The storage unit 12 is a non-volatile storage unit including a semiconductor memory, a hard disk drive (HDD), or a solid state drive (SSD) for storing various information. The storage unit 12 stores control programs such as a vacant seat determination program (information processing program) for causing the control unit 11 to execute a vacant seat determination process (see FIG. 9, etc.) described below. For example, the vacant seat determination program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a reading device (not shown) such as a CD drive or DVD drive provided in the information processing device 1 and stored in the storage unit 12.

[0041] Here, in a restaurant, for example, a staff member places (sets) a chopstick case, an order terminal, and the like on a table before guiding a customer to a table. In addition, when the staff member accepts a food order from the customer at the table, the staff member serves the ordered food to the customer's table. In addition, when the customer finishes his / her meal and pays the bill and leaves the restaurant, the staff member clears away the tableware (plates, cups, etc.) and sets up the chopstick case, order terminal, and the like again in order to guide another customer to the table. The set table is determined to be vacant and is registered as a table that can be guided to (or can be reserved). In a restaurant, in order to guide more customers, it is required to accurately determine whether a customer is using each table (whether the customer is seated) or has left the restaurant (whether the customer is vacant). The information processing system 100 according to this embodiment is capable of accurately determining whether a table in a restaurant is vacant or not.

[0042] The control unit 11 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The control unit 11 controls the information processing device 1 by executing various control programs stored in advance in the ROM or the storage unit 12 by the CPU.

[0043] Specifically, as shown in Fig. 1, the control unit 11 includes various processing units such as an image acquisition processing unit 111, a user determination processing unit 112, a facility acquisition processing unit 113, a vacant seat determination processing unit 114, a calculation processing unit 115, and an output processing unit 116. The control unit 11 functions as the various processing units by executing various processes according to the vacant seat determination program with the CPU. Some or all of the processing units included in the control unit 11 may be configured with electronic circuits. The vacant seat determination program may be a program for causing a plurality of processors to function as the various processing units.

[0044] The image acquisition processing unit 111 acquires an image TP1 of a predetermined usable area AR1 including tables arranged in a restaurant, which is captured by the imaging device 3. An example of the captured image TP1 is shown in Fig. 5. For example, the imaging device 3 captures an image of the usable area AR1 including one table arranged in a restaurant and four chairs associated with the table. The image acquisition processing unit 111 is an example of the image acquisition processing unit of the present invention.

[0045] The user determination processing unit 112 determines whether or not a user (person) is included in the use area AR1 based on the captured image TP1. Specifically, the user determination processing unit 112 performs image analysis on the captured image TP1 acquired by the image acquisition processing unit 111 to extract a person. If the user determination processing unit 112 extracts a person, it determines that the user is included in the use area AR1, and if the user is not extracted, it determines that the user is not included in the use area AR1. Note that feature data relating to the features of people is registered in advance in the storage unit 12, and the user determination processing unit 112 determines the presence or absence of a user by comparing the captured image TP1 with the feature data.

[0046] If the captured image TP1 includes restaurant staff, the user determination processing unit 112 determines whether or not there is a user, excluding the staff. Specifically, if a person stays within a predetermined range from a table for a predetermined time in the captured image TP1, or if the person is seated in a chair, the user determination processing unit 112 determines that a user is included in the usage area AR1. If the captured image TP1 includes a person who does not stay for a predetermined time, a person who has a pre-registered terminal (a staff terminal), or a person who is wearing pre-registered clothing (a uniform), the user determination processing unit 112 may determine that the person is a staff member and exclude the person from the determination of whether or not there is a user.

[0047] The facility acquisition processing unit 113 acquires facility information including at least one of reservation information D1 (see FIG. 2) related to a reservation for a restaurant, order information D2 (see FIG. 3) related to an order for a product (food, drink, etc.) provided by the restaurant, and payment information D3 (see FIG. 4) related to a payment of a user's spending amount at the restaurant. Specifically, the facility acquisition processing unit 113 acquires at least one of the reservation information D1, order information D2, and payment information D3 from the facility terminal 2. For example, when the information processing device 1 performs a process of determining whether a table has an available seat based on the reservation information, the facility acquisition processing unit 113 acquires the reservation information D1 from the facility terminal 2. Also, for example, when the information processing device 1 performs a process of determining whether a table has an available seat based on the order information, the facility acquisition processing unit 113 acquires the order information D2 from the facility terminal 2. Also, for example, when the information processing device 1 performs a process of determining whether a table has an available seat based on the payment information, the facility acquisition processing unit 113 acquires the payment information D3 from the facility terminal 2. The information processing device 1 may perform a process of determining whether a table has an available seat by combining the reservation information D1, the order information D2, and the payment information D3.

[0048] The vacant seat determination processing unit 114 determines whether or not a table (target of use) is vacant based on the captured image TP1 acquired by the image acquisition processing unit 111 and the facility information acquired by the facility acquisition processing unit 113. The vacant seat determination processing unit 114 is an example of the vacant seat determination processing unit of the present invention.

[0049] Specifically, the vacant seat determination processing unit 114 determines that the table is vacant when the user is not included in the use area AR1 in the captured image TP1 and the current time has passed the end time of the available use corresponding to the reservation information D1 of the user. For example, in the reservation information D1 shown in FIG. 2, the user A1 is scheduled to arrive at the restaurant at 18:00 and the use time is two hours, so the available time during which the user A1 can use the table T1 is from 18:00 to 20:00, and the available time ends at 20:00. Therefore, the vacant seat determination processing unit 114 determines that the table T1 is vacant when the user A1 is not included in the use area AR1 in the captured image TP1 and the current time has passed the available time end time "20:00". Here, if the user A1 is included in the use area AR1 even though the current time has passed the available time end time "20:00", the control unit 11 may notify the order terminal of the table T1, the mobile terminal of the staff, etc. that the available time has passed.

[0050] Furthermore, the vacant seat determination processor 114 determines that the table is occupied if no user is included in the usable area AR1 in the captured image TP1 and the current time has not passed the end time of usable use. For example, if user A1, who can use table T1 from 18:00 to 20:00, is not included in the usable area AR1 of table T1 at 19:00, user A1 may have temporarily left his / her seat. For this reason, the vacant seat determination processor 114 does not determine that table T1 is vacant, but determines that table T1 is occupied, if user A1 is not included in the usable area AR1 and the current time has not passed the end time of usable use, "20:00."

[0051] In the above configuration, the vacant seat determination processing unit 114 performs the vacant seat determination processing of the table based on the available end time specified from the reservation information D1, but in another embodiment, the vacant seat determination processing unit 114 may perform the vacant seat determination processing based on the available end time included in the facility information. For example, when a user (a so-called walk-in customer) visits a restaurant without making a reservation, the available end time of the user is the time of visit plus the usage time set by the restaurant under the time system (e.g., 2-hour system, 3-hour system). For example, when a user A4 visits a restaurant with a 3-hour system at 7 p.m. without making a reservation, the available end time of the user A4 is 10 p.m. In this case, the vacant seat determination processing unit 114 determines that the table used by the user A4 is vacant when the user A4 is not included in the usage area AR1 in the captured image TP1 and the current time is past the available end time "10 p.m."

[0052] In addition, the vacant seat determination processing unit 114 determines a table as vacant when a user is not included in the usage area AR1 in the captured image TP1 and the items ordered by the user have been served. For example, in the order information D2 shown in Fig. 3, the serving of dishes M1, M2, and M3 corresponding to table T1 with table number "1" has been completed, so when the user determination processing unit 112 determines that a user is not included in the usage area AR1 corresponding to table T1, the vacant seat determination processing unit 114 determines that table T1 is vacant.

[0053] In contrast, because the serving of dishes M4 and M5 corresponding to table T2 with table number "2" has not been completed, even if the user determination processor 112 determines that no users are included in the usage area AR1 corresponding to table T2, the vacant seat determination processor 114 determines that table T2 is not vacant or that a user is seated at the table, in the same manner as for table T2, the vacant seat determination processor 114 determines that table T3 is not vacant or that a user is seated at the table,

[0054] Here, for example, it is possible that a user may temporarily leave the table in the middle of eating the food after the food has been served. In this case, it is appropriate for the vacant seat determination processing unit 114 not to determine that the table is vacant or to determine that the user is occupied. Therefore, as another embodiment, the vacant seat determination processing unit 114 determines that the table is vacant when the user is not included in the use area AR1 in the captured image TP1, the product ordered by the user has been served, and a predetermined time has passed since the product was served to the user. Also, the vacant seat determination processing unit 114 determines that the table is occupied when the user is not included in the use area AR1 in the captured image TP1, the product ordered by the user has been served, and a predetermined time has not passed since the product was served to the user. This makes it possible to avoid determining that the table is vacant when the user has temporarily left the table.

[0055] The predetermined time may be set according to the content or type of the product. For example, the control unit 11 sets the predetermined time to one hour for dishes that take a long time to eat (hot pot dishes, set menus, etc.) and dishes that the user orders early in the usage time. On the other hand, the control unit 11 sets the predetermined time to 30 minutes for dishes that do not take a long time to eat and dishes that the user orders late in the usage time (desserts, etc.).

[0056] In addition, the vacant seat determination processing unit 114 determines a table as vacant when a user is not included in the usage area AR1 in the captured image TP1 and the payment of the usage fee corresponding to the user has been completed. For example, in the payment information D3 shown in Fig. 4, the payment process of the usage fee corresponding to table T1 with table number "1" has been completed, so when the user determination processing unit 112 determines that a user is not included in the usage area AR1 corresponding to table T1, the vacant seat determination processing unit 114 determines that table T1 is vacant.

[0057] On the other hand, because the payment process for the usage amount corresponding to table T2 with table number "2" has not been completed, even if the user determination processing unit 112 determines that the user is not included in the usage area AR1 corresponding to table T2, the vacant seat determination processing unit 114 determines that table T2 is not vacant or that a seat is occupied. As with table T2, the vacant seat determination processing unit 114 also determines that table T3 is not vacant or that a seat is occupied.

[0058] In addition, before accepting an order from a user, the vacant seat determination processing unit 114 determines whether or not the table is vacant based on the captured image TP1, and after accepting an order from a user, determines whether or not the table is vacant based on the captured image TP1 and at least one of the order information and the payment information.

[0059] For example, before accepting an order from a user, if the usage area AR1 of the captured image TP1 includes at least one of an item owned by the user (owner's item) and a reservation display medium such as a place card or reservation plate that is pre-installed on the table of the reserved seat, the vacant seat determination processing unit 114 determines that the table is not vacant (e.g., an occupied seat).

[0060] For example, before the control unit 11 accepts an order from a user, if the user determination processing unit 112 determines that the user is not included in the usage area AR1 and the control unit 11 extracts the user's items (smartphone 31, bag 32) (see Figure 6) or a reservation display medium (reservation plate 33) (see Figure 7) in the captured image TP1 of the usage area AR1, the vacancy determination processing unit 114 determines that the user is using the table and is present at the seat.

[0061] On the other hand, for example, after the control unit 11 accepts an order from a user, if the user determination processing unit 112 determines that the use area AR1 does not include the user, the user's item, and the reservation display medium, and the provision of the product ordered by the user has been completed, or if the payment of the usage fee corresponding to the user has been completed, the vacant seat determination processing unit 114 determines that the user is not using the table and determines that the seat is vacant. Also, for example, after the control unit 11 accepts an order from a user, if the user determination processing unit 112 determines that the use area AR1 does not include the user, the user's item, and the reservation display medium, and the provision of the product ordered by the user has not been completed, or if the payment of the usage fee corresponding to the user has not been completed, the vacant seat determination processing unit 114 determines that the user is using the table and determines that the user is present.

[0062] The calculation processing unit 115 calculates the number of vacant seats in the restaurant based on the result of the determination (vacant or occupied) made by the vacant seat determination processing unit 114 for each table arranged in the restaurant. Specifically, the calculation processing unit 115 calculates the total number of tables determined to be vacant as the number of vacant seats in the restaurant. For example, the vacant seat determination processing unit 114 determines whether each of tables T1 to T5 arranged in the restaurant is vacant or occupied, and registers the determination result in vacant seat information D4 (see FIG. 8). The vacant seat information D4 is stored in the storage unit 12. The vacant seat determination processing unit 114 updates the vacant seat information D4 in real time. The calculation processing unit 115 calculates the number of vacant seats in the restaurant by adding up the number of vacant seats registered in the vacant seat information D4 (see FIG. 8).

[0063] The output processing unit 116 outputs the vacancy status of tables in the restaurant. Specifically, the output processing unit 116 presents information indicating whether a table is vacant or occupied to a store terminal of the restaurant or a user terminal.

[0064] Furthermore, the output processing unit 116 may output information on the number of vacant seats calculated by the calculation processing unit 115. Specifically, the output processing unit 116 causes the number of vacant seats calculated by the calculation processing unit 115 to be displayed on a reservation site as the vacant seating situation of the restaurant. Furthermore, the output processing unit 116 may cause the reservation site to display "vacant seats" or "occupied (or full)" for each table of the restaurant. This allows users to obtain information for making reservations at the restaurant. Furthermore, the output processing unit 116 may cause the number of vacant seats calculated by the calculation processing unit 115 to be displayed on a store terminal (not shown) of the restaurant. This allows staff at the restaurant to easily grasp the vacant seating situation at tables.

[0065] [Vacancy determination process] Hereinafter, the vacant seat determination process executed by the control unit 11 of the information processing device 1 will be described with reference to Fig. 9. For example, the control unit 11 of the information processing device 1 starts execution of the vacant seat determination program by the imaging device 3 capturing an image of a predetermined usable area AR1 of a restaurant, thereby starting execution of the vacant seat determination process. Note that, when the usable area AR1 includes multiple tables, the control unit 11 may execute the vacant seat determination process for each table individually in parallel.

[0066] The present invention can be understood as an invention of a vacant seat determination method for executing one or more steps included in the vacant seat determination process. The vacant seat determination method is one example of an information processing method of the present invention.

[0067] One or more steps included in the vacancy determination process described here may be omitted as appropriate. The steps in the vacancy determination process may be executed in a different order as long as the same action and effect is achieved. Although the steps are executed by the control unit 11 as an example, in other embodiments, the steps in the vacancy determination process may be executed in a distributed manner by multiple processors.

[0068] In step S11, the control unit 11 acquires a captured image TP1 of the usable area AR1 captured by the imaging device 3. For example, the control unit 11 acquires a captured image TP1 of the usable area AR1 including one table and four chairs arranged in a restaurant from the imaging device 3. The above-mentioned step S11 is an example of an acquisition step of the present invention.

[0069] Next, in step S12, the control unit 11 determines whether or not a user (person) is included in the use area AR1 based on the captured image TP1. As described above, the control unit 11 determines whether or not a user is included, excluding staff of the restaurant. The control unit 11 determines that a user is included in the use area AR1 when an image of a person is extracted from the captured image TP1, and determines that a user is not included in the use area AR1 when a person is not extracted from the captured image TP1. If the control unit 11 determines that a user is not included in the use area AR1 (S12: No), the process proceeds to step S13. On the other hand, if the control unit 11 determines that a user is included in the use area AR1 (S12: Yes), the process proceeds to step S16. If a table is used by multiple users, the control unit 11 determines that a user is included in the use area AR1 when at least one user is included in the use area AR1, and determines that a user is not included in the use area AR1 when not even one user is included in the use area AR1.

[0070] In step S13, the control unit 11 acquires payment information D3 (see FIG. 4), which is an example of facility information. Specifically, the control unit 11 acquires payment information D3 corresponding to the table used by the user from the storage unit 12. Step S13 is an example of a facility acquisition step of the present invention.

[0071] In step S14, the control unit 11 determines whether or not the payment process for the usage amount corresponding to the user has been completed based on the payment information D3. If the control unit 11 determines that the payment process has been completed (S14: Yes), the process proceeds to step S15. On the other hand, if the control unit 11 determines that the payment process has not been completed (S14: No), the process proceeds to step S16.

[0072] In step S15, the control unit 11 determines that the table is vacant. For example, if no user is included in the usage area AR1 (S12: No) and the payment process is completed (S14: Yes), the control unit 11 determines that the table is vacant. After that, the process proceeds to step S17. Step S15 is an example of a vacant seat determination step of the present invention.

[0073] On the other hand, in step S16, the control unit 11 determines that the table is occupied. For example, if the user is included in the usage area AR1 (S12: Yes) and the payment process is not completed (S14: No), the control unit 11 determines that the table is occupied. After that, the process proceeds to step S17. Step S17 is an example of an empty seat determination step of the present invention.

[0074] In step S17, the control unit 11 outputs the vacancy status of the table. For example, the control unit 11 presents information indicating whether the table in the restaurant is vacant or occupied to a store terminal or a user terminal of the restaurant. The control unit 11 may present information on the number of vacant seats in the restaurant (for example, the total number of vacant seats) to the store terminal or the user terminal. In this manner, the control unit 11 executes the vacancy determination process.

[0075] As another example of the vacant seat determination process, in step S13 shown in Fig. 9, the control unit 11 may acquire reservation information D1 (see Fig. 2), which is an example of facility information. Fig. 10 shows another example of the vacant seat determination process. In Fig. 10, the same steps as those shown in Fig. 9 are indicated with the same step numbers, and the description thereof will be omitted.

[0076] Specifically, in step S23 shown in Fig. 10, control unit 11 acquires reservation information D1 corresponding to the user from storage unit 12, and acquires the end time of use based on reservation information D1. Then, in step S24, control unit 11 determines whether the current time has passed the end time of use. If control unit 11 determines that the current time has passed the end time of use (S24: Yes), the process proceeds to step S15. On the other hand, if control unit 11 determines that the current time has not passed the end time of use (S24: No), the process proceeds to step S16.

[0077] As another example of the vacant seat determination process, in step S13 shown in Fig. 9, the control unit 11 may acquire order information D2 (see Fig. 3), which is an example of facility information. Fig. 11 shows another example of the vacant seat determination process. In Fig. 11, the same steps as those shown in Fig. 9 are indicated with the same step numbers, and the description thereof will be omitted.

[0078] Specifically, in step S33 shown in FIG. 11, the control unit 11 may acquire order information D2 (see FIG. 3), which is an example of facility information. Specifically, the control unit 11 acquires order information D2 corresponding to the table used by the user from the memory unit 12. Then, in step S34, the control unit 11 determines whether or not the provision of the product ordered by the user has been completed. If the control unit 11 determines that the provision of the product ordered by the user has been completed (S34: Yes), the process proceeds to step S15. On the other hand, if the control unit 11 determines that the provision of the product ordered by the user has not been completed (S34: No), the process proceeds to step S16.

[0079] The control unit 11 can also combine the vacant seat determination processes shown in Figs. 9 to 11 as appropriate.

[0080] As described above, the information processing system 100 according to the present embodiment determines whether or not a table is vacant based on a captured image TP1 capturing an area AR1 including objects of use (such as a table) used by a user in a facility, and facility information including at least one of payment information D3, order information D2, and reservation information D1. As a result, even if a user is not included in the area AR1, if the current time has not passed the end time of use, if the product ordered by the user has not been provided, or if the payment of the usage fee has not been completed, the table is determined to be occupied rather than vacant.

[0081] In addition, for example, if the user is not included in the usage area AR1 and the current time has passed the end of the usage time, the product ordered by the user has been provided, or the payment of the usage fee has been completed, the table is determined to be vacant.

[0082] In this way, the information processing system 100 is equipped with a configuration for determining whether a table is being used by a user based on the captured image TP1 and the facility information, thereby making it possible to accurately determine whether the table is vacant or not.

[0083] The information processing system 100 of the present invention is not limited to the above-described embodiment. The information processing system 100 may have the following forms.

[0084] As another embodiment, the calculation processing unit 115 may calculate a first vacancy value of a table based on the captured image TP1, and calculate a second vacancy value of the table based on facility information including at least one of the reservation information D1, the order information D2, and the payment information D3. The higher the probability that the table is vacant, the higher the vacancy value is calculated. For example, when the user determination processing unit 112 determines that the usage area AR1 does not include a user based on the captured image TP1, the calculation processing unit 115 calculates the first vacancy value of the corresponding table as "0.6". Also, for example, when the control unit 11 determines that the usage area AR1 includes a user item (smartphone 31) based on the captured image TP1, the calculation processing unit 115 calculates the first vacancy value of the corresponding table as "0.1".

[0085] For example, when the food ordered by the user is served, the calculation processing unit 115 calculates the second vacancy value of the corresponding table as "0.7". For example, when the payment of the usage fee corresponding to the user is completed, the calculation processing unit 115 calculates the second vacancy value of the corresponding table as "0.8".

[0086] Then, the vacancy determination processing unit 114 may determine the usage status of the table based on the first vacancy value and the second vacancy value calculated by the calculation processing unit 115. For example, the vacancy determination processing unit 114 determines that the table is vacant when the sum of the first vacancy value and the second vacancy value is "1.0" or more, and determines that the table is occupied when the sum is "0.9" or less.

[0087] Here, for example, if a plate without food (empty plate) is placed on a table, the staff must clear the plate, clean the table and chair, and perform other tasks (setting) to guide the next user to the table. If the setting of the table is not complete, the staff cannot guide the next user to the table. In another embodiment, the vacant seat determination processing unit 114 may determine that the table is a preparation seat when the use area AR1 does not include a user, the current time has passed the end time of use, the product ordered by the user has been provided, or the payment of the usage fee has been completed, and the table in the use area AR1 includes an empty plate. If the table is determined to be a preparation seat, the control unit 11 sets the table to a state (preparation seat) in which guidance or reservation by a user is not permitted. Note that the vacant seat determination processing unit 114 may determine that the setting is complete when the use area AR1 subsequently does not include an empty plate and an installation item (e.g., a chopstick case, an ordering terminal) is placed in a predetermined position in the use area AR1 (table), and switch the table from a preparation seat to an vacant seat.

[0088] In this case, the calculation processing unit 115 may calculate the vacant seat value of the prepared seat as "0.5" when the vacant seat value of the vacant table is "1." As a result, for example, when there are two prepared seats in a restaurant, the calculation processing unit 115 calculates the total number of vacant seats of the two prepared seats as "1."

[0089] As another embodiment of the present invention, the output processing unit 116 may notify the store terminal of information indicating that the user has left the table when the user is no longer included in the usage area AR1, that is, when the user leaves the chair at the table. Also, the output processing unit 116 may change the display mode (blinking display, lighting display, etc.) of the table from which the user has left the floor layout (table arrangement diagram) displayed on the store terminal.

[0090] In addition, the output processing unit 116 may change the display mode (blinking display, lighting display, etc.) of tables determined to be vacant in the floor layout. This allows the restaurant staff to grasp the vacancy status of tables in real time.

[0091] In addition, the information processing system of the present invention may be considered as a system including an image acquisition processing unit that acquires an image of a usage area including a usage object used by a user in a facility, a facility acquisition processing unit that acquires facility information including at least one of payment information regarding payment of the facility usage fee, order information regarding orders of products provided by the facility, and reservation information regarding reservations for the facility, and a vacancy determination processing unit that determines whether the usage object is vacant or not based on the image acquired by the image acquisition processing unit and the facility information acquired by the facility acquisition processing unit.

[0092] Furthermore, the information processing method of the present invention may be regarded as a method executed by one or more processors, which includes an image acquisition step of acquiring an image of an area of ​​use including a target seat used by a user in a facility; a facility acquisition step of acquiring facility information including at least one of payment information regarding payment of the facility usage fee, order information regarding orders for products offered by the facility, and reservation information regarding reservations for the facility; and a vacancy determination step of determining whether the target seat is vacant or not based on the image acquired by the image acquisition step and the facility information acquired by the facility acquisition step. [Explanation of symbols]

[0093] 1: Information processing equipment 2: Facility terminal 3: Imaging device 100: Information processing systems 111: Image acquisition processing unit 112: User determination processing unit 113: Facility Acquisition Processing Unit 114: Vacancy determination processing unit 115: Calculation processing unit 116: Output processing section 211: Reservation processing unit 212: Order Processing Unit 213: Payment processing unit

Claims

1. an image acquisition processing unit that acquires an image of a usage area including a usage object used by a user in a facility; a user determination processing unit that, when a staff member of the facility is included in the usage area in the captured image acquired by the image acquisition processing unit, determines whether or not the user is included in the usage area excluding the staff member; a vacancy determination processing unit that, when it is determined by the user determination processing unit that the user is not included in the usage area, determines whether the usage target is vacant or not based on at least one of payment information related to payment of the usage fee of the facility, order information related to orders of products provided by the facility, reservation information related to reservations for the facility, and item information related to items included in the usage area; and An information processing system comprising:

2. The vacant seat determination processing unit determines that the use target is present when the user determination processing unit determines that the user is included in the use area. The information processing system according to claim 1 .

3. the user determination processing unit determines that the user is included in the usage area when a person included in the usage area in the captured image stays within a predetermined range from the usage target for a predetermined time or when the person is seated in a chair. The information processing system according to claim 1 .

4. the user determination processing unit, when the captured image includes a person who does not stay for a predetermined period of time, a person who has a pre-registered terminal, or a person who is wearing pre-registered clothing, determines that the person is the staff member and excludes the person from the determination of the presence or absence of the user. The information processing system according to claim 1 .

5. An image acquisition step of acquiring an image of a use area including a use object used by a user in a facility; a user determination step of determining, when the staff of the facility are included in the usage area in the captured image acquired in the image acquisition step, whether or not the user is included in the usage area excluding the staff; a vacancy determination step of determining whether the target seat is vacant or not based on at least one of payment information related to payment of the facility usage fee, order information related to orders for products provided by the facility, reservation information related to reservations for the facility, and item information related to items included in the usage area when it is determined in the user determination step that the user is not included in the usage area; An information processing method executed by one or more processors.

6. An image acquisition step of acquiring an image of a use area including a use object used by a user in a facility; a user determination step of determining, when the staff of the facility are included in the usage area in the captured image acquired in the image acquisition step, whether or not the user is included in the usage area excluding the staff; a vacancy determination step of determining whether the target seat is vacant or not based on at least one of payment information related to payment of the facility usage fee, order information related to orders for products provided by the facility, reservation information related to reservations for the facility, and item information related to items included in the usage area when it is determined in the user determination step that the user is not included in the usage area; An information processing program for causing one or more processors to execute the above.

Images