Product sales system, product sales method, and product sales program

Abstract

In order to identify a plurality of customers making up a group and to handle purchasing of a product by the group, this product sales system comprises: a customer detection sensor that tracks a customer who has entered a store; a product detection sensor that detects a product to be purchased by the customer; a management device that uses the customer detection sensor and the product detection sensor to manage and associate the customer and the product to be purchased by the customer; a checkout device by which the customer performs payment for the product associated with the customer by the management device; and a paying customer detection sensor that detects a customer in a checkout area in which the checkout processing is installed. When the paying customer detection sensor detects a plurality of customers in the checkout area, the management device accepts an operation of selecting a product, from among products associated with each of the other customers excluding the paying customer among the plurality of customers, by the paying customer who pays for the product, and includes the product selected on the basis of the operation as a target of payment to be performed by the paying customer at the checkout device.

Description

Product sales system, product sales method, and product sales program

[0001] The present disclosure relates to a product sales system, a product sales method, and a product sales program for selling products in a store.

[0002] In recent years, labor-saving stores and unmanned stores have emerged to reduce the number of store clerks. A customer who enters an unmanned store can select a product at a product sales area, complete the payment for the product using a checkout device, and then leave the store. Products are displayed and sold on a plurality of product shelves installed in the store. Using cameras that capture images of the inside of the store and sensors installed on the product shelves, the type and quantity of products taken out from the product shelves and purchased by the customer are automatically detected. Based on the detection results of the customer and the products, the price of the product purchased by the customer is specified, and a checkout process in which the checkout device receives the product price from the customer is executed.

[0003] In an unmanned store, a customer is associated with the product the customer purchases, and each customer pays for the product separately. However, there are cases where a plurality of customers visit the store as a group, and one customer wants to collectively pay for all products purchased by the group. In order to cope with such a case, Patent Document 1 discloses a system that detects a plurality of customers in a checkout area and can collectively settle the prices of products associated with each customer. In this system, customers can also be grouped based on the customers' behavior, such as the distance between customers and whether they are acting together.

[0004] Japanese Unexamined Patent Publication No. 2023-14207

[0005] In the above-mentioned conventional technology, when detecting a plurality of customers constituting a group in the checkout area, there is a possibility that customers unrelated to the group may be detected. Even when grouping is performed based on customer behavior, there is a possibility that unrelated customers may be mistakenly regarded as a group.

[0006] The present disclosure has been made in view of the above-mentioned problems related to the conventional technology, and one of the objects thereof is to provide a product sales system, a product sales method, and a product sales program that can identify a plurality of customers constituting a group and respond to product purchases by the group.

[0007] The product sales system relating to this disclosure includes a customer detection sensor that tracks customers entering the store, a product detection sensor that detects products purchased by the customer, a management device that uses the customer detection sensor and the product detection sensor to associate and manage customers with the products purchased by the customer, a payment device in which the customer makes a payment for products associated with the customer by the management device, and a customer detection sensor that detects customers in the payment area where the payment process is set up. If the customer detection sensor detects multiple customers in the payment area, the management device accepts an operation from the customer making the payment to select products from among the products associated with each of the other customers excluding the customer making the payment, and includes the products selected based on the operation as the items to be paid for by the customer making the payment.

[0008] In the above configuration, the management device may display information on a display device indicating each of the multiple customers detected in the predetermined area, and accept an operation to select the customer to pay from among the multiple customers.

[0009] In the above configuration, the payment area where the payment device is installed is designated as the predetermined area, and the system further includes a payment area gate for entering the payment area from the merchandise sales area, and the management device may lock the payment area gate after the multiple customers have passed through the payment area gate and entered the payment area, in order to prevent other customers from entering the payment area.

[0010] In the above configuration, the store may further include a store exit gate that leads out of the checkout area, and the management device may lock the store exit gate to prevent the customers from leaving the store from the time the customers enter the checkout area until they have finished paying for all the products associated with each of them.

[0011] In the above configuration, if the product selected by the paying customer and associated with a customer other than the paying customer is a product that requires purchase eligibility, the management device may perform a process to confirm whether the paying customer and the other customer are eligible to purchase.

[0012] In the above configuration, the operation by which the paying customer selects products includes the operation by which the management device displays information on a display device indicating products associated with each of the other customers among the multiple customers, excluding the paying customer, and the paying customer individually selects products, and the management device may add the selected products to the items to be paid for by the paying customer.

[0013] In the above configuration, the operation by which the paying customer selects products includes the operation by which the management device displays on a display device information indicating each of the other customers excluding the paying customer and information indicating the products associated with each of the other customers, and the paying customer selects other customers, and the management device may add all products associated with the selected other customers to the items to be paid for by the paying customer.

[0014] In the above configuration, the operation by which the paying customer selects a product includes the operation by which the management device displays information on a display device indicating each of the other customers excluding the paying customer, and after the paying customer receives an operation to select a customer from among the multiple customers displayed on the display device, the management device displays information on the display device indicating the product associated with the selected customer, and the paying customer selects a product from among the multiple products displayed on the display device, and the management device may add the selected product to the items to be paid for by the paying customer.

[0015] In the above configuration, the operation by which the paying customer selects products includes an operation in which the paying customer selects all products associated with each of the other customers excluding the paying customer, and the management device may add all products associated with the multiple customers to the items to be paid for by the paying customer.

[0016] In the above configuration, when the payment by the paying customer is completed and another customer among the multiple customers other than the paying customer makes a payment, the management device may exclude the paid-for items from the items associated with that customer, and make the remaining unpaid items subject to payment by the other customer.

[0017] In the above configuration, the management device may display information indicating settled items and information indicating unsettled items on a display device in a manner that allows for distinction between them.

[0018] In the above configuration, the management device may display information on the display device indicating that the customer has paid for the paid-for items.

[0019] In the above configuration, the management device may detect when goods have been exchanged between the first customer and the second customer, and manage information regarding the first customer, the second customer, and the exchanged goods.

[0020] In the above configuration, the management device may associate the delivered goods with the second customer and include them in the items to be settled by the second customer.

[0021] In the above configuration, if the management device detects the second customer in the predetermined area where the settlement process is performed, but does not detect the first customer, it may display information on the screen of the display device that shows the items to be settled by the second customer indicating that the delivered goods handed over by the first customer are included.

[0022] In the above configuration, the management device may display an image on the screen of the display device showing the first customer handing over the goods to the second customer.

[0023] In the above configuration, the management device may display an image of the first customer on the screen of the display device.

[0024] The product sales method relating to this disclosure is a product sales method performed by a device, and includes the steps of: tracking a customer who has entered the store using a customer detection sensor; detecting the product to be purchased by the customer using a product detection sensor; managing the customer in association with the product purchased by the customer; detecting a customer in a payment area where a payment device is installed using a payment customer detection sensor; and, when multiple customers are detected in the payment area by the payment customer detection sensor, accepting an operation from the payment customer to select a product from among the products associated with each of the other customers excluding the payment customer, and including the selected product as the subject of payment by the payment customer at the payment device.

[0025] The product sales program relating to this disclosure is a product sales program recorded on a computer-readable non-temporary recording medium, which causes the computer to perform the following steps: tracking customers who enter the store using a customer detection sensor; detecting products to be purchased by the customer using a product detection sensor; managing the relationship between the customer and the products purchased by the customer; detecting customers in the payment area where the payment device is installed using a payment customer detection sensor; and, when the payment customer detection sensor detects multiple customers in the payment area, accepting an operation from the payment customer to select products from among the products associated with each of the other customers excluding the payment customer, and including the selected products as the target of payment by the payment customer at the payment device.

[0026] The product sales system relating to this disclosure includes a customer detection sensor that tracks customers entering the store, a product detection sensor that detects products purchased by the customer, a management device that associates customers with the products purchased by the customer using the customer detection sensor and the product detection sensor, a payment device that allows the customer to pay for the products associated with the customer by the management device, and a detection device that detects multiple customers who come to the store as a group of group members. The management device manages the products purchased by each of the multiple group members constituting the group as products purchased by the group, and when one of the group members makes a payment at the payment device, the products purchased by the group are the subject of the payment.

[0027] In the above configuration, the detection device includes an authentication device for authenticating store members who have been registered in advance, and a camera for capturing images of customers entering the store from the store entrance. The management device may associate customers who enter the store between the time the store member is authenticated by the authentication device and the time the store member enters the store with the store member as group members constituting a group.

[0028] In the above configuration, the detection device includes an authentication device for authenticating store members who have been registered in advance, and a camera for capturing images of customers entering the store from the store entrance. The management device may associate customers who enter the store between the time a store member is authenticated by the authentication device and the time another store member is authenticated by the authentication device with the store member, as group members constituting a group.

[0029] In the above configuration, the detection device includes an authentication device for authenticating store members who have been registered in advance, a store entrance gate that is normally locked to restrict entry to store members and is unlocked when a store member is authenticated by the authentication device, and a camera that captures images of customers entering the store through the store entrance gate. The management device may associate customers who enter the store during a predetermined period of time when a store member is authenticated by the authentication device and the store entrance gate is unlocked with the store member as group members constituting a group.

[0030] In the above configuration, the authentication device may authenticate store members by facial recognition.

[0031] In the above configuration, the authentication device may restrict the group members who can settle payments for group purchases to store members.

[0032] In the above configuration, the detection device includes a camera that captures images of customers entering the store from the entrance, and a selection device that accepts an operation to select group members from among a plurality of customers captured by the camera. The management device may display images of the plurality of customers entering the store captured by the camera on the selection device, and associate the customers selected by operating the selection device with group members that constitute a group.

[0033] In the above configuration, the detection device includes a camera that captures images of customers entering the store from the entrance, and a selection device that accepts an operation to select group members from among a plurality of customers captured by the camera. The management device may display images of a plurality of customers entering the store, captured by the camera, on the selection device from the time the selection device starts the group member selection operation until a predetermined time has passed, and associate the customers selected by operating the selection device with group members that constitute the group.

[0034] In the above configuration, the detection device includes a camera that captures images of customers entering the store from the store entrance, and an operating device including an operator installed near the entrance. The management device may associate multiple customers captured by the camera as group members constituting a group, based on the operation performed by the operator.

[0035] In the above configuration, the management device may associate a customer who operates the push-type control with a customer who enters the store while the customer is performing the push operation of the control, as group members constituting a group.

[0036] In the above configuration, the management device may associate a customer who operates the control device with customers who entered the store between the time the customer performed the operation to start grouping and the time the customer performed the operation to end grouping, as group members constituting the group.

[0037] In the above configuration, the management device may associate the customer who operates the control device with the customers who enter the store as designated by the customer operating the control device, as group members constituting a group.

[0038] In the above configuration, the management device may manage information regarding the delivered goods when it detects that goods have been exchanged between a group member and a customer who is not a group member.

[0039] In the above configuration, the store may further include a lockable store exit gate to restrict customers from leaving the store, and the management device may lock the store exit gate so that group members cannot leave the store until the payment for the goods associated with the group has been completed.

[0040] In the above configuration, the store further includes an exit gate that is normally locked to restrict customers from leaving the store, and is unlocked when customers who have finished paying at the payment device and customers who are not associated with any products pass through, and the management device may unlock the store exit gate when other group members pass through, provided that at least one of the group members remains inside the store until the payment for the products associated with the group is completed.

[0041] The product sales method relating to this disclosure is a product sales method performed by a device, and includes the steps of: using a detection device to detect multiple customers who have come to the store as a group; associating the detected multiple customers as members of the group; using a customer detection sensor to track each group member; using a product detection sensor to detect the products that each group member will purchase; associating the group members with the products that the group members will purchase, and managing the products that each group member will purchase as products purchased by the group; and when one of the group members makes a payment at a payment device, including the products purchased by the group as the subject of the payment.

[0042] The product sales program according to the present disclosure is a product sales program recorded on a computer-readable non-transitory recording medium, which causes a computer to execute: a step of detecting a plurality of customers who have visited a store in a group using a detection device; a step of associating the detected plurality of customers as group members of the group; a step of tracking each group member individually using a customer detection sensor; a step of detecting products purchased by each group member using a product detection sensor; a step of associating group members with the products purchased by the group members, and managing the products purchased by each group member as products to be purchased by the group; and a step of setting the products to be purchased by the group as targets of the payment when one of the group members performs payment with a payment apparatus.

[0043] According to the product sales system, product sales method and product sales program according to the present disclosure, it is possible to identify a plurality of customers constituting a group and handle product purchases by the group.

[0044] FIG. 1 is a diagram for explaining a product sales method performed by a product sales system for individual customers in an unmanned store. FIG. 2 is a schematic diagram for explaining an outline of a method for selling products to a group. FIG. 3 is a block diagram showing a configuration example of the product sales system. FIG. 4 is a diagram for explaining a grouping method using an authentication device. FIG. 5 is a diagram for explaining a grouping method using a selection device including an operating element. FIG. 6 is a diagram for explaining a grouping method using a selection device including an operation unit and a display unit. FIG. 7 is a diagram for explaining a method of excluding some products from products purchased by the group. FIG. 8 is a diagram for explaining a grouping method in a payment area. FIG. 9 is a diagram showing an example of a screen displayed on an operation terminal of a payment device during payment processing. FIG. 10 is a diagram for explaining another grouping method in the payment area. FIG. 11 is a diagram for explaining still another grouping method in the payment area. FIG. 12 is a diagram for explaining a method in which a management device manages products transferred between customers. FIG. 13 is a diagram for explaining an example of payment processing including transferred products.

[0045] Hereinafter, the product sales system, product sales method, and product sales program according to the present disclosure will be described with reference to the accompanying drawings. The product sales system according to the present disclosure can be used in various forms of stores, and in the present embodiment, a case where the system is used in an unmanned store will be described as an example.

[0046] The product sales system can collectively process products purchased by a group in an unmanned store targeting a group consisting of a plurality of customers. After describing a product sales method in an unmanned store performed for a single customer, that is, a customer who does not form a group, a product sales method performed for a group consisting of a plurality of customers will be described.

[0047] [Method of Selling Products to Single Customers] Figure 1 is a diagram for explaining a product sales method performed by a product sales system in an unmanned store targeting a single customer. The unmanned store is not limited to a completely unmanned store, and may be a labor-saving store. The product sales system includes a camera 10, a sensor unit 20, a management device 100, and a checkout device 200.

[0048] As shown in Figure 1, the management device 100 detects a customer 800 using the camera 10 (S1). The management device 100 uses the sensor unit 20 to detect a product 700 purchased by the customer 800 (S2).

[0049] The camera 10 is an example of a customer detection sensor that detects a customer 800 in a store. The camera 10 detects the position of a customer 800 moving in the store. In addition to the position of the customer 800, the camera 10 may detect the movement of the customer 800 and the movement of the hands of the customer 800. The type of the camera 10 is not particularly limited. For example, the camera 10 capable of detecting the customer 800 using electromagnetic waves such as visible light, infrared light, and laser may be used. Specifically, a visible light camera, an infrared light camera, a TOF (Time of Flight) camera, a depth camera, a LiDAR (Light Detection and Ranging) camera, or the like may be used as the camera 10. One or more cameras 10 are provided corresponding to each of a plurality of product shelves 600 installed in the store. The management device 100 can detect the customer 800 looking around the product shelves 600 and track the customer 800 moving in the store.

[0050] Sensor unit 20 is an example of a product detection sensor for detecting products 700 purchased by customer 800. Sensor unit 20 includes a sensor for detecting the name and quantity of products 700 purchased by customer 800. The type of sensor included in sensor unit 20 is not particularly limited. Any sensor capable of detecting the presence or absence of each product 700 arranged on the product shelf 600 is acceptable. For example, a sensor capable of detecting the presence or absence of product 700 based on weight changes may be used. Alternatively, a sensor capable of detecting the presence or absence of product 700 by irradiating or transmitting / receiving electromagnetic waves such as infrared light, visible light, laser, or ultrasound may be used. Alternatively, a sensor that detects product 700 using a tag attached to product 700 with short-range wireless technology such as NFC (Near Field Communication) or RFID (Radio Frequency Identifier) ​​may be used. Alternatively, a camera-type sensor that detects product 700 by analyzing captured images may be used.

[0051] For example, in a product shelf 600 in which multiple shelves arranged vertically are each divided into multiple rows, and products 700 are displayed in the front-to-back direction in each row, a sensor unit 20 including a weight sensor is provided corresponding to each row. Note that the products 700 are not limited to being distinguished by name, but may also be distinguished by other information such as identification information to uniquely identify each product, but in this embodiment, the product names will be used as the basis for explanation.

[0052] Multiple types of products sold by the store are displayed in a front-to-back direction on each shelf 600, on each shelf and in each row. The same product 700 is arranged in a row, and a sensor unit 20 is provided corresponding to each row. Therefore, the management device 100 can detect when a customer 800 takes a product 700 from the shelf 600. From the position of the sensor unit 20 that detected the weight change, the management device 100 can identify which shelf 600, which shelf, and which row the product 700 was taken from.

[0053] The management device 100 manages information about the products 700 sold in the store, such as the name (identification information), price, and weight per product of each product 700 displayed on each shelf and row of each product shelf 600. Therefore, based on the detection results from the sensor unit 20, the management device 100 can identify the name, price, and quantity of the products 700 taken from the product shelf 600.

[0054] The management device 100 uses the camera 10 and the sensor unit 20 to detect a customer 800 and the product 700 that the customer 800 has taken from the product shelf 600, and then manages the product 700 and the customer 800 in association with each other.

[0055] Customer 800 moves to the checkout counter with the purchased goods 700 and performs a settlement process at the settlement device 200 to pay for the goods, i.e., the transaction amount between the store and customer 800. When customer 800 starts the settlement process, the management device 100 has already detected the name and quantity of the goods 700 that customer 800 is purchasing. The management device 100 has pre-registered the correspondence between the name of each item 700 and its selling price. The management device 100 can identify the price of the goods 700 from the name of the goods 700 and calculate the transaction amount with customer 800 based on the price and quantity of the goods 700. The management device 100 displays the name and quantity of the goods 700 that customer 800 is purchasing and the transaction amount on the screen of the operation terminal 210 provided on the settlement device 200. Therefore, customer 800 can start the settlement process without having to input the information of the goods 700 to be purchased into the settlement device 200.

[0056] Customer 800 confirms the name and quantity of the goods 700 and the transaction amount displayed on the screen of the settlement device 200 (S3), pays the transaction amount, and the transaction is completed (S4). For example, customer 800 can pay the transaction amount electronically using the electronic payment device 230 of the settlement device 200. Electronic payment is made using a credit card or electronic money. Customer 800 can also pay the transaction amount in cash using the cash processing device 220 of the settlement device 200. When customer 800 deposits cash into the cash processing device 220, the cash processing device 220 identifies and counts the cash and calculates the deposit amount. If the deposit amount exceeds the transaction amount, the cash processing device 220 dispenses change to return to customer 800.

[0057] [Method of selling products to the group] <Overview of the product sales method> Figure 2 is a schematic diagram illustrating the overview of the method of selling products to the group. Figure 2 shows a view from above of a part of an unmanned store, including the store entrance and exit, and the payment area where the payment device 200 is installed. In Figure 2, the camera 10, sensor unit 20, management device 100, etc., described in Figure 1 are omitted.

[0058] The product sales system includes, in addition to the camera 10, sensor unit 20, and payment device 200 described in Figure 1, cameras 30 (30a, 30b) used for group detection, a store entrance gate 41, a payment area gate 42, and a store exit gate 43. The product sales system may also include an authentication device 310 and a selection device 320. The product sales system can handle the sale of products to groups of multiple customers using a method selected from the multiple methods shown in Figure 2.

[0059] The first method involves using a camera 30a installed near the store entrance gate 41 to detect a group of customers 811, 812 at the store entrance (A1). After detecting the group at the store entrance, the management device 100 uses the camera 10 and sensor unit 20, as explained in Figure 1, to track each customer 811, 812 included in the group, i.e., the group members 811, 812 belonging to the group, and identifies the products purchased by the group (A2). The management device 100 associates the products that each group member 811, 812 takes from the product shelf 600 with the group. The management device 100 accepts payment processing at the payment device 200 installed in the payment area to settle the total amount of payment for the products associated with the group (A3).

[0060] The second method involves using a camera 30b installed in the checkout area to detect a group of multiple customers 821, 822 in the checkout area. As explained in Figure 1, the management device 100 uses the camera 10 and sensor unit 20 to track each customer who enters the store from the entrance and to identify the products each customer purchases (B1). The management device 100 associates each customer with the products each customer purchases. Inside the store, a checkout area with a checkout device 200 is provided, separate from the merchandise sales area where merchandise shelves 600 are installed. Customers enter the checkout area from the merchandise sales area by passing through the checkout area gate 42 provided at the entrance to the checkout area. When multiple customers 821, 822 constituting a group enter the checkout area, the management device 100 uses the camera 30b to detect the group (B2). The management device 100 accepts settlement processing at the settlement device 200 to settle the total amount of money for the goods associated with each group member 811, 812 (B3).

[0061] The product sales system can sell products to groups using either the first or second method. The product sales system can also sell products to groups using both the first and second methods. For example, if the first method is used to detect multiple customers who come to the store together as a group, it may not be able to handle situations where other group members arrive later while the group members are already shopping. In such cases, the second method can be used to detect the group in the checkout area.

[0062] <Example of Product Sales System Configuration> Figure 3 is a block diagram showing an example of the configuration of a product sales system. Camera 30 is an example of a group detection sensor that detects groups of multiple customers at the entrance or checkout area of ​​a store. In Figure 3, camera 10, which is a customer detection sensor, and camera 30, which is a group detection sensor, are shown separately, but a single camera may also perform both the functions of a customer detection sensor and a group detection sensor.

[0063] The management device 100 is configured to include a computer device comprising a communication unit, a control unit, and a storage unit. The management device 100 may also include an operation unit and a display unit. For example, a program corresponding to the control unit is pre-stored in the storage unit or a dedicated storage device, and the functions and operations of the control unit are realized by the execution of this program by hardware such as a CPU. The functions and operations of the management device 100 and the product sales system described in this embodiment are realized by the control unit executing control based on information input and output using the operation unit and the display unit, information transmitted and received via the communication unit, information stored in the storage unit, etc. In this embodiment, the functions and operations of the management device 100 and the product sales system realized by the control unit of the management device 100 may be simply described as the functions and operations of the management device 100.

[0064] The management device 100 is connected to external devices in a communicative manner. The external devices include cameras 10 and 30, a sensor unit 20, a payment device 200, an authentication device 310, and a selection device 320. The configuration of the management device 100 is not particularly limited as long as the functions and operations described in this embodiment can be realized. For example, the management device 100 may be composed of multiple computer devices, or it may be configured to include a cloud-type server.

[0065] The authentication device 310 is an authentication device installed near the store entrance gate 41. The authentication device 310 is used when a store restricts entry to members of the store. In this case, the management device 100 normally keeps the store entrance gate 41 locked and unlocks the store entrance gate 41 only when a member of the store is authenticated by the authentication device 310. The authentication method is not particularly limited. Password input, reading information from a membership card that stores authentication information in a memory element, biometric authentication such as facial recognition, and other conventionally known authentication methods can be employed, and the authentication device 310 compatible with the adopted authentication method should be used. Note that in some cases, a store may be open to anyone, the product sales system may not include the authentication device 310, and the store entrance gate 41 may be open at all times.

[0066] The selection device 320 is installed near the store entrance. The selection device 320 is used to select customers who form a group from among multiple customers passing through the store entrance gate 41, but the selection method will be described later. Depending on the group detection method adopted by the store, the product sales system may not include the selection device 320.

[0067] The settlement device 200 includes an operation terminal 210, a cash processing device 220, and an electronic payment device 230. The operation terminal 210 is configured to include, for example, a touch panel type liquid crystal display. The operation terminal 210 functions as an operation unit for inputting various information related to product transactions with customers. The operation terminal 210 also functions as a display unit for displaying various information related to product transactions. For example, when a customer starts the settlement process, a transaction screen containing information such as the name, quantity, and price of the products being traded is displayed on the screen of the operation terminal 210. Customers can operate the operation terminal 210 as needed to modify, add, delete, or otherwise perform operations on the information displayed on the screen. Depending on the operation, permission from a store employee may be required, or the operation may need to be performed by a store employee. In this case, the process can proceed by having a store employee waiting in the back room of the unmanned store provide face-to-face assistance, or by having a store employee located away from the unmanned store provide online assistance.

[0068] The cash processing device 220 can process the receipt of cash paid by the customer and the dispensing of change to be returned to the customer. The electronic payment device 230, when a customer requests electronic payment, obtains the necessary information from the electronic payment medium held by the customer and performs processing related to the electronic payment.

[0069] The store entrance gate 41 is the gate for entering the store. If entry is restricted to store members, an entry restriction device equipped with a locking mechanism that can be controlled by the management device 100 is used as the store entrance gate 41. As mentioned above, the store entrance gate 41 may also be used in an unlocked state, allowing passage at all times.

[0070] The checkout area gate 42 is a gate for entering the checkout area from the merchandise sales floor. For example, an entry restriction device equipped with a lock mechanism that can be controlled by the management device 100 is used as the checkout area gate 42. The lock mechanism of the checkout area gate 42 is normally unlocked. After all customers in a group have entered the checkout area through the checkout area gate 42, the management device 100 locks the gate by locking the lock mechanism of the checkout area gate 42 to prevent customers outside the group from entering the checkout area. Locking the checkout area gate 42 also prevents customers who have started checking out in the checkout area from returning to the merchandise sales floor. For example, the management device 100 can receive an operation from the operation terminal 210 of the checkout device 200 to confirm that all customers in the group have entered the checkout area, and then lock the checkout area gate 42 based on that operation. Depending on the store's operation, the checkout area gate 42 may not be locked and may be used in a state where it can be passed through at all times.

[0071] The store exit gate 43 is a gate for exiting the store. For example, an entry restriction device equipped with a locking mechanism that can be controlled by the management device 100 is used as the store exit gate 43. The locking mechanism of the store exit gate 43 is normally in a locked state. Once payment is completed at the payment device 200, the management device 100 unlocks the store exit gate 43, allowing the customer to exit the store. However, depending on the store's operation, the store exit gate 43 may be used in a state where it is not locked and can be passed through at all times.

[0072] Furthermore, if the store exit gate 43 is locked to restrict customers who can leave the store to those who have completed payment, for customers who leave the store without purchasing any goods, the management device 100, which detected this customer with camera 30b, can unlock the store exit gate 43 after confirming that there are no goods associated with the customer based on the detection results from camera 10 and sensor unit 20. For group members of a group detected at the store entrance by the first method, even if there are goods associated with the group member based on the detection results from camera 10 and sensor unit 20, the management device 100 can unlock the store exit gate 43 on the condition that other group members belonging to the same group remain inside the store.

[0073] <Grouping at the Store Entrance> The first method, as explained in Figure 2, is described below. In the first method, a grouping process is performed at the store entrance to detect a group consisting of multiple customers and associate the customers constituting this group as group members.

[0074] (Grouping by Authentication Device) Figure 4 is a diagram illustrating a method for detecting groups using an authentication device 310. This method is used in membership-based stores where the authentication device 310 is used to restrict entry to store members.

[0075] Figure 4(a) is a schematic diagram of the area near the entrance of the store viewed from above. When a store member 840 uses the store together with other customers 841 and 842, the member 840 first inputs authentication information into the authentication device 310 and the authentication process is performed. After confirming that the member is a member through the authentication process, the management device 100 unlocks the store entrance gate 41, which is normally locked. As shown in Figure 4(a), the store member 840 opens the store entrance gate 41 and allows customers 841 and 842, who are using the store with him, to enter first. Based on the images captured by the camera 30a, the management device 100 recognizes each of the entering customers 841 and 842 and starts tracking them using the cameras 10 installed inside the store.

[0076] Store member 840 enters the store last, after customers 841 and 842 have entered, and closes the store entrance gate 41. The management device 100 recognizes that store member 840 has entered the store based on the image captured by camera 30a. The member information managed by the management device 100 includes the facial image of each member. The management device 100 performs facial recognition processing based on the image captured by camera 30a and recognizes that store member 840 has entered the store. After store member 840 has entered the store, the management device 100 starts tracking store member 840 using cameras 10 installed inside the store and locks the store entrance gate 41.

[0077] The management device 100 associates the store member 840, authenticated by the authentication device 310, with the customers 841 and 842, captured by camera 30a, between the time the store member 840 is authenticated and the lock on the store entrance gate 41 is released, and the store member 840's entry is confirmed by facial recognition. The customers 841 and 842 and the store member 840 are managed as group members constituting a group.

[0078] Figure 4(b) shows an example of information managed by the management device 100. As shown in Figure 4(b), the management device 100 registers the facial images of other customers 841 and 842 in the group member field, associating them with the membership number of store member 840.

[0079] The management device 100 registers information about the products that store member 840 has taken from the product shelf 600, as detected using the camera 10 and sensor unit 20, in the product items shown in Figure 4(b). Similarly, the management device 100 registers information about the products that group members 841 and 842 have taken from the product shelf 600, as detected using the camera 10 and sensor unit 20, in the product items.

[0080] The products purchased by store member 840 and the products purchased by other group members 841 and 842 are managed in association with store member 840's membership number. For example, store member 840's membership number is used as the group's identification number.

[0081] When store member 840 performs a settlement process at the settlement device 200, the settlement process is performed on all products registered in the product field in association with store member 840's membership number. For example, the management device 100 uses a camera 30b installed in the settlement area to perform facial recognition on store member 840 and starts the settlement process for products associated with store member 840. In this way, the sale of products to a group including store member 840 and other group members 841 and 842 is realized. Even if group members 841 and 842 are not store members, they can still purchase products at the store together with store member 840.

[0082] The payment terminal 200 may not restrict customers to being store members 840, and may allow group members 841 and 842 to make payments. Alternatively, the payment terminal 200 may be restricted to being store members 840. If it is restricted to store members 840, information indicating this should be displayed on the screen of the payment terminal 210 of the payment terminal 200.

[0083] When shopping in a group, the group will be formed when the member enters the store last. By informing the store members in advance of this, as described above, it is possible to group the customers detected by the camera 30a between the time the authentication process for the store member 840 is performed and the store entrance gate 41 opens, and the time the store member 840 enters the store. For example, information explaining the grouping method can be displayed on a display unit on the authentication device 310 or on a display device installed near the authentication device 310, and the store member 840 can be notified.

[0084] The grouping of store members may be carried out by other means. The management device 100 authenticates the store members and unlocks the store entrance gate 41. When the store entrance gate 41 closes, it locks the store entrance gate 41 again. The next store member to visit the store is authenticated by the authentication device 310, and the store entrance gate 41 is unlocked again.

[0085] Using this, for example, the management device 100 may use the camera 30a to detect and group customers who enter the store between the time the management device 100 authenticates a store member or unlocks the store entrance gate 41 based on the authentication result, and the time the store entrance gate 41 is closed and locked again. Alternatively, for example, the management device 100 may use the camera 30a to detect and group customers who enter the store between the time the management device 100 authenticates a store member or unlocks the store entrance gate 41, and the time the next store member is authenticated.

[0086] (Grouping by Operator) Figure 5 is a diagram illustrating a method for detecting groups using a selection device 320 that includes an operator. This method can be used in both membership-based stores where entry is restricted to store members and stores where anyone can enter without being restricted to store members. In stores that do not employ a membership system, the store entrance gate 41 is always openable and closable, and customers can enter at any time. In the case of membership-based stores, as in the example described in Figure 4, an authentication device 310 can be installed at the store entrance, and according to the authentication result, the normally locked store entrance gate 41 can be unlocked so that only store members can enter.

[0087] Figure 5(a) is a schematic diagram of the area near the entrance of the store, viewed from above. When a group uses the store, one of the group members 851 to 853 performs an operation on the selection device 320 to indicate that the group is using the store. The type and structure of the operator are not particularly limited, as long as the management device 100 can detect that the operator is being operated. For example, toggle switches, locker switches, slide switches, touch switches, etc., can be used as operators. The following explanation will continue using a push-button operator as an example.

[0088] The management device 100 recognizes customer 851 who entered the store and pressed the button on the selection device 320, and customers 852 and 853 who entered the store through the store entrance gate 41 while customer 851 was pressing the button, based on the images captured by the camera 30a, and groups them together. The management device 100 associates the group members 851 to 853, that is, the grouped customers 851 to 853. The management device 100 manages the group members 851 to 853 by assigning a group number to identify each group.

[0089] Figure 5(b) shows an example of information managed by the management device 100. As shown in Figure 5(b), the management device 100 associates group members 851 to 853 with a group number. The management device 100 registers the facial images of group members 851 to 853 and associates them with the group number.

[0090] The management device 100 registers information on the products that each group member 851 to 853 has taken from the product shelf 600, as detected using the camera 10 and sensor unit 20, in the product items shown in Figure 5(b). All products purchased by group members 851 to 853 are managed in association with the group number. The management device 100 manages all products associated with each group member 851 to 853 as products purchased by the group.

[0091] When group members 851 to 853 perform the settlement process at the settlement device 200, the settlement process is performed on all products registered in the product field in association with the group number. For example, the management device 100 recognizes at least one of the group members 851 to 853 using a camera 30b installed in the settlement area and starts the settlement process for the products associated with each group member 851 to 853. Recognition of group members can be performed using facial recognition or the like with captured images. In this way, the sale of products to groups consisting of multiple group members 851 to 853 is realized.

[0092] The method described above groups customers who enter the store while a push-type operator is being operated, i.e., while a push operation is being performed. However, the operator of the selection device 320 may be operated in other ways. For example, a customer who operates the operator of the selection device 320 may be grouped with customers who enter the store between the time the customer initiates the grouping operation using the operator and the time the customer initiates the grouping operation. Specifically, the operator may operate the operator once to start grouping, and then operate the operator once more to end grouping. In this way, the management device 100 can group customers detected by the camera 30a and the customer who operated the operator between the time the grouping start operation is performed and the time the grouping end operation is performed. The selection device 320 may also be configured to include an operator for instructing the start of grouping and a separate operator for instructing the end of grouping. For example, customers detected by the camera 30a and the customer who performed the operation may be grouped between the time the start operator is operated and the time the end operator is operated.

[0093] The selection device 320 may group customers who operate the operator with customers who have performed an operation using the operator to indicate that they are a group member. Specifically, the management device 100 can group customers who operate the operator upon entry with the customer who performed the operation, by having the customer who operated the operator operate the operator once each time a group member enters the store. The selection device 320 may also be provided with a display unit, and the management device 100 may display images of customers who have entered the store as recognized by the camera 30a on the display unit, so that customers who operate the operator can confirm that the displayed image is that of a group member before operating the operator.

[0094] The selection device 320 may be provided with an operator for adding group members and a separate operator for canceling. The management device 100 displays images of customers entering the store to be added to the group based on customer input. If a customer has mistakenly added an unrelated customer to the group and checks the image on the display and operates the cancel operator, the canceled customer can be removed from the group.

[0095] (Grouping by Image) Figure 6 is a diagram illustrating a method for detecting groups using a selection device 320 including an operation unit and a display unit. This method can be used in both membership-based stores where entry is restricted to store members and stores where anyone can enter without being restricted to store members. In the case of stores that do not employ a membership system, the store entrance gate 41 is always in an openable and closable state, and customers can enter at any time. In the case of membership-based stores, as in the example described in Figure 4, an authentication device 310 can be installed at the store entrance, and according to the authentication result, the normally locked store entrance gate 41 can be unlocked so that only store members can enter.

[0096] Figure 6(a) is a schematic diagram of the area near the entrance of the store viewed from above. Group members 861 to 863, who use the store as a group, operate the selection device 320 after entering the store to request that the group use the store.

[0097] When the selection device 320 is used to request a group to use the store, the management device 100 uses the camera 30a to capture an image of the area near the selection device 320, recognizes the faces included in the captured image, and displays each customer's face image on the display unit of the selection device 320, as shown in Figure 6(b).

[0098] The management device 100 captures images of each customer passing through the store entrance gate 41 using the camera 30a and stores and manages them in the memory unit. Using this, the selection device 320 may display images accumulated up to a predetermined time prior to the date and time when a grouping operation was performed on the selection device 320. For example, the selection device 320 may display images of customers captured by the camera 30a within 10 seconds before the operation was performed.

[0099] As shown in Figure 6(b), the screen of the selection device 320 displays images of multiple customers captured by the camera 30a and information requesting the selection of customers to be grouped. Although it is possible that images of customers 864 other than group members 861 to 863 may be displayed on the screen of the selection device 320, any of the group members 861 to 863 can select the images of each of the group members 861 to 863 and press the "finish selection" button on the screen to group only the group members 861 to 863.

[0100] The management device 100 manages the group members 861 to 863 selected by the selection device 320, associating them with the group number assigned to the group. This management is performed as described in Figure 5(b).

[0101] As explained in Figure 5(b), the management device 100 registers information on the products that each group member 861 to 863 has taken from the product shelf 600, as detected using the camera 10 and sensor unit 20, in the product items. All products purchased by group members 861 to 863 are managed in association with the group number. The management device 100 manages all products associated with each group member 861 to 863 as products purchased by the group.

[0102] When group members 861 to 863 perform settlement processing using the settlement device 200, settlement processing is performed for all products registered in the product field in association with the group number. In this way, the sale of products to a group consisting of multiple group members 861 to 863 is realized.

[0103] (Selection of purchased items by a group) In the product sales system, after grouping multiple customers at the store entrance using the authentication device 310, selection device 320, and camera 30a as explained in Figures 4 to 6, it is possible to exclude some of the purchased items from the group and have them paid for separately when the payment processing is performed by the payment device 200.

[0104] Figure 7 is a diagram illustrating a method for excluding certain items from a group's purchase. During settlement processing, the management device 100 displays the screen shown in Figure 7(a) on the screen of the operating terminal 210 of the settlement device 300.

[0105] Figures 4(b) and 5(b) illustrate that the products taken from the shelves 600 by each group member are associated with a membership number or group number used as the group's identification number. In this way, the management device 100 manages the products taken from the shelves 600 by each group member as products purchased by the group. In addition, as explained in Figure 1, the management device 100 also manages each customer in association with the products that each customer takes from the shelves 600. Therefore, as shown in Figure 7(a), the management device 100 can display the products purchased by the group separately for each group member.

[0106] For example, as shown in Figure 7(a), the screen displays an image of each group member, information about the items each group member has taken from the shelves 600, and information to confirm whether or not they wish to pay separately for some of the items purchased by the group. If it is acceptable to pay for all items together, the user operates the operation terminal 210 to indicate that they do not wish to pay separately, and the payment process will begin for all items displayed on the screen in Figure 7(a).

[0107] On the other hand, if there are items that need to be paid for separately, the user can operate the operation terminal 210 to request separate payment, and the screen shown in Figure 7(b) will be displayed on the operation terminal 210. For example, if the group had planned to pay for everything together when entering the store, but later decided to purchase some items separately, they can simply request separate payment for those items.

[0108] When a user requests separate payment for some items on the screen shown in Figure 7(a), the screen on the operation terminal 21 is updated, as shown in Figure 7(b), to display a screen that includes a frame for selecting items to be paid for separately. Group members can use the operation terminal 210 to select the items they want to pay for separately on the screen shown in Figure 7(b). For example, if a group member moves an icon representing an item into the separate payment frame displayed on the screen, as shown by the dashed line in Figure 7(b), the items within the frame will be processed as separate payments. When the user presses the button labeled "Next" on the operation terminal 210 screen, the payment device 200 excludes the items to be paid for separately and begins processing the remaining items. After completing this processing, the payment device 200 then begins processing the items that were selected for separate payment.

[0109] In the example shown in Figure 7, of the five products displayed on the screen shown in Figure 7(a), the payment process is performed on the remaining four products, excluding "cup noodles," which were selected as the product to be paid for separately on the screen shown in Figure 7(b). After this payment process is completed, the payment process for "cup noodles" is performed. However, the order of the payment process is not limited; the payment process for the products to be paid for separately may be performed first, followed by the payment process for the products purchased as a group.

[0110] In this way, the management device 100 accepts an operation to select some products from among the products associated with a group, and causes the settlement device 200 to execute settlement processing for the selected products and settlement processing for the remaining products excluding the selected products. Settlement processing for products purchased as a group and settlement processing for products excluded from the group purchase are executed. This allows for flexible responses to customer requests.

[0111] <Grouping in the Payment Area> The second method, as explained in Figure 2, will now be described. In the second method, multiple customers are grouped together in the store's payment area. In the second method, a group consisting of multiple customers is detected in the payment area where the payment device 200 is installed, and payment processing is performed for this group.

[0112] As shown in Figure 2, the checkout area is isolated from the merchandise area where the shelves 600 are located by a checkout area gate 42. Customers enter the checkout area through the checkout area gate 42 with all the items they wish to purchase in their possession. After the customer enters the checkout area, the management device 100 locks the checkout area gate 42. The store exit gate 43 is also locked, but once the checkout is complete, the management device 100 unlocks the store exit gate 43, allowing the customer to leave the store.

[0113] The settlement area where the settlement device 200 is installed is equipped with a camera 30b capable of capturing images of the entire settlement area. The management device 100 detects when a person enters the settlement area using the camera 30b and starts the process of detecting a group.

[0114] Figure 8 illustrates a method for detecting groups in the payment area. As shown in Figure 8(a), the management device 100 displays the image captured by camera 30b on the screen of the operating terminal 210 of the payment device 200. The image captured by camera 30b of all customers in the payment area is displayed on the screen. The management device 100 determines that the multiple customers in the captured image form a group and recognizes each customer as a group member. Information is displayed on the screen prompting the user to select the customer who will pay for the goods, i.e., the customer who will perform the payment process at the payment device 200, from among the group members in the image.

[0115] When a group member in the checkout area selects a customer to check out using the operation terminal 210, the management device 100 displays the screen shown in Figure 8(b) on the operation terminal 210. The screen displays the products that the management device 100 has detected being taken from the shelves 600 using the cameras 10 and sensor units 20 in the sales area for each group member. The products purchased by each group member are displayed in a way that makes it clear which group member took which product from the shelves 600. In other words, all products that the management device 100 has associated with each group member in the sales area are displayed in correspondence with the associated group member. In addition, the customer selected on the screen shown in Figure 8(a) is displayed in a way that makes it distinguishable from other group members.

[0116] In the example shown in Figure 8(b), information about the products each group member will purchase is displayed, corresponding to the facial image of each group member. The product information includes information indicating the product to be purchased, such as an image showing the product's appearance and the product name, as well as information indicating the product's price. In addition, information indicating that the selected group member who is the paying customer is displayed, corresponding to their facial image.

[0117] As shown in Figure 8(b), information prompting the customer to select the items to be paid for, i.e., the items the customer will pay for, is displayed on the screen. Items that the management device 100 has associated with the customer are already displayed as selected items for payment. In the example shown in Figure 8(b), the items for payment are enclosed in a frame to distinguish them from other items. The customer can select items from the other items displayed on the screen to be paid for together with the items associated with the customer.

[0118] A button is displayed on the screen that allows all items shown on the screen to be selected at once. When a customer uses the operation terminal 210 to select all items at once, all items are selected. When the customer who has finished selecting items presses the button displayed as "Next" on the screen of the operation terminal 210, the management device 100 starts the payment process on the payment device 200, making all items eligible for payment. This allows one customer to pay for the items of all group members who have entered the payment area.

[0119] A paying customer can individually select items to pay for from among the items associated with other group members. For example, the paying customer operates the operation terminal 210 to select items from the list of items displayed as shown in Figure 8(b). The management device 100 adds the selected items to the items to be paid for by the paying customer. The items selected by the paying customer are displayed in a way that distinguishes them from other items not included in the items to be paid for.

[0120] A customer can select group members to pay with from among the other group members. For example, the customer can operate the operation terminal 210 to select group members on the screen shown in Figure 8(b). A group member is selected by pressing the image of the group member on the screen of the operation terminal 210. The management device 100 adds all products associated with the selected group member to the items to be paid for by the customer. All products corresponding to one or more group members selected by the customer are displayed in a way that distinguishes them from other products that are not included in the items to be paid for.

[0121] After selecting items and group members, the customer presses a button on the operating terminal 210 screen to proceed to the next step. In response to this, the management device 100 initiates a settlement process at the settlement device 200, which will settle the items associated with the customer at the merchandise counter and the items selected by the customer in the settlement area. The settlement process begins, where the customer pays for the items they themselves picked up from the shelves 600 at the merchandise counter and for the items selected by the customer in the settlement area from among the items picked up by other group members from the shelves 600 at the merchandise counter.

[0122] Figure 9 shows an example of a screen displayed on the operating terminal 210 of the payment device 200 during the payment process. For example, on the screen shown in Figure 8(b), if the customer selects "sports drink" to be purchased by other group members in addition to "tea" associated with the customer, the payment screen shown in Figure 9(a) will be displayed. The screen displays information about the items purchased by the customer and information about the items for which the customer will pay on behalf of other group members. Information about other group members associated with the items the customer has chosen to pay for may also be displayed on the screen. This information about group members and the information about the customer may be displayed in a way that allows them to be distinguished.

[0123] In the example shown in Figure 9(a), the screen displays the face image of the paying customer, along with the product name and price associated with the paying customer. It also displays the product name and price of the item the paying customer has chosen to pay for on behalf of other group members, along with the face image of the group member associated with that item. The face image and item of the paying customer are clearly distinguishable from the face images and items of the other group members.

[0124] The customer checks the payment screen shown in Figure 9(a) and presses the payment button to begin payment. As explained in Figure 1, the customer pays the transaction amount using the cash processing device 220 or electronic payment device 230 of the payment device 200 to complete the transaction.

[0125] In the example shown in Figure 9(a), once a customer completes their payment, the payment for the "sports drink" item, which was associated with other group members, is recorded as having been paid by the customer. Therefore, the management device 100 updates the information on the items associated with other group members.

[0126] As a result, when a group member whose two products, "instant noodles" and "sports drink," were associated with the screen shown in Figure 8(b) proceeds with payment, the payment screen will display only the unpaid product, "instant noodles," as shown in Figure 9(b). Because some of the products associated with this group member by the management device 100 have been paid for by other group members, the payment process is performed only on the remaining unpaid products. After confirming the payment screen shown in Figure 9(b), the group member can press the payment button and pay the price of the "instant noodles" as the transaction amount at the payment device 200, thereby completing the transaction.

[0127] In the example shown in Figure 9(b), the management device 100 displays information only on unsettled items, excluding settled items, from among the multiple items associated with the group member. However, as shown in Figure 9(c), information on settled items may also be displayed on the screen, or information indicating the customer who settled the payment for settled items may be displayed. In the example shown in Figure 9(c), the product name and price of the settled item, information indicating that it has been settled, and a facial image of the customer who made the payment are displayed. Information on unsettled items and information on settled items are displayed in a way that allows for distinction.

[0128] In the example shown in Figure 9, the paying customer pays for some of the items associated with other group members, so the group members are then processed to pay for the remaining items. However, if the paying customer pays for all the items associated with the group members, the processing for those group members is not performed.

[0129] Furthermore, in the example shown in Figure 8, since there are four group members, after the paying customer and the group members who have paid for some of the items have completed the payment process as explained in Figure 9, the payment process for the remaining two group members is then carried out.

[0130] The management device 100 determines, each time any of the group members detected in the settlement area complete the settlement process, whether or not there are any unsettled items among the items associated with each group member. If there are any unsettled items, the settlement device 200 performs the settlement process for the unsettled items and the group members associated with those items.

[0131] Once all items associated with a group member who has entered the checkout area have been paid for, the management device 100 unlocks the store exit gate 43. This allows the group member to leave the store. In this way, after multiple customers constituting a group have entered the checkout area, the store exit gate 43 is locked until all items associated with each group member have been paid for, preventing all group members from leaving the store even if there are unpaid items. However, depending on the store's operation, it is also possible to configure the management device 100 to unlock the store exit gate 43 once one of the group members has completed the checkout process, allowing group members to leave the store in order as they complete their checkout.

[0132] Figure 8 illustrates an example where, after a customer is selected from multiple group members who have entered the checkout area, a list of items purchased by each group member is displayed, and the customer selects the items to be added to the checkout. However, the customer may also select other group members first.

[0133] For example, after a customer is selected on the screen shown in Figure 8(a), the management device 100 displays the screen shown in Figure 10(a) on the operating terminal 210, prompting the customer to select other group members to pay with. The screen displays an image of all group members, information indicating the group member who is the customer, and information prompting the customer to select other group members. When the customer operates the operating terminal 210 to select other group members, the selected group members are displayed in a way that distinguishes them from the other group members. In the example shown in Figure 10(a), "Customer" is displayed corresponding to the image of the customer, and "Pay Together" is displayed corresponding to the group members selected by the customer. After selecting one or more group members, when the customer presses the button displayed as "Next" on the operating terminal 210, the management device 100 displays the screen shown in Figure 10(b). The screen displays information indicating the group members selected by the customer. Additionally, information is displayed for each group member, indicating the products associated with that group member, that is, information showing the products that each group member purchases.

[0134] In the example shown in Figure 10(b), the facial images of each group member selected by the customer, along with the product names and prices of the items associated with each group member, are displayed. Similar to the operation described in Figure 8(b), the customer can select all items at once by pressing the "Select All Items" button on the screen shown in Figure 10(b), select a group member and select all items associated with that group member at once, or select items individually. On the screen, the items selected by the customer are displayed in a way that makes them distinguishable from other items. The screen shown in Figure 10(b) is an example where the customer has selected "sports drink," similar to the examples described in Figures 8(b) and 9.

[0135] Once a customer has finished selecting the items they wish to purchase and the items they will pay for on behalf of other group members, and has pressed the button on the operating terminal 210 screen to proceed to the next step, the payment process will begin, as described above, for the items the customer has decided to purchase at the merchandise counter and the items associated with other group members that the customer selected by operating the operating terminal 210 in the payment area.

[0136] The following examples were explained: one in which the customer is selected on the screen shown in Figure 8(a), and then the customer selects group members and products on the screen shown in Figure 8(b); and another in which the customer first selects other group members on the screen shown in Figure 10(a), and then selects products on the screen shown in Figure 10(b). In addition, the selection of the customer and the selection of group members and products may be performed on the same screen.

[0137] For example, the management device 100 recognizes group members in the payment area based on images captured by the camera 30b and displays information indicating each group member and the products associated with each group member on a single screen, as shown in Figure 11. On this screen, the management device 100 accepts the selection of the customer and the selection of products for which the customer will pay on behalf of other group members, and then executes the payment process in the same manner as in the example described above.

[0138] In addition, in the examples explained in Figures 10 and 11, as in the examples explained in Figures 8 and 9, if the customer pays for all the items, the payment process is completed at that point. If the customer pays for some of the items, the payment process will be carried out so that each group member pays for the remaining items. At this time, the remaining group members can pay for items associated with them as well as items associated with other group members.

[0139] For example, if four customers A, B, C, and D are in the checkout area, after customer A has paid for all the goods purchased by customers A and B, the management device 100 can, as described above, have the remaining customers C and D select a customer to check out and select the goods that the customer will pay for, and then execute the checkout process for the selected goods. This allows, for example, customer A to select and check out some or all of the goods associated with customer B, and another customer C to select and check out some or all of the goods associated with customer D. It is also possible to select a customer to check out, so customer A can check out some of the goods associated with customer B, and another customer C can select and check out some or all of the remaining goods associated with customer B.

[0140] In the first method, where a group and its members are identified at the store entrance, even if goods are exchanged between group members in the sales area, those goods can be treated as goods purchased by the group.

[0141] On the other hand, with the second method of detecting groups in the checkout area, it is not possible to identify customers shopping as a group in the merchandise area. Therefore, the merchandise sales system is designed to manage the customer who exchanged goods and the goods exchanged when goods are exchanged between customers in the merchandise area.

[0142] Figure 12 illustrates how the management device 100 detects and manages the handover of goods. The management device 100 uses the camera 10 and sensor unit 20 to track each customer moving around the sales floor, detects the goods each customer takes from the shelves 600, and associates them with each customer. As shown in Figure 12(a), the management device 100 can detect when customer 831 hands over the goods 700 to another customer 832 based on the movements of customers 831 and 832 captured in the images from the camera 10 and the movement of the goods 700.

[0143] The method for detecting the transfer of goods is not particularly limited. For example, conventionally known techniques such as using a depth sensor to identify the skeleton and detect its movement can be used. For example, as shown in Figure 12(b), the management device 100 can recognize the position and movement of the head, neck, shoulders, arms, hands, waist, feet, etc. of each customer 831, 832, and detect that the goods 700 have been transferred from the hands of one customer 831 to the hands of the other customer 832.

[0144] When the management device 100 detects the handover of goods, it manages information about each customer who has received goods and the goods that have been received. For example, normally the management device 100 manages customer IDs, which are identification information for identifying each customer, facial images of each customer, and information about the goods that each customer has taken from the product shelf 600. When the handover of goods is detected, in addition to this information, the management device 100 manages information about the received goods and handover images captured by the camera 10 at the time of handover, as shown in Figure 12(c).

[0145] The example shown in Figure 12 illustrates that product "X" is transferred from customer 831 (customer ID "001") to customer 832 (customer ID "002"). The management device 100 treats the transferred product "X" as a retrieved item for customer "002" who received it.

[0146] Specifically, when the management device 100 detects that customer 831 with customer ID "001" has taken product "X" from the product shelf 600, it registers product "X" in the retrieved product item in association with customer ID "001". When the management device 100 detects that product "X" has been passed from customer 831 with customer ID "001" to customer 832 with customer ID "002", it deletes product "X" from the retrieved products of customer ID "001" and adds product "X" to the retrieved products of customer ID "002".

[0147] In the example shown in Figure 12(c), if two customers 831 and 832 who have received product "X" enter the checkout area together, the management device 100 will receive the customers and their selection of products to be paid for, as explained in Figures 8 to 11, and then have the checkout device 200 execute the checkout process.

[0148] On the other hand, when customer 832, who has received the goods, enters the checkout area alone, the management device 100, which recognizes this via camera 30b, displays the screen shown in Figure 13 on the screen of the operating terminal 210 of the checkout device 200. The screen displays information indicating customer 832 who is performing the checkout process, and information indicating the goods associated with this customer 832. The screen also displays information indicating that goods handed over by another customer 831 are included in the checkout process, and information indicating the customer 831 who handed these goods to customer 832. The goods that customer 832 himself took from the product shelf 600 and the goods that another customer 831 took from the product shelf 600 and handed over to customer 832 are displayed on the screen in a distinguishable manner. The screen may also display a picture of the goods being handed over at the time of handover.

[0149] In this way, when goods are handed over, the store can receive payment for the handed-over goods even if the goods are handed over between customers who are not part of the same group, by including the handed-over goods in the settlement details for the customer 832 who received them.

[0150] Furthermore, even when customers are grouped using the first method, the management device 100 may perform the processing related to the delivered goods as described in Figure 12. When goods are delivered between group members belonging to the same group, the delivered goods are processed as goods purchased by the group, so the store can receive payment for the delivered goods regardless of whether the delivered goods are associated with the group member who delivered them or the group member who received them. For this reason, when grouping using the first method, the management device 100 may perform the processing related to the delivered goods only when goods are delivered between a group member and a customer who is not a group member.

[0151] The management device 100, upon detecting the handover of goods, determines whether the handover is within the group or between a group member and a customer who is not a group member. If the handover is between a group member and a customer who is not affiliated with the group, the management device 100 executes processing related to the handover goods. If goods are handed over from a group member to a customer who is not a group member, the customer and the handover goods are associated. If goods are handed over from a customer who is not a group member to a group member, the group member and the handover goods are associated. In either case, the store can receive payment for the handover goods.

[0152] In some stores, products require verification of purchase eligibility. For example, when selling alcohol or tobacco, it is necessary to verify whether the customer meets the age restrictions. When the payment processing is performed by the payment device 200, if the products to be paid for include products that require verification of purchase eligibility, a reading device for verifying purchase eligibility should be installed adjacent to the payment device 200 to perform the purchase eligibility verification process.

[0153] For example, when selling alcohol, the management device 100 displays information on the operating terminal 210 of the payment device 200 instructing the customer to have an identification document that can verify their age read by the reader. The management device 100 then recognizes the driver's license or other identification document that the customer has read by the reader and verifies their eligibility to purchase. If eligibility cannot be verified, the management device 100 displays information on the operating terminal 210 indicating that the product cannot be sold, and proceeds with the payment process after excluding the product from the payment. The management device 100 then sends a notification to a communication terminal held by an employee waiting in the back room of the unmanned store, and the employee who receives the notification can then collect the product that has been excluded from the payment.

[0154] If the products purchased by a group include items that require verification of purchase eligibility, the management device 100 may verify that all group members are eligible to purchase, and if any group members are not eligible, the device may exclude the items from the settlement process. This prevents eligible customers from purchasing items that require eligibility on behalf of ineligible customers.

[0155] In Figures 8 and 10 of this embodiment, an example is described in which images of multiple customers in the payment area captured by camera 30b are displayed on the screen of the operation terminal 210, and a manual operation to select a customer to pay for the goods is accepted. However, the customer to pay may be selected automatically. For example, the management device 100 may recognize the customer closest to the operation terminal 210 of the payment device 200 from the images captured by camera 30b and designate that customer to pay. Alternatively, the management device 100 may recognize the actions of customers and recognize the customer operating the operation terminal 210 as the customer to pay. If the management device 100 is set to automatically recognize the customer to pay, in the example described in Figures 8 and 10, the operation to select the customer to pay can be omitted, and the process can start with the customer selecting the goods to pay for.

[0156] When the system is configured to automatically recognize the paying customer, the management device 100 may display information on the screen of the operation terminal 210 indicating which customer has been automatically recognized as the paying customer on the images of multiple customers displayed, and may accept the operation to change the paying customer as needed. For example, on the screen shown in Figure 8, information indicating that the customer has been automatically recognized as the paying customer may be displayed near the customer who has been automatically recognized as the paying customer, similar to the "payer" display in Figure 10. Instead of the message shown in Figure 8, a message instructing the user to specify the new paying customer if the paying customer is to be changed may be displayed, and the system may accept the manual operation to change the paying customer. Similarly, in the example of Figure 10, if a customer other than the one displayed as the paying customer on the screen makes the payment, the management device 100 may accept the operation to change the paying customer.

[0157] The configuration of the product sales system described in this embodiment is functionally general, and its configuration is not physically limited to this configuration. The form of distribution and integration of each device is not limited to the example described above, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0158] While embodiments of the product sales system according to this disclosure have been described above with reference to the drawings, the configuration and operation of the product sales system and each of its constituent devices are not limited to the above embodiments, and may be implemented in various ways based on the knowledge of a person skilled in the art, without departing from the spirit of the invention.

Claims

1. A product sales system comprising: a customer detection sensor that tracks customers who enter the store; a product detection sensor that detects products purchased by the customer; a management device that uses the customer detection sensor and the product detection sensor to associate and manage customers with the products purchased by the customer; a payment device for which the customer makes payment for products associated with the customer by the management device; and a customer payment detection sensor that detects customers in a payment area where the payment process is installed, wherein when the customer payment detection sensor detects multiple customers in the payment area, the management device accepts an operation for the customer to select products from among the products associated with each of the other customers excluding the customer, and includes the products selected based on the operation as the target of payment by the customer.

2. The product sales system according to claim 1, characterized in that the management device displays information indicating each of the plurality of customers detected in the predetermined area on a display device and accepts an operation to select the customer to make a payment from among the plurality of customers.

3. The product sales system according to claim 1 or 2, wherein the payment area where the payment device is installed is designated as the predetermined area, and the system further comprises a payment area gate for entering the payment area from the product sales area, and the management device locks the payment area gate to prevent another customer from entering the payment area after the multiple customers have passed through the payment area gate and entered the payment area.

4. The product sales system according to any one of claims 1 to 3, further comprising a store exit gate for exiting the store from the checkout area, wherein the management device locks the store exit gate to prevent the customers from leaving the store from the time the customers enter the checkout area until all the products associated with each of the customers have been settled.

5. The product sales system according to any one of paragraphs 1 to 4, characterized in that, if the product selected by the paying customer and associated with a customer other than the paying customer is a product that requires purchase qualifications, the management device performs a process to confirm whether the paying customer and the customer other than the paying customer have the purchase qualifications.

6. The product sales system according to any one of claims 1 to 5, wherein the operation by which the paying customer selects products includes the operation by which the management device displays information on a display device indicating products associated with each of the other customers among the plurality of customers, excluding the paying customer, and the operation by which the paying customer selects products individually, and the management device adds the selected products to the items to be paid for by the paying customer.

7. The product sales system according to any one of claims 1 to 6, wherein the operation by the paying customer to select products includes the operation by which the management device displays on a display device information indicating each of the other customers excluding the paying customer and information indicating the products associated with each of the other customers, and the paying customer selects other customers, and the management device adds all products associated with the selected other customers to the items to be paid for by the paying customer.

8. The product sales system according to any one of claims 1 to 7, wherein the operation by the paying customer to select a product includes the operation by which the management device displays information on a display device indicating each of the other customers excluding the paying customer, and the paying customer selects a customer from among the multiple customers displayed on the display device, and the management device displays information on the display device indicating a product associated with the selected customer, and the paying customer selects a product from among the multiple products displayed on the display device, and the management device adds the selected product to the items to be paid for by the paying customer.

9. The product sales system according to any one of claims 1 to 8, wherein the operation by which the paying customer selects products includes an operation by which the paying customer selects all products associated with each of the other customers excluding the paying customer, and the management device adds all products associated with the multiple customers to the items to be paid for by the paying customer.

10. The product sales system according to any one of paragraphs 1 to 9, characterized in that, when a customer other than the paying customer makes a payment after the paying customer has completed payment, the management device excludes the paid-for products from the products associated with the paying customer, and makes the remaining unpaid products subject to payment by the customer other than the paying customer.

11. The product sales system according to claim 10, characterized in that the management device displays information indicating settled products and information indicating unsettled products on a display device in a distinguishable manner.

12. The product sales system according to claim 11, characterized in that the management device displays on the display device information indicating that the customer has paid for the products that have been paid for.

13. The product sales system according to any one of claims 1 to 12, characterized in that the management device detects when goods have been exchanged between a first customer and a second customer, and manages information relating to the first customer, the second customer, and the exchanged goods.

14. The product sales system according to claim 13, characterized in that the management device associates the delivered product with the second customer and includes it as a product to be settled by the second customer.

15. The product sales system according to claim 13 or 14, characterized in that, when the management device detects the second customer in the predetermined area where settlement processing is performed, but does not detect the first customer, it displays information on the screen of the display device that shows the items to be settled by the second customer that include the goods handed over by the first customer.

16. The product sales system according to claim 15, characterized in that the management device displays an image on the screen of the display device showing the first customer handing over the product to the second customer.

17. The product sales system according to claim 15 or 16, characterized in that the management device displays an image of the first customer on the screen of the display device.

18. A method of selling goods performed by a device, comprising: a step of tracking a customer who has entered the store using a customer detection sensor; a step of detecting the goods to be purchased by the customer using a product detection sensor; a step of associating and managing the customer with the goods to be purchased by the customer; a step of detecting a customer in a payment area where a payment device is installed using a payment customer detection sensor; and, when multiple customers are detected in the payment area by the payment customer detection sensor, an operation is accepted in which the customer who is to pay for the goods selects a product from among the products associated with each of the other customers excluding the customer, and the products selected based on the operation are included as the subject of payment to be made by the customer at the payment device.

19. A product sales program recorded on a computer-readable non-temporary recording medium, characterized in that it causes a computer to execute the following steps: tracking a customer who has entered the store using a customer detection sensor; detecting the product to be purchased by the customer using a product detection sensor; managing the customer in association with the product purchased by the customer; detecting a customer in the payment area where the payment device is installed using a payment customer detection sensor; and, when multiple customers are detected in the payment area by the payment customer detection sensor, receiving an operation from the customer who is to pay for the goods to select a product from among the products associated with each of the other customers excluding the customer, and including the product selected based on the operation as the subject of payment by the customer at the payment device.

Images