Sales system

Abstract

To provide a sales system capable of efficiently performing accounting processing even on an article to which an RF tag cannot be attached.SOLUTION: A sales system 100 according to the present invention comprises: a recognition part which recognizes a customer based upon an image generated by a camera 10; a specifying part which specifies an article G picked up by the customer; and a register device 40 which presents accounting information on the specified article to the recognized customer.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a sales system used for selling products in a store. [Background technology]

[0002] For example, Patent Document 1 discloses a technology for improving the efficiency of accounting procedures in product sales. The technology disclosed in Patent Document 1 is a sales system that uses RFID technology to improve the efficiency of product sales at event venues and the like, and includes a gate installed at the entrance and exit of the event venue. The gate includes a reader that reads product IDs and user IDs from RF tags attached to each product or from ID cards held by users, and a transmitter that transmits the read product IDs and user IDs to a server. The server stores the product IDs and user IDs received from the gate in memory as purchase information and transmits payment information generated based on the purchase information to a cash register. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-191042 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the sales system described in the above patent document requires that RF tags be attached to all products sold, making it difficult to use the above sales system in stores that handle products that cannot be RF-tagged, such as stores where customers pick up food cooked in the store and pay for it as is.

[0005] An object of the present invention is to provide a sales system that can efficiently process transactions even for products that cannot be affixed with an identification medium such as an RF tag. [Means for solving the problem]

[0006] In order to achieve the above object, the sales system of the present invention is characterized by comprising a recognition unit that recognizes a customer based on an image generated by a camera, an identification unit that identifies a product picked up by the customer, and an accounting processing device that presents accounting information for the identified product to the recognized customer.

[0007] The recognition unit may recognize multiple customers, and the transaction processing device may present transaction information for one of the customers and transaction information for other customers subsequent to the one customer. [Effects of the Invention]

[0008] The sales system of the present invention can efficiently process transactions even for products that cannot be RF-tagged. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a store in which a sales system according to a first embodiment of the present invention is used; [Figure 2] (a) A hardware configuration diagram of the sales system, (b) A hardware configuration diagram of a camera of the sales system, (c) A hardware configuration diagram of a GPU of the sales system, (d) A hardware configuration diagram of a server of the sales system, and (e) A hardware configuration diagram of a cash register device of the sales system. [Figure 3] FIG. 10A is a front view of a display device included in a sales system according to a second embodiment of the present invention; FIG. 10B is a side cross-sectional view of the display device; [Figure 4] FIG. 2 is a diagram showing a hardware configuration of the display device. DETAILED DESCRIPTION OF THE INVENTION

[0010] A sales system according to an embodiment of the present invention will be described below with reference to the drawings.

[0011] [First embodiment]

[0012] As shown in FIG. 1, a store equipped with a sales system 100 (FIG. 2) according to a first embodiment of the present invention has a sales area including a tray table 1 on which a large number of unused trays T are placed, a display area E1 in which products G are displayed, and a checkout area E2 in which transactions are made. The tray table 1, display area E1, and checkout area E2 are arranged linearly, and a lane 3 for placing trays T and a customer movement path R are provided along the display area E1 and the checkout area E2. The display area E1 has a display stand 2, which is a display device for displaying multiple product groups. The display stand 2 has a placement surface 2a that extends horizontally along the movement path R, and products G are placed on the placement surface 2a. A lane 3 for placing trays T is provided horizontally from the placement surface 2a toward the movement path R. In the checkout area E2, two cash registers 40 (checkout processing devices) are provided side by side along the movement path R. Examples of such stores include a store that sells donuts, which are the product G, and an udon restaurant that sells toppings such as tempura as the product G.

[0013] As shown in Fig. 2, the sales system 100 used in the store includes, in addition to the two cash registers 40, a camera 10, an image processing device (hereinafter referred to as GPU 20), and a server 30. The camera 10 is connected to the GPU 20 via a wired or wireless connection so as to be able to communicate with each other. A network N is also formed in the store, and the GPU 20 and each cash register 40 are connected to the server 30 via the network N so as to be able to communicate with each other.

[0014] As shown in Fig. 1, camera 10 is installed on the ceiling of the sales area, captures images of the sales area at a predetermined frame rate, and generates images of the sales area. As shown in Fig. 2, camera 10 includes image sensor 11 that functions as an imaging unit that captures images of the sales area, CPU 12 that functions as a generation unit that controls image sensor 11 and generates images, memory 13 that functions as a storage unit that temporarily stores the generated images, and communication module 14 that functions as a transmission unit that transmits images to GPU 20.

[0015] The GPU 20 includes a communication module 21 that functions as a receiver for receiving images from the camera 10, a CPU 22 that functions as a processor for processing the received images, a memory 23 that functions as a storage for storing the image processing program and the images, and a network module 24 that functions as a transmitter for transmitting information acquired by the image processing to the server 30. The CPU 22, which is the processor, recognizes the customers and products G included in the images each time it receives an image from the camera 10. The method for recognizing the customers and products G is not particularly limited, but for example, the recognition can be based on the appearance characteristics of the customers and products G. Specifically, the appearance characteristics (shape, color, size) of the customers and products G when observed from above are predetermined, and pixel patterns that match or are similar to the appearance characteristics are recognized as the customers and products G. Alternatively, machine learning may be used as another recognition method. Specifically, a trained model is generated in advance using images of people and products G as training data. Then, the image received from the camera 10 is input to the trained model to recognize the customers and products G included in the image.

[0016] Here, when the CPU 22, which is a processing unit, recognizes a product G in an image, it identifies a product ID based on the appearance features and trained model of the recognized product G. Once the product ID is identified, the network module 24, which functions as a transmission unit, associates the product ID with location information of the recognized product G (coordinates of the product G in the image) and transmits the information to the server 30. Furthermore, when the CPU 22 recognizes a customer, it assigns a customer ID to the recognized customer and associates the customer's location information (coordinates of the customer in the image) with the customer ID and transmits the information to the server 30 via the network module 24.

[0017] The server 30 includes a network module 34 that functions as a receiving unit that receives information from the GPU 20 and a transmitting unit that transmits information to the cash register device 40, a CPU 32 that executes processes such as identification processing, determination processing, and generation processing, and a memory 33 that functions as a storage unit that stores programs that cause the CPU 32 to execute these processes and information received from the GPU 20.

[0018] The identification process executed by the CPU 32 is a process of identifying the picked-up product G and the customer who picked up the product G by associating them with each other based on information received from the GPU 20. The picked-up product G is identified, for example, by identifying the product G having position information (coordinate values) on lane 3 (FIG. 1) as the picked-up product G. Furthermore, the customer who picked up the product G is identified, for example, by identifying the customer whose position information is similar to the position information of the identified product G as the customer who picked up the product G. The product ID of the identified product G is then associated with the customer ID of the identified customer. In this way, the CPU 32 of the server 30 functions as an identification unit that identifies the picked-up product G and the customer who picked up the product G by associating them with each other. Furthermore, the memory 33, which functions as a storage unit, stores the picked-up product G (product ID) and the customer who picked up the product G (customer ID) in association with each other.

[0019] The determination process executed by the CPU 32 includes a first determination process that determines whether the customer's location information received from the GPU 20 is included in the transaction area E2, and a second determination process that determines the cash register 40 closest to the customer based on the customer's location information if the first determination process determines that the customer's location information is included in the transaction area E2. In this way, the CPU 32 of the server 30 functions as a determination unit that determines the cash register 40 closest to the customer. In addition, the memory 33, which functions as a storage unit, stores the determination results of the first determination process and the second determination process.

[0020] The generation process executed by the CPU 32 generates transaction information. This transaction information includes the name and price of the picked-up product G and is generated based on the product ID identified in the identification process. Specifically, a product table that defines the correspondence between the name and price of each product G and the product ID is pre-stored in the memory 33 (storage unit) of the server 30. The transaction information is generated by extracting the name and price of the product G using the identified product ID as a search key. The transaction information generated by the generation process is stored in the memory 33, which functions as a storage unit, and sent via the network module 34 to the cash register 40 identified in the determination process.

[0021] The cash register 40 is equipped with a network module 44 that functions as a receiver that receives transaction information from the server 30, a memory 43 that functions as a memory that stores the received transaction information, a display 45 that functions as a display that shows the received transaction information to the customer, a CPU 42 that functions as a transaction unit that processes the transaction based on the transaction information, and a cashier 46. Once the transaction has been processed, the network module 44 of the cash register 40 also functions as a transmitter that sends a message to the server 30 that the transaction has been completed.

[0022] In the sales system 100 configured as described above, an image of the sales area is generated by a camera 10 that captures images of the sales area at a predetermined frame rate, and the presence of products G and customers in the sales area is recognized by a GPU 20 that processes the image. The server 30 then identifies the product G picked up by the customer, and transaction information for the identified product G is sent to a cash register 40, which then processes the transaction. In this way, the sales system 100 of this embodiment can identify and process the product G picked up by the customer without attaching an identification medium such as an RF tag to the product G as in the past. Furthermore, since there is no need to input information about the picked product G during the transaction, the transaction can be carried out efficiently.

[0023] Furthermore, in the above embodiment, when the GPU 20 recognizes multiple customers, it transmits the customer IDs and their location information for the multiple customers, as well as the product IDs and location information for the recognized products G, to the server 30. The server 30, upon receiving this information, performs an identification process to identify the products G picked up by the customers based on the customer's location information and the product G's location information. Specifically, the server 30 calculates the difference between the customer's location information and the location information for each product G, identifies the products G picked up by the customers if the difference value is within a certain value, and associates the product IDs of the products G with the customer IDs. Furthermore, in the second determination process of the above determination process, if the location information of the multiple customers is close to a common cash register 40, the server 30 determines that the multiple customers are a group and transmits the transaction information for the multiple customers to the cash register 40. When the cash register 40 receives multiple transaction information from the server 30, it displays a screen on its display that allows the customer to select whether to process the transactions individually or together. This configuration makes it possible to process the transactions of a group of customers all at once.

[0024] [Second embodiment]

[0025] In the first embodiment described above, the customer and the commodity G are recognized based on the image generated by the camera 10, but the second embodiment differs from the first embodiment in that the customer is recognized based on the image generated by the camera 10 and the commodity G is recognized based on the output of the load sensor. The following description will focus on the differences from the first embodiment.

[0026] As shown in FIG. 3, the display area E1 of the second embodiment is provided with a display device 50 for displaying products G. The display device 50 includes a display shelf 51 for displaying the products G and a sensor 55 for detecting the displayed products G. The display shelf 51 includes a plurality of upper and lower shelf boards 52 and a side wall 53 for supporting the shelf boards 52. A plurality of trays T are placed on each shelf board 52, and a plurality of products G of the same type are placed on each tray T. A load cell 55 (load sensor), which is the sensor 55, is provided between each tray T and the shelf board 52. The load cell 55 is a sensor for detecting the weight of the plurality of products G placed on the tray T, and as shown in FIG. 4, its output signal is input to a microcomputer (hereinafter referred to as MCU 57) via an analog-to-digital converter (hereinafter referred to as ADC 56). The MCU 57 includes a CPU and a memory, and the CPU executes programs including recognition processing and transmission processing that are pre-stored in the memory.

[0027] The recognition process is a process for recognizing that a product G placed on a tray T has been picked up based on the output of the load cell 55, and includes an acquisition process, a difference process, a comparison process, and a determination process. The acquisition process is a process for periodically acquiring the output value of the load cell 55 via the ADC 56, and the acquired output value is stored in memory. The difference process is a process for calculating the difference between the output value acquired in the acquisition process and the output value acquired in the previous acquisition process. The comparison process is a process for comparing the difference value obtained in the difference process with a threshold value. This threshold value corresponds to the weight of each product G placed on the tray T and is determined in advance by experimentally acquiring the output value of the load cell 55 with one product G placed on the tray T. The determination process is a process for determining whether a product G has been picked up. If the difference value is less than the threshold value as a result of the comparison process, it is determined that the product G has not been picked up from the tray T. If the difference value exceeds the threshold value, it is determined that the product G has been picked up from the tray T. The CPU executes the above acquisition process through the determination process on the output of the load cell 55 of each tray T. As a result of the CPU performing the recognition process in this manner, when it determines that product G has been picked up, the CPU identifies the tray ID of the tray T corresponding to that load cell 55 and transmits the tray ID to the server 30 via the network module 58 (transmission process).

[0028] The server 30 of this embodiment receives customer position information and customer IDs from the GPU 20, as in the first embodiment, and also receives tray IDs from the display device 50. In the identification process, the CPU 32 of the server 30 associates the picked-up product G with the customer who picked up the product G based on the received tray ID and customer position information. Specifically, in addition to the product table of the first embodiment, the memory 33 of the server 30 of this embodiment pre-stores a tray table and a product-tray correspondence table. The tray table associates tray IDs with position information of trays T. The product-tray correspondence table associates tray IDs with product IDs. The CPU 32 of the server 30 then extracts the position information of the tray T corresponding to the received tray ID from the tray table and also extracts the product ID corresponding to the tray ID from the product-tray correspondence table. The CPU 32 of the server 30 then searches for customer position information close to the extracted position information of the tray T and associates the extracted product ID with the customer ID of the customer.

[0029] Then, the CPU 32 of the server 30 executes the determination process and generation process, as in the first embodiment, and sends the transaction information to the corresponding cash register 40, which then executes the transaction process.

[0030] The sales system of the present invention has been described based on the first and second embodiments, but each of the above embodiments may be modified as follows.

[0031] <Variation 1> It may be possible to recognize that a product G has been picked up based on an image generated by a camera capturing an image of the display area E1 and the output of a load sensor provided in the display area E1. For example, in the second embodiment, a display shelf may be provided with multiple cameras. Specifically, a camera may be provided on the underside of a shelf board located above each tray T, and each camera may capture an image of the product G placed on the tray T. The images of the product G on the tray T generated individually by each camera are sent to the GPU 20, and the GPU 20 recognizes the product G picked up from the tray T.

[0032] <Variation 2> In the first embodiment described above, a single camera 10 is used to capture an image of the sales area, but multiple cameras 10 may also be used. When multiple cameras 10 are used to capture an image of the sales area, each camera 10 transmits the generated image to the GPU. The GPU combines the images received from the multiple cameras 10 to generate an area image of the sales area, and performs the image processing of the first embodiment on the area image.

[0033] Alternatively, a GPU may be provided corresponding to each of the multiple cameras. In this case, each GPU performs the image processing of the first embodiment, associates the product ID with the location information of the certified product G, and transmits the associated information to the server 30. Furthermore, each GPU associates the customer ID with the location information of the certified customer and transmits the associated information to the server 30. The server 30 identifies the location information of the product G and the customer within the entire sales area based on the location information of the product G and the customer received from each GPU, and then executes the above-mentioned identification process, determination process, and generation process.

[0034] <Variation 3> The camera 10 used in the first and second embodiments may be a TOF camera. The TOF camera outputs color information and distance information for each pixel and transmits the color and distance information to a GPU. The GPU generates an image based on the color information received from the TOF camera and can accurately detect the location information of the product or customer recognized based on the generated image based on the distance information. The GPU can also detect the customer's arm recognized based on the distance information, detect the arm motion based on the distance information received for each frame, and identify the picked-up product based on the motion.

[0035] <Variation 4> In the above embodiment, the product G is picked up on a tray T, but it may be a bag or box brought by the customer, or a bag or box prepared in advance by the store for carrying the product G home. In this embodiment, the customer can pick up the product and at the same time place the product G in the bag or box brought with them.

[0036] <Variation 5> A handover area may be provided between the display area E1 and the checkout area E2. The handover area is provided with a table on which the product to be handed can be placed, and the product is recognized by the camera 10. This type of arrangement can also accommodate products specially ordered by customers, such as "less ice."

[0037] <Variation 6> The payment processing device may process payments using electronic money or credit cards in addition to cash payments. It may also process payments only using electronic money or credit cards. When processing payments using credit cards, the payment processing device may also be equipped with a credit card reader located upstream from the tray base 1. The credit card reader reads the customer's credit card and sends the read credit card information to the server. Upon receiving the credit card information, the server queries the credit card company's system for the credit card information and determines whether the credit card can be used. The server performs the above-mentioned identification process, determination process, and generation process, associates the credit card information with the transaction information generated in the generation process, and sends the resulting information to the payment processing device. The payment processing device processes the transaction using the transaction information and credit card information received from the server. [Explanation of symbols]

[0038] 10 Camera 20 GPU 30 servers 40 Cash register device 50 Display equipment 100 Sales System

Claims

1. a recognition unit that recognizes customers based on images generated by the camera; an identification unit that identifies the product picked up by the customer; a transaction processing device that presents transaction information for the identified product to the identified customer; A sales system comprising:

2. The recognition unit recognizes a plurality of customers, The transaction processing device is characterized in that it presents transaction information of one of the customers and transaction information of other customers consecutive to the one customer. The sales system of claim 1 .

Images