Product registration support device, product registration support system, and program

Abstract

To provide an article registration support device, an article registration support system, and a program that can prevent an illegal act when a customer registers an article by himself or herself.SOLUTION: An edge gateway (article registration support device) supports the operation of a cart terminal (article registration device) which registers an article, and comprises: a video acquisition part which acquires video that a camera (photographing device) photographs of a basket containing articles having information read and the identification number of the basket, the video showing that the articles are thrown in the basket, an article counting part (counting part) which counts a first number through the video, the first number being the number of articles thrown in the basket; an article registration information acquisition part (article number acquisition part) which acquires a second number, the second number being the number of articles that the cart terminal reads until the completion of the registration; and an article number comparison information output part which outputs the identification number, and the first number and the second number to let a portable terminal (terminal device) that a shop assistant carries give notice.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] Embodiments of the present invention relate to a product registration support device, a product registration support system, and a program.

Background Art

[0002] Conventionally, a shopping system has been proposed in which an image of a product taken when a customer puts a product into a shopping cart is transmitted to a server, and the product is identified by the server to perform product registration and settlement (for example, Patent Document 1).

[0003] Such a shopping system is an effective system from the viewpoint of shortening the waiting time at the cash register. However, in the system of Patent Document 1, in order to surely photograph the product put into the shopping cart, the product is put in only from a fixed location. Therefore, it was necessary to significantly modify the conventional shopping cart.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide a product registration support device, a product registration support system, and a program that allow a customer to register a product himself / herself without significantly modifying a conventional shopping cart.

Means for Solving the Problems

[0005] The product registration support device according to the embodiment supports the operation of a product registration device that registers a product by reading information that uniquely identifies the product to be purchased, and includes a video acquisition unit, a counting unit, a product quantity acquisition unit, and a product quantity comparison information output unit. The video acquisition unit acquires an image captured by a photographing device that photographs a storage unit that stores the product from which the information has been read, and an identification number that identifies the storage unit. The counting unit counts from the video acquired by the video acquisition unit The film the image and the identification number for identifying the storage unit. By detecting when an object has detached from the customer's hand, whose skeleton has been estimated,The first count is the number of items placed in the storage compartment. The item count acquisition unit acquires the second count, which is the number of items read by the item registration device before registration is complete. The item count comparison information output unit outputs the identification number, the first count, and the second count to the terminal device for notification, provided that the first count and the second count do not match. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 is a block diagram showing an example of a schematic configuration of the product sales data processing system according to the embodiment. [Figure 2] Figure 2 is a hardware block diagram showing an example of the hardware configuration of a shopping cart terminal. [Figure 3] Figure 3 shows an example of the data structure of a product master. [Figure 4] Figure 4 shows an example of the data structure of a product registration file. [Figure 5] Figure 5 is a hardware block diagram showing an example of the hardware configuration of an edge gateway. [Figure 6] Figure 6 shows an example of the data structure of a product count file. [Figure 7] Figure 7 is a hardware block diagram showing an example of the hardware configuration of a mobile device. [Figure 8] Figure 8 is a hardware block diagram showing an example of the hardware configuration of an accounting machine. [Figure 9] Figure 9 is a functional block diagram showing an example of the functional configuration of a shopping cart terminal. [Figure 10] Figure 10 is a functional block diagram showing an example of the functional configuration of an edge gateway. [Figure 11] Figure 11 is a functional block diagram showing an example of the functional configuration of a mobile device. [Figure 12] Figure 12 is a functional block diagram showing an example of the functional configuration of an accounting machine. [Figure 13] Figure 13 is a flowchart showing an example of the processing flow performed by a shopping cart terminal. [Figure 14] Figure 14 is a flowchart showing an example of the processing flow performed by the edge gateway. [Figure 15] Figure 15 is a flowchart showing an example of the processing flow performed by a mobile device. [Figure 16] Figure 16 is a flowchart showing an example of the processing flow performed by an accounting machine. [Figure 17] Figure 17 is a schematic perspective view of a self-service POS terminal included in the product sales data processing system of the second embodiment. [Figure 18] Figure 18 is a hardware block diagram showing an example of the hardware configuration of a self-service POS terminal. [Figure 19] Figure 19 is a functional block diagram showing an example of the functional configuration of a self-service POS terminal. [Figure 20] Figure 20 is a flowchart showing an example of the processing flow performed by a self-service POS terminal. [Modes for carrying out the invention]

[0007] The embodiments will be described in detail below with reference to the drawings. However, the invention is not limited to the embodiments described below.

[0008] (First Embodiment) Hereinafter, a product sales data processing system 1, which is a first embodiment of this disclosure, will be described with reference to the drawings.

[0009] (Outline configuration of the product sales data processing system) Figure 1 is a block diagram showing an example of the schematic configuration of the product sales data processing system of the embodiment. The product sales data processing system 1 is a system that supports customers' shopping in stores.

[0010] The product sales data processing system 1 comprises a shopping cart 5, an edge gateway 20, a store server 30, a mobile terminal 40, and a checkout machine 50.

[0011] The shopping cart 5 allows a customer to read identification information (e.g., product code) for identifying the products to be purchased while pushing it around, and accommodates the products for which the identification information has been read. The shopping cart 5 includes a basket 7, a cart terminal 10, a scanner 18, and a camera 19.

[0012] The basket 7 receives the products that the customer purchases and accommodates the products that have been put in. The basket 7 is an example of the accommodating part in the present disclosure. Since there are a plurality of shopping carts 5 in the store, each shopping cart 5 is assigned an identification number that uniquely identifies the basket 7 of that shopping cart 5. This identification number is hereinafter referred to as the cart ID.

[0013] The cart terminal 10 registers the products for which the identification information has been read. Further, the cart terminal 10 transmits the registration result of the products to an edge gateway 20 or a store server 30, which will be described later, via an access point 8, which is a wireless communication repeater installed in multiple locations in the store, and an in-store network N. The cart terminal 10 is an example of the product registration device in the present disclosure.

[0014] The scanner 18 obtains the identification information (e.g., product code) of the product registered in the code symbol by reading the code symbol such as a barcode attached to the product.

[0015] The camera 19 is installed on the shopping cart 5 so as to be able to photograph the entire basket 7, and photographs the video of the products being put into the basket 7 until the customer reads the code symbols of all the products to be purchased. Further, the camera 19 associates the photographed video with the cart ID that identifies the shopping cart 5 on which the camera 19 is installed, and transmits it to an edge gateway 20, which will be described later, via an access point 8 and an in-store network N. When the shopping cart 5 has upper and lower two-stage baskets, the camera 19 is provided in the upper basket and the lower basket, respectively.

[0016] The edge gateway 20 collects video footage captured by the camera 19 via interfaces such as serial, USB, and wireless / wired LAN. The edge gateway 20 also obtains the product registration results from the cart terminal 10. The edge gateway 20 then analyzes the acquired video footage to count the number of items placed in the basket 7. At this time, the edge gateway 20 recognizes the position of the customer's arms and hands by performing image processing, such as well-known skeletal estimation. Furthermore, by recognizing that the customer's hands are above the basket 7 and that an object has been removed from the hands (detached), it detects that the customer has placed items in the basket 7 of the shopping cart 5 and the number of items placed. The edge gateway 20 also compares the counted number of items (first number) with the number of items in the product registration results (second number). If the first quantity and the second quantity do not match, the edge gateway 20 notifies the mobile terminal 40 (described later) of the cart ID and the fact that the first quantity and the second quantity do not match. The edge gateway 20 is an example of a product registration support device in this disclosure.

[0017] The store server 30 manages all transactions involving goods in the store. For example, the store server 30 sends the contents of the product master, a master file containing product information for the goods handled, which is updated sequentially, to the shopping cart terminal 10. The store server 30 also retrieves the product registration results from the shopping cart terminal 10 and issues a transaction ID that uniquely identifies the transaction. The store server 30 notifies the shopping cart terminal of the issued transaction ID. In other words, the transaction ID corresponds one-to-one with the product registration results sent from the shopping cart terminal 10. Furthermore, when a customer makes a payment at the checkout machine 50 (described later), the store server 30 receives the transaction ID sent from the checkout machine 50 and notifies the checkout machine 50 of the product registration results corresponding to that transaction ID.

[0018] The mobile terminal 40 is held by a store employee and exchanges various information with the store server 30 or with other mobile terminals 40. The mobile terminal 40 is, for example, a smartphone or a tablet. The mobile terminal 40 also obtains information from the edge gateway 20 regarding the customer's product registration results, such as the cart ID and information indicating that the first quantity and the second quantity do not match. The mobile terminal 40 then notifies the store employee if the first quantity and the second quantity do not match. Upon noticing the notification, the store employee goes to the shopping cart 5 with the corresponding cart ID, explains the situation to the customer, and checks the items placed in the basket 7. Note that the mobile terminal 40 is an example of a terminal device in this disclosure.

[0019] The payment terminal 50 is a device used by customers who have completed registering their items to complete the payment process. The payment terminal 50 reads the transaction ID from the shopping cart terminal 10 and notifies the store server 30 of the read transaction ID. The payment terminal 50 then retrieves the item registration result corresponding to the transaction ID from the store server 30. After that, the payment terminal 50 processes the payment using the payment method specified by the customer. To improve the efficiency of the payment process, multiple payment terminals 50 may be installed in the store.

[0020] (Hardware configuration of the shopping cart terminal) The hardware configuration of the cart terminal 10 will be explained using Figure 2. Figure 2 is a hardware block diagram showing an example of the hardware configuration of the cart terminal.

[0021] The cart terminal 10 comprises a control unit 11, a storage unit 12, a peripheral device controller 13, and a communication interface 14.

[0022] The control unit 11 controls the overall operation of the cart terminal 10. The control unit 11 comprises a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, and a RAM (Random Access Memory) 113. The CPU 111 connects to the ROM 112 and the RAM 113 via internal buses such as an address bus and a data bus. The CPU 111 loads various programs stored in the ROM 112 and the memory unit 12 into the RAM 113. The CPU 111 controls the operation of the cart terminal 10 by operating according to the various programs loaded into the RAM 113. In other words, the control unit 11 has the configuration of a typical computer.

[0023] The storage unit 12 is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) connected to the control unit 11 via an internal bus 15. Alternatively, the storage unit 12 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 12 stores the control program 121, the product master 122, the product registration file 123, and the cart ID 124.

[0024] The control program 121 is a program that controls the overall operation of the cart terminal 10. The control program 121 may be provided stored in the storage unit 12, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD. Alternatively, the control program 121 may be stored on a computer connected to a network and provided by downloading it via the network. Furthermore, the control program 121 may be provided or distributed via a network such as the Internet.

[0025] Product Master 122 is a master file that stores product codes that uniquely identify products sold in stores, associating them with product information for those products. The data structure of Product Master 122 will be described in detail later (see Figure 3).

[0026] The product registration file 123 is a file that stores the product codes and other information of the products that the customer has scanned with the scanner 18, i.e., the products the customer intends to purchase. The data structure of the product registration file 123 will be described in detail later (see Figure 4).

[0027] The control unit 11 is connected to the display device 16, the operation device 17, the scanner 18, and the camera 19 via the peripheral device controller 13, which is connected by an internal bus 15.

[0028] The display device 16 is a display device such as an LCD monitor or an OLED monitor provided by the cart terminal 10. The display device 16 displays the product list, etc., read by the scanner 18.

[0029] The operating device 17 is an information input device such as a touch panel stacked on the display device 16. When a customer starts shopping, they instruct the cart terminal 10 to start shopping by pressing an operator such as a shopping start button displayed on the display device 16. After the customer has finished registering all the items they wish to purchase, they inform the cart terminal 10 that product registration is complete by pressing an operator such as a checkout button displayed on the display device 16.

[0030] The scanner 18 reads a code symbol, such as a barcode, held over the reading window. The scanner 18 is, for example, a barcode reader. The scanner 18 decodes the read code symbol and outputs the registered product code.

[0031] Camera 19 is an imaging device equipped with an imaging device such as a CCD or CMOS, and is installed to capture images of the entire cage 7. Camera 19 starts capturing images upon receiving a capture start instruction from the control unit 11 and stops capturing images upon receiving a capture end instruction from the control unit 11. Camera 19 captures images of the cage 7 at a speed of, for example, 30 frames per second. Camera 19 outputs the captured images to the edge gateway 20.

[0032] The communication interface 14 enables wireless communication between the cart terminal 10 and the edge gateway 20 or store server 30.

[0033] (Data structure of the product master) Figure 3 illustrates the data structure of product master 122. Figure 3 shows an example of the data structure of product master.

[0034] The product master 122 stores the product information, such as the product name and price, associated with a product code that uniquely identifies the product. Note that the product information may also include other information, such as special sale information.

[0035] (Data structure of product registration file) Figure 4 illustrates the data structure of product registration file 123. Figure 4 shows an example of the data structure of a product registration file.

[0036] Product registration file 123 stores the product code, unit price, quantity, and price (excluding tax) of each registered product. It also stores the transaction ID, cart ID, total quantity, and total price of the products.

[0037] The transaction ID is an identification number that uniquely identifies a transaction. The transaction ID is issued by the store server 3, which retrieves the product registration file 123, so it is blank when the shopping cart terminal 10 generates the product registration file 123. The transaction ID field stores the transaction ID that the shopping cart terminal 10 received from the store server 3.

[0038] The cart ID is an identification number that identifies the shopping cart 5 on which the cart terminal 10 used to register the product is installed. The cart ID is the cart ID 124 stored in the storage unit 12 of the cart terminal 10 (see Figure 2).

[0039] The total number of items is the sum of the quantities of each individual item.

[0040] The total amount is the sum of the individual prices (excluding tax) of each item. Alternatively, the total amount can be calculated by remembering the sum of the individual prices (excluding tax) and the amount of consumption tax.

[0041] (Edge gateway hardware configuration) Figure 5 illustrates the hardware configuration of the edge gateway 20. Figure 5 is a hardware block diagram showing an example of the hardware configuration of the edge gateway.

[0042] The edge gateway 20 comprises a control unit 21, a storage unit 22, and a communication interface 23.

[0043] The control unit 21 controls the overall operation of the edge gateway 20. The control unit 21 comprises a CPU 211, a ROM 212, and a RAM 213. The CPU 211 connects to the ROM 212 and the RAM 213 via internal buses such as an address bus and a data bus. The CPU 211 loads various programs stored in the ROM 212 and the memory unit 22 into the RAM 213. The CPU 211 controls the operation of the edge gateway 20 by operating according to the various programs loaded into the RAM 213. In other words, the control unit 21 has the configuration of a typical computer.

[0044] The storage unit 22 is a storage device such as an HDD or SSD, connected to the control unit 11 via the internal bus 24. Alternatively, the storage unit 22 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 22 stores the control program 221 and the product counting file 222.

[0045] The control program 221 is a program that controls the overall operation of the edge gateway 20. The control program 221 may be provided stored in the storage unit 12, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk, CD-R, or DVD. Alternatively, the control program 221 may be stored on a computer connected to a network and provided by downloading it via the network. Furthermore, the control program 221 may be provided or distributed via a network such as the Internet.

[0046] The product counting file 222 is a file that registers the date and time of the purchase and the number of items placed in shopping cart 7 for each shopping cart ID of shopping cart 5. The data structure of the product counting file 222 will be described in detail later (see Figure 6).

[0047] Communication I / F23 enables wireless communication between the edge gateway 20, the store server 30, the cart terminal 10, the mobile terminal 40, and the checkout machine 50.

[0048] (Data structure of product counting file) Figure 6 illustrates the data structure of the product count file 222. Figure 6 shows an example of the data structure of a product count file.

[0049] Product counting file 222 stores the date and time and the number of items added, associated with the cart ID.

[0050] The cart ID is the identification information of the shopping cart 5, transmitted in association with the video footage captured by camera 19.

[0051] The date and time is, for example, the date and time when camera 19 started taking pictures. The date and time may also include the date and time when camera 19 finished taking pictures.

[0052] The number of items loaded is the number of items (first count) counted by the edge gateway 20 from the video footage captured by camera 19 installed on shopping cart 5 with cart ID.

[0053] (Hardware configuration of mobile devices) The hardware of the mobile terminal 40 will be explained using Figure 7. Figure 7 is a hardware block diagram showing an example of the hardware configuration of the mobile terminal.

[0054] The mobile terminal 40 comprises a control unit 41, a storage unit 42, a peripheral device controller 43, and a communication interface 23.

[0055] The control unit 41 controls the overall operation of the mobile terminal 40. The control unit 41 comprises a CPU 411, a ROM 412, and a RAM 413. The CPU 411 connects to the ROM 412 and the RAM 413 via internal buses such as an address bus and a data bus. The CPU 411 loads various programs stored in the ROM 412 and the memory unit 42 into the RAM 413. The CPU 411 controls the operation of the mobile terminal 40 by operating according to the various programs loaded into the RAM 413. In other words, the control unit 41 has the configuration of a typical computer.

[0056] The storage unit 42 is a storage device such as an HDD or SSD, connected to the control unit 41 via an internal bus 45. Alternatively, the storage unit 42 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 42 stores the control program 421.

[0057] The control program 421 is a program that controls the overall operation of the mobile terminal 40. The control program 421 may be provided stored in the memory unit 42, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk, CD-R, or DVD. The control program 421 may also be provided by storing it on a computer connected to a network and downloading it via the network. Furthermore, the control program 421 may be provided or distributed via a network such as the Internet.

[0058] The control unit 41 is connected to a display device 46, an operating device 47, and a speaker 48 via a peripheral device controller 43 connected by an internal bus 45.

[0059] The display device 46 is a display device such as an LCD monitor or an OLED monitor installed in the mobile terminal 40. The display device 46 displays various information transmitted from the store server 30 and the edge gateway 20.

[0060] The operating device 47 is an information input device such as a touch panel stacked on the display device 46. Store employees carrying a mobile terminal 40 can operate the operating device 47 to, for example, check information sent from the store server 30 or edge gateway 20, or exchange information with other store employees.

[0061] The communication interface 44 enables wireless communication between the mobile terminal 40 and the edge gateway 20 or store server 30.

[0062] (Hardware configuration of the accounting machine) The hardware of the accounting machine 50 will be explained using Figure 8. Figure 8 is a hardware block diagram showing an example of the hardware configuration of the accounting machine.

[0063] The accounting machine 50 comprises a control unit 51, a storage unit 52, a peripheral device controller 53, and a communication interface 54.

[0064] The control unit 51 controls the overall operation of the accounting machine 50. The control unit 51 comprises a CPU 511, a ROM 512, and a RAM 513. The CPU 511 connects to the ROM 512 and the RAM 513 via internal buses such as an address bus and a data bus. The CPU 511 loads various programs stored in the ROM 512 and the memory unit 52 into the RAM 513. The CPU 511 controls the operation of the accounting machine 50 by operating according to the various programs loaded into the RAM 513. In other words, the control unit 51 has the configuration of a typical computer.

[0065] The storage unit 52 is a storage device such as an HDD or SSD, connected to the control unit 51 via an internal bus 55. Alternatively, the storage unit 52 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 52 stores the control program 521 and the settlement file 522.

[0066] The control program 521 is a program that controls the overall operation of the accounting machine 50. The control program 521 may be provided stored in the memory unit 52, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD. The control program 521 may also be provided by storing it on a computer connected to a network and downloading it via the network. Furthermore, the control program 521 may be provided or distributed via a network such as the Internet.

[0067] The settlement file 522 is a file that adds settlement details such as the settlement date and time, settlement method, and transaction amount to the contents of the product registration file 123. The settlement file 522 is sent to the store server 30 at a predetermined time along with an identification number that uniquely identifies the accounting machine 50, and is stored and managed as a sales file in the store server 30.

[0068] The control unit 51 is connected to a display device 56, an operating device 57, a scanner 58, a receipt printer 59, and a warning light 60 via a peripheral device controller 53 connected by an internal bus 55.

[0069] The display device 56 is a display device such as an LCD monitor or an OLED monitor provided by the accounting machine 50. The display device 56 displays various information related to accounting processing.

[0070] The scanner 58 is, for example, a handheld scanner that reads a code symbol including a transaction ID displayed on the display device 16 of the cart terminal 10.

[0071] The receipt printer 59 issues a receipt with the transaction details printed on it.

[0072] The warning light 60 illuminates or flashes in response to customer operation when assistance from a store employee is needed. Upon noticing the illumination or flashing of the warning light 60, a store employee rushes to the payment machine 50 to assist the customer. Alternatively, instead of illuminating or flashing the warning light 60, the payment machine 50 may send a signal requesting assistance to the handheld terminal 40 carried by the store employee.

[0073] Although not shown in Figure 8, the accounting machine 50 is also equipped with a change dispenser and a card reader. The change dispenser accepts banknotes or coins when a customer makes a cash payment and dispenses banknotes or coins as change. The card reader reads information registered on media such as credit cards and electronic money cards when a customer makes a cashless payment.

[0074] The communication interface 54 enables wireless communication between the accounting machine 50 and the edge gateway 20 and store server 30.

[0075] (Functional configuration of the shopping cart terminal) The functional configuration of the cart terminal 10 will be explained using Figure 9. Figure 9 is a functional block diagram showing an example of the functional configuration of the cart terminal.

[0076] The control unit 11 of the cart terminal 10 functions as a product registration start instruction acquisition unit 61, a shooting control unit 62, a video output unit 63, a product code acquisition unit 64, a product registration unit 65, a product registration completion instruction acquisition unit 66, a product registration information output unit 67, a transaction ID acquisition unit 68, a payment instruction acquisition unit 69, and a transaction ID display unit 70, with the CPU 111 operating according to a control program 121 stored in the memory unit 12. Note that all or some of these functions may be configured using dedicated circuits or other hardware.

[0077] The product registration start instruction acquisition unit 61 acquires an instruction that product registration has started when the operation device 17 detects that the shopping start button displayed on the display device 16 has been pressed.

[0078] The shooting control unit 62 gives instructions to the camera 19 to start shooting and to stop shooting.

[0079] The video output unit 63 associates the video captured by the camera 19 with the cart ID 124 and outputs it to the edge gateway 20.

[0080] The product code acquisition unit 64 reads a barcode or other code symbol attached to the product held up to the scanner 18 and acquires the product code registered to the code symbol.

[0081] The product registration unit 65 registers the product having the product code obtained by the product code acquisition unit 64 in the product registration file 123.

[0082] The product registration completion instruction acquisition unit 66 acquires an instruction that product registration is complete when the operation device 17 detects that the checkout button, for example, has been pressed, which is displayed on the display device 16 and indicates that product registration is complete.

[0083] The product registration information output unit 67 outputs the product registration file 123 to the edge gateway 20.

[0084] The transaction ID acquisition unit 68 obtains the transaction ID assigned to the product registration file 123 by the store server 30 from the edge gateway 20. Alternatively, the transaction ID acquisition unit 68 may obtain the transaction ID from the store server 30.

[0085] The payment instruction acquisition unit 69 acquires a payment start instruction by detecting, for example, that a payment button has been pressed, as displayed on the display device 16, using the operation device 17.

[0086] The transaction ID display unit 70 displays a code symbol, such as a barcode containing the transaction ID, on the display device 16 in a format that can be read by the scanner 58 of the accounting machine 50.

[0087] (Edge gateway functional configuration) The functional configuration of the edge gateway 20 will be explained using Figure 10. Figure 10 is a functional block diagram showing an example of the functional configuration of the edge gateway.

[0088] The control unit 21 of the edge gateway 20 functions as a video acquisition unit 71, a product counting unit 72, a product registration information acquisition unit 73, a product quantity comparison unit 74, a product registration information output unit 75, a transaction ID acquisition unit 76, a transaction ID output unit 77, and a product quantity comparison information output unit 78, with the CPU 211 operating according to a control program 221 stored in the memory unit 22. Note that all or some of these functions may be configured using dedicated hardware such as circuits.

[0089] The video acquisition unit 71 acquires from the cart terminal 10 video footage of products being placed into the shopping cart basket 7, captured by the camera 19 that films the basket 7 of the shopping cart 5, and a cart ID that identifies the shopping cart 5.

[0090] The product counting unit 72 counts a first number, which is the number of products placed in the basket 7, from the video acquired by the video acquisition unit 71. Note that the product counting unit is an example of a counting unit in this disclosure. The product counting unit 72 recognizes the position of the customer's arms and hands by performing image processing such as well-known skeletal estimation. Furthermore, by recognizing that the customer's hands are above the basket 7 and that an object has left (detached from) the hand, it determines that the customer has placed a product in the basket 7. The product counting unit 72 then counts the number of products placed in the basket 7. Alternatively, the product counting unit 72 may acquire video footage taken by the camera 19 of the inside of the basket 7 and recognize the number of products inside the basket 7 using object recognition technology.

[0091] The product registration information acquisition unit 73 acquires a second number, which is the number of products scanned by the scanner 18 of the cart terminal 10 before registration is completed. More specifically, the product registration information acquisition unit 73 acquires the product registration file 123 from the cart terminal 10. Note that the product registration information acquisition unit 73 is an example of a product count acquisition unit in this disclosure.

[0092] The product count comparison unit 74 compares the first count calculated by the product counting unit 72 with the second count, which is the number of products registered in the product registration file 123.

[0093] The product registration information output unit 75 outputs the identification number of the shopping cart 5 (basket 7), the first quantity, and the second quantity to the mobile terminal 40 held by the store clerk, provided that the first quantity and the second quantity do not match.

[0094] The transaction ID acquisition unit 76 acquires the transaction ID assigned by the store server 30 to the product registration file 123 output to the store server 30 by the edge gateway 20.

[0095] The transaction ID output unit 77 outputs the transaction ID obtained from the store server 30 to the corresponding cart terminal 10.

[0096] The product quantity comparison information output unit 78, on the condition that the first quantity and the second quantity do not match, outputs the cart ID identifying the shopping cart 5, the first quantity, and the second quantity to the mobile terminal 40 held by the store clerk to provide notification. The product quantity comparison information output unit 78 may also output the cart ID, the first quantity, and the product registration file 123 to the mobile terminal 40. The product quantity comparison information output unit 78 outputs this information to all mobile terminals 40.

[0097] (Functional configuration of mobile devices) The functional configuration of the mobile terminal 40 will be explained using Figure 11. Figure 11 is a functional block diagram showing an example of the functional configuration of a mobile terminal.

[0098] The control unit 41 of the mobile terminal 40 functions as a product count comparison information acquisition unit 81, a notification control unit 82, and a product count comparison information display unit 83, with the CPU 411 operating according to a control program 421 stored in the memory unit 42. Note that all or part of these functions may be configured using dedicated hardware such as circuits.

[0099] The product quantity comparison information acquisition unit 81 acquires the cart ID that identifies the shopping cart 5, the first quantity, and the second quantity from the edge gateway 20. The product quantity comparison information acquisition unit 81 may also acquire the cart ID, the first quantity, and the product registration file 123 from the edge gateway 20. The product quantity comparison information acquisition unit 81 acquires this information only if the first quantity and the second quantity do not match.

[0100] The notification control unit 82 issues a notification on the condition that the product quantity comparison information acquisition unit 81 has acquired the cart ID, the first quantity, and the second quantity from the edge gateway 20. The notification method is not limited, but for example, an alarm sound may be output from the speaker 48 of the mobile terminal 40. Alternatively, the mobile terminal 40 may vibrate an oscillator (vibrator) that it has built into it (not shown).

[0101] The product number comparison information display unit 83 displays the information acquired by the product number comparison information acquisition unit 81 on the display device 46 provided by the mobile terminal 40.

[0102] (Functional configuration of the accounting machine) The functional configuration of the accounting machine 50 will be explained using Figure 12. Figure 12 is a functional block diagram showing an example of the functional configuration of the accounting machine.

[0103] The control unit 51 of the accounting machine 50 functions as a transaction ID acquisition unit 91, a product registration information acquisition unit 92, a payment method acquisition unit 93, a payment processing unit 94, a payment completion information display unit 95, and a receipt issuing unit 96, with the CPU 511 operating according to a control program 521 stored in the memory unit 52. Note that all or some of these functions may be configured using dedicated circuits or other hardware.

[0104] The transaction ID acquisition unit 91 acquires the transaction ID by reading the code symbol, which includes the transaction ID, displayed on the display device 16 of the cart terminal 10, with the scanner 58 provided by the accounting machine 50.

[0105] The product registration information acquisition unit 92 outputs the transaction ID acquired by the transaction ID acquisition unit 91 to the store server 30, and obtains the product registration file corresponding to the transaction ID from the store server 30.

[0106] The payment method acquisition unit 93 acquires the payment method instructed by the customer by detecting, using the operation device 57, whether one of the selection buttons for selecting a payment method displayed on the display device 56 of the accounting machine 50 has been pressed.

[0107] The payment processing unit 94 performs the payment using the payment method acquired by the payment method acquisition unit 93. If cash payment is selected, the payment processing unit 94 receives the cash using the change dispenser provided by the accounting machine 50. If cashless payment is selected, the payment processing unit 94 reads the customer's credit card registration information or the amount charged to the customer's electronic money card using the card reader provided by the accounting machine 50, according to the payment method, and sends the payment to an external server (not shown) that performs payment processing.

[0108] The settlement completion information display unit 95 displays information indicating that the settlement has been completed, along with the transaction details, on the display device 56 of the accounting machine 50.

[0109] The receipt issuing unit 96 issues a receipt with the payment details printed on it from the receipt printer 59 installed in the accounting machine 50.

[0110] (Store server hardware and functional configuration) Although the hardware and functional configuration of the store server 30 is not shown in the diagram, the store server 30, like the edge gateway 20, has a typical computer configuration.

[0111] The store server 30 sends the product master, which is updated as needed, to the shopping cart terminal 10. The store server 30 also retrieves the product registration file 123 from the shopping cart terminal 10, issues a transaction ID, and notifies the shopping cart terminal 10 of the issued transaction ID. The store server 30 issues a transaction ID only if the number of products counted by the edge gateway 20 (first quantity) matches the number of products registered in the product registration file 123 (second quantity). Therefore, the edge gateway 20 may output the product registration file 123 retrieved from the shopping cart terminal 10 to the store server 30 only if it determines that the first quantity and the second quantity are equal.

[0112] The store server 30 issues a transaction ID that uniquely identifies the product registration file 123 to the product registration file 123 and associates it with the product registration file 123.

[0113] (Process flow performed by the shopping cart terminal) Figure 13 illustrates the processing flow performed by the shopping cart terminal 10. Figure 13 is a flowchart showing an example of the processing flow performed by the shopping cart terminal.

[0114] The product registration start instruction acquisition unit 61 acquires an instruction that product registration has started when the operation device 17 detects that the shopping start button displayed on the display device 16 has been pressed (step S11).

[0115] The shooting control unit 62 instructs the camera 19 to start shooting video (step S12).

[0116] The video output unit 63 associates the video captured by the camera 19 with the cart ID 124 and outputs it to the edge gateway 20 (step S13).

[0117] The product code acquisition unit 64 determines whether a product code has been acquired from the code symbol read by the scanner 18 (step S14). If it is determined that a product code has been acquired (step S14: Yes), the process proceeds to step S15. On the other hand, if it is not determined that a product code has been acquired (step S14: No), the process returns to step S13.

[0118] In step S14, if it is determined that a product code has been obtained, the product registration unit 65 registers the product with the product code obtained by the product code acquisition unit 64 in the product registration file 123 (step S15).

[0119] The product registration completion instruction acquisition unit 66 determines whether there is an instruction that product registration has been completed (step S16). If it is determined that there is an instruction that product registration has been completed (step S16: Yes), the process proceeds to step S17. On the other hand, if it is not determined that there is an instruction that product registration has been completed (step S16: No), the process returns to step S13. The product registration completion instruction acquisition unit 66 determines that there is an instruction that product registration has been completed when the operation device 17 detects that the checkout button displayed on the display device 16 has been pressed.

[0120] In step S16, if it is determined that an instruction has been received indicating that product registration is complete, the shooting control unit 62 gives an instruction to the camera 19 to end shooting (step S17). Although not shown in the flowchart of Figure 13, at this time the video output unit 63 stops outputting video to the edge gateway 20.

[0121] The product registration information output unit 67 outputs the product registration file 123 to the edge gateway 20 (step S18).

[0122] The transaction ID acquisition unit 68 determines from the edge gateway 20 whether it has acquired the transaction ID that the store server 30 has assigned to the product registration file 123 (step S19). If it is determined that the transaction ID has been acquired (step S19: Yes), the process proceeds to step S20. On the other hand, if it is not determined that the transaction ID has been acquired (step S19: No), the cart terminal 10 terminates processing. The transaction ID acquisition unit 68 may also acquire the transaction ID from the store server 30.

[0123] In step S19, if it is determined that a transaction ID has been obtained, the settlement instruction acquisition unit 69 determines whether it has received a settlement initiation instruction from the customer (step S20). If it is determined that a settlement initiation instruction has been received from the customer (step S20: Yes), the process proceeds to step S21. On the other hand, if it is not determined that a settlement initiation instruction has been received from the customer (step S20: No), step S20 is repeated. The settlement instruction acquisition unit 69 determines that a settlement initiation instruction has been received by detecting, for example, that a settlement button displayed on the display device 16 has been pressed, using the operation device 17.

[0124] In step S20, if it is determined that a payment start instruction has been received from the customer, the transaction ID display unit 70 displays a code symbol, such as a barcode containing the transaction ID, on the display device 16 in a format that can be read by the scanner 58 of the accounting machine 50 (step S21). After that, the cart terminal 10 terminates processing.

[0125] (Processing flow performed by the edge gateway) Figure 14 illustrates the processing flow performed by the edge gateway 20. Figure 14 is a flowchart showing an example of the processing flow performed by the edge gateway.

[0126] The video acquisition unit 71 acquires the video captured by the camera 19 and the cart ID 124 from the cart terminal 10 (step S31).

[0127] The product counting unit 72 analyzes the video acquired by the video acquisition unit 71 to determine whether a product has been placed in the basket 7 (step S32). If it is determined that a product has been placed in the basket 7 (step S32: Yes), the process proceeds to step S33. On the other hand, if it is not determined that a product has been placed in the basket 7 (step S32: No), the process returns to step S31. The product counting unit 72 determines whether a product has been placed in the basket 7 by, for example, recognizing that an object has detached from the customer's arm or hand, which has been detected by skeletal detection, and is moving toward the basket 7.

[0128] In step S32, if it is determined that an item has been placed in basket 7, the item counting unit 72 adds the number of items placed in basket 7 with the corresponding cart ID (step S33).

[0129] The product registration information acquisition unit 73 determines whether it has acquired the product registration file 123 from the cart terminal 10 installed in the shopping cart 5 with the corresponding cart ID (step S34). If it is determined that the product registration file 123 has been acquired (step S34: Yes), the process proceeds to step S35. On the other hand, if it is not determined that the product registration file 123 has been acquired (step S34: No), the process returns to step S31.

[0130] In step S34, if it is determined that the product registration file 123 has been obtained, the product count comparison unit 74 determines whether the number of products counted by the product counting unit 72 (the number of products placed in the basket 7) is equal to the number of products registered in the product registration file 123 (step S35). If it is determined that the two are equal (step S35: Yes), the process proceeds to step S36. On the other hand, if it is determined that the two are not equal (step S35: No), the process proceeds to step S39.

[0131] In step S35, if it is determined that the number of items placed in the basket 7 is equal to the number of items registered in the product registration file 123, the product registration information output unit 75 outputs the product registration file 123 to the store server 30 (step S36).

[0132] The transaction ID acquisition unit 76 acquires the transaction ID assigned by the store server 30 to the product registration file 123 output in step S36 from the store server 30 (step S37).

[0133] The transaction ID output unit 77 outputs the transaction ID obtained by the transaction ID acquisition unit 76 in step S37 to the cart terminal 10 (step S38). After that, the edge gateway 20 terminates processing.

[0134] The explanation returns to step S35. In step S35, if it is determined that the number of items placed in the basket 7 is not equal to the number of items registered in the product registration file 123, the product count comparison information output unit 78 outputs the cart ID and the product registration file 123 to the mobile terminal 40 (step S39). After that, the edge gateway 20 terminates processing.

[0135] (Process flow performed by mobile devices) Figure 15 illustrates the processing flow performed by the mobile terminal 40. Figure 15 is a flowchart showing an example of the processing flow performed by the mobile terminal.

[0136] The product count comparison information acquisition unit 81 determines whether it has acquired the product count comparison information output by the product count comparison information output unit 78 from the edge gateway 20 (step S41). The product count comparison information is information that includes the cart ID, the first quantity, and the second quantity, output by the edge gateway 20, provided that the first quantity and the second quantity do not match. If it is determined that the product count comparison information has been acquired (step S41: Yes), the process proceeds to step S42. On the other hand, if it is not determined that the product count comparison information has been acquired (step S41: No), the process repeats step S41.

[0137] In step S41, if it is determined that product number comparison information has been acquired, the notification control unit 82 will notify the user by outputting sound or voice from the speaker 48 of the mobile terminal 40 (step S42).

[0138] The product number comparison information display unit 83 displays the information acquired by the product number comparison information acquisition unit 81 on the display device 46 of the mobile terminal 40 (step S43). After that, the mobile terminal 40 terminates processing.

[0139] Among the store employees who have received the notification and are carrying a mobile terminal 40, any employee who can rush to the location of shopping cart 5 with the notified cart ID goes to shopping cart 5, obtains the customer's consent to verify the items, and confirms whether the items placed in basket 7 match the registered items.

[0140] (Process flow performed by the accounting machine) Figure 16 illustrates the processing flow performed by the accounting machine 50. Figure 16 is a flowchart showing an example of the processing flow performed by the accounting machine.

[0141] The transaction ID acquisition unit 91 acquires the transaction ID by reading the code symbol containing the transaction ID displayed on the display device 16 of the cart terminal 10 with the scanner 58 provided by the accounting machine 50 (step S51).

[0142] The product registration information acquisition unit 92 outputs the transaction ID acquired by the transaction ID acquisition unit 91 to the store server 30 (step S52).

[0143] The product registration information acquisition unit 92 acquires the product registration file 123 corresponding to the transaction ID from the store server 30 (step S53).

[0144] The payment method acquisition unit 93 acquires the payment method instructed by the customer by detecting with the operation device 57 whether one of the selection buttons for selecting a payment method displayed on the display device 56 of the accounting machine 50 has been pressed (step S54).

[0145] The settlement processing unit 94 performs settlement processing according to the settlement method obtained in step S54 (step S55).

[0146] The settlement completion information display unit 95 displays information indicating that the settlement has been completed, along with the transaction details, on the display device 56 of the accounting machine 50 (step S56).

[0147] The receipt issuing unit 96 issues a receipt with the payment details printed on it from the receipt printer 59 installed in the accounting machine 50 (step S57). After that, the accounting machine 50 terminates the process.

[0148] Although not shown in the flowchart in Figure 16, after the settlement process is completed, the accounting machine 50 generates a settlement file 522 showing the settlement details and outputs it to the store server 30.

[0149] (Effects of the first embodiment) As described above, the edge gateway 20 (product registration support device) of the product sales data processing system 1 of the first embodiment supports the operation of the cart terminal 10 (product registration device) that registers products by reading information that uniquely identifies the products to be purchased, and the video acquisition unit 71 acquires the video of the product being placed into the basket 7 (storage unit) captured by the camera 19 (shooting device) that photographs the basket 7 (storage unit) in which the information of the product is stored, and an identification number that identifies the basket 7, and the video acquisition unit 7 The system includes a product counting unit 72 (counting unit) that counts a first number of products placed in basket 7 from the video footage acquired by 1, a product registration information acquisition unit 73 (product quantity acquisition unit) that acquires a second number of products that the scanner 18 has read by the cart terminal 10 until registration is complete, and a product quantity comparison information output unit 78 that, on the condition that the first number and the second number do not match, outputs an identification number that identifies basket 7, the first number, and the second number to be notified to a handheld terminal 40 (terminal device) carried by a store employee. Therefore, customers can register products themselves without making significant modifications to conventional shopping carts. Furthermore, since the process of customers placing products into basket 7 of shopping cart 5 is monitored and the number of products placed into basket 7 is counted, fraudulent activity such as customers placing products into basket 7 without reading the code symbol with scanner 18 can be reliably detected and reported.

[0150] Furthermore, in the edge gateway 20 (product registration support device) of the product sales data processing system 1 of the first embodiment, the video acquisition unit 71 acquires video footage captured by the camera 19 (shooting device) from the time the cart terminal 10 (product registration device) is instructed to start product registration until the cart terminal 10 is instructed to complete product registration. Therefore, it is possible to capture without fail the process of the customer placing products into the basket 7 of the shopping cart 5.

[0151] Furthermore, in the product sales data processing system 1 of the first embodiment, if the first quantity and the second quantity do not match, the first quantity and the second quantity are notified to the handheld terminal 40 (terminal device) carried by the store clerk. Therefore, the store clerk can be informed without fail that the first quantity, which is the number of items placed in the basket 7 of the shopping cart 5 by the customer, is different from the second quantity, which is the number of items read by the scanner 18 by the time the purchase is completed. As a result, a store clerk who is near the shopping cart 5 can immediately go to the shopping cart 5 and assist the customer.

[0152] Furthermore, the product sales data processing system 1 of the first embodiment includes a scanner 18 installed in the shopping cart 5 that reads a code symbol containing the product's identification number attached to the product, a camera 19 (shooting device) installed in the shopping cart 5 that captures images of the product whose code symbol has been read being placed into the basket 7 (storage section) of the shopping cart 5, and acquires the images and the identification number of the shopping cart 5 to count a first number, which is the number of products placed in the shopping cart 5, and acquires a second number, which is the number of products read by the scanner 18 until registration is complete. The system includes an edge gateway 20 (product registration support device) that outputs the identification number of the shopping cart 5 and the first and second quantities, provided that the first and second quantities do not match; a portable terminal 40 (terminal device) carried by a store employee that notifies the employee when it has obtained the identification number of the shopping cart 5 and the first and second quantities output by the edge gateway 20; a cart terminal 10 (product registration device) that registers products scanned by the scanner 18; and a payment machine 50 (payment device) that processes payment for products registered by the cart terminal 10. Therefore, customers can register products themselves without making significant modifications to conventional shopping carts. Furthermore, the system monitors how customers place products into the basket 7 of the shopping cart 5 and counts the number of products placed in the basket 7, thus preventing fraudulent acts such as customers placing products into the basket 7 without reading the code symbol with the scanner 18.

[0153] (Second embodiment) Next, a second embodiment of the present disclosure will be described. In the second embodiment of the present disclosure, the product sales data processing system 2 (not shown) is the same as the product sales data processing system 1 shown in Figure 1, but the shopping cart 5 does not include the cart terminal 10, scanner 18, and camera 19. Also, the product sales data processing system 2 includes a self-service POS terminal 130 shown in Figure 17 instead of the checkout machine 50 shown in Figure 1.

[0154] Customers shop by pushing a shopping cart 5 or carrying a basket (not shown) and placing items into the basket 7 of the shopping cart 5 or their own basket. After finishing their shopping, customers register their purchases and pay at the self-service POS terminal 130. In other words, the self-service POS terminal 130 combines the functions of the cart terminal 10 and the checkout machine 50 in the first embodiment.

[0155] The self-service POS terminal 130 is equipped with a camera 189 (see Figure 17), which will be described later. The camera 189 captures images of the customer placing items (i.e., registered items) that have had their code symbol scanned by the scanner 184 (see Figure 17) into the shopping cart.

[0156] The self-service POS terminal 130 transmits the video footage captured by the camera 189 and a product registration file containing the product information registered by the self-service POS terminal 130 to the edge gateway 20.

[0157] The edge gateway 20 has the same functions as described in the first embodiment. Specifically, the edge gateway 20 counts the number of items placed in the basket by the customer by analyzing the video captured by the camera 189. The edge gateway 20 also compares the number of items placed in the basket by the customer (i.e., the first number in the first embodiment) with the number of items registered in the product registration file (i.e., the second number in the first embodiment). If the first number and the second number do not match, the edge gateway 20 sends information to the self-service POS terminal 130 indicating this fact.

[0158] When the self-service POS terminal 130 receives information from the edge gateway 20 indicating that the first quantity and the second quantity do not match, it notifies a store employee near the self-service POS terminal 130 by illuminating or flashing its built-in warning light 190 (see Figure 17). Upon noticing the notification, the employee approaches the customer operating the self-service POS terminal 130, explains the situation to the customer, and confirms the items, etc. Note that the self-service POS terminal 130 is an example of a terminal device in this disclosure.

[0159] (Outline configuration of the accounting machine) The schematic configuration of the self-service POS terminal 130 will be explained using Figure 17. Figure 17 is a schematic perspective view of the self-service POS terminal included in the product sales data processing system of the second embodiment.

[0160] The self-service POS terminal 130 includes a display device 182, a scanner 184, and a hand scanner 185. An operating device 183 is laminated on the surface of the display device 182.

[0161] The display device 182 is, for example, an LCD monitor or an OLED monitor. The display device 182 displays various information related to the product transaction to the customer. For example, the display device 182 displays product information registered by reading a code symbol attached to the product using the scanner 184 or hand scanner 185. The display device 182 also displays an accounting screen (not shown) that displays the total price of the products, the amount received, the amount of change, etc., and allows the customer to select a payment method.

[0162] The customer checks the information displayed on the display device 182 and, if necessary, proceeds with registering products and making payments by operating the operation buttons displayed on the screen. When the customer operates an operation button, this is detected by an operation device 183, which is for example a touch panel, and transmitted to the self-service POS terminal 130.

[0163] Scanner 184 and hand scanner 185 read code symbols such as barcodes and two-dimensional codes attached to products. Scanner 184 and hand scanner 185 are equipped with optical image sensors such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) and capture images of the imaging area in front of the reading window. The customer causes scanner 184 to read the code symbol attached to the product by holding it over the reading window of scanner 184. For large products such as cardboard boxes or long products, it is difficult to hold the code symbol attached to the product over the reading window of scanner 184. In such cases, the customer causes scanner 185 to read the code symbol by holding it over the barcode or other code symbol attached to the product.

[0164] To the right of the self-service POS terminal 130 is a product placement tray 191 where customers can place unpaid items in their baskets. To the left of the self-service POS terminal 130 is a product placement tray 192 where customers can place paid items. Customers place registered items into the basket on the product placement tray 192. If a customer has prepared a reusable shopping bag, they hook the bag's handles onto a hook 193 located above the product placement tray 192 and place the registered items into the bag.

[0165] A downward-facing camera 189 is installed above the product display stand 192. The camera 189 films customers placing registered products into baskets placed on the product display stand 192 or into bags with handles hooked onto hooks 193. The self-service POS terminal 130 transmits the captured video to the edge gateway 20, associating it with an identification number that uniquely identifies the self-service POS terminal 130.

[0166] The self-service POS terminal 130 is further equipped with a card reader 186. The card reader 186 reads the customer's inserted loyalty card, credit card, etc. The self-service POS terminal 130 is also equipped with a receipt printer 187. The receipt printer 187 prints a receipt with the transaction result when payment is completed.

[0167] Furthermore, the self-service POS terminal 130 is equipped with a change dispenser 188 that calculates the amount of banknotes and coins deposited by the customer at the time of payment and dispenses change banknotes and coins as needed.

[0168] Furthermore, a warning light 190 is installed at the tip of a pole positioned above the self-service POS terminal 130. The warning light 190 lights up or flashes when there is an abnormality in the operation performed on the self-service POS terminal 130, when there is an abnormality in the operation of the self-service POS terminal 130, or when a store employee needs to be called. The warning light 190 also lights up or flashes to notify a store employee when it receives information from the edge gateway 20 indicating that the number of items placed in the basket by the customer does not match the number of items read by the scanner 184 or hand scanner 185.

[0169] (Hardware configuration of self-service POS terminals) The hardware configuration of the self-service POS terminal 130 will be explained using Figure 18. Figure 18 is a hardware block diagram showing an example of the hardware configuration of a self-service POS terminal.

[0170] The self-service POS terminal 130 comprises a control unit 151, a storage unit 152, a peripheral device controller 153, and a communication interface 154.

[0171] The control unit 151 controls the overall operation of the self-service POS terminal 130. The control unit 151 comprises a CPU 161, a ROM 162, and a RAM 163. The CPU 161 connects to the ROM 162 and the RAM 163 via internal buses such as an address bus and a data bus. The CPU 161 loads various programs stored in the ROM 162 and the memory unit 152 into the RAM 163. The CPU 161 controls the operation of the self-service POS terminal 130 by operating according to the various programs loaded into the RAM 163. In other words, the control unit 151 has the configuration of a typical computer.

[0172] The storage unit 152 is a storage device such as an HDD or SSD, connected to the control unit 151 via an internal bus 155. Alternatively, the storage unit 152 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 152 stores the control program 164, the product master 165, the member master 166, and the product registration file 167.

[0173] The control program 164 is a program that controls the overall operation of the self-service POS terminal 130. The control program 164 may be provided stored in the memory unit 152, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD. The control program 164 may also be provided by storing it on a computer connected to a network and downloading it via the network. Furthermore, the control program 164 may be provided or distributed via a network such as the Internet.

[0174] Product master 165 is the same as product master 122 mentioned above (see Figure 3).

[0175] The member master file 166 is a master file that stores the customer's name, date of birth, gender, nationality, mobile device email address, etc., associated with the customer's member ID. Based on the member ID read by the card reader 186 from the member card, the member master file 166 reads the information of the customer who possesses that member card and updates the customer's purchase history, etc. The purchase history is used to improve the quality of service provided to the customer.

[0176] Product registration file 167 is the same as product registration file 123 mentioned above (see Figure 4).

[0177] The control unit 151 is connected to various peripheral devices shown in Figure 18 via a peripheral device controller 153 connected by an internal bus 155. Since the general characteristics of these peripheral devices have been described above, a further explanation will be omitted.

[0178] The communication interface 154 enables wireless communication between the self-service POS terminal 130 and the edge gateway 20 or store server 30.

[0179] (Functional configuration of self-service POS terminals) The functional configuration of the self-service POS terminal 130 will be explained using Figure 19. Figure 19 is a functional block diagram showing an example of the functional configuration of a self-service POS terminal.

[0180] The control unit 151 of the self-service POS terminal 130 functions as a product code acquisition unit 231, product registration unit 232, shooting control unit 233, video output unit 234, product registration completion instruction acquisition unit 235, product registration information output unit 236, product number comparison information acquisition unit 237, notification control unit 238, payment method acquisition unit 239, payment processing unit 240, payment completion information display unit 241, and receipt issuing unit 242, with the CPU 161 operating according to the control program 164 stored in the memory unit 152.

[0181] The product code acquisition unit 231 reads a barcode or other code symbol attached to a product held over the scanner 184 or hand scanner 185 and acquires the product code registered to the code symbol.

[0182] The product registration unit 232 registers the products with the product codes obtained by the product code acquisition unit 231 in the product registration file 167.

[0183] The shooting control unit 233 gives instructions to the camera 189 to start and stop shooting.

[0184] The video output unit 234 associates the video captured by the camera 189 with an identification number that uniquely identifies the self-service POS terminal 130, and outputs it to the edge gateway 20.

[0185] The product registration completion instruction acquisition unit 235 acquires an instruction that product registration is complete when the operation device 183 detects, for example, that the checkout button has been pressed, which is displayed on the display device 182 and indicates that product registration is complete.

[0186] The product registration information output unit 236 outputs the product registration file 167 to the edge gateway 20.

[0187] The product quantity comparison information acquisition unit 237 acquires product quantity comparison information from the edge gateway 20, based on the video transmitted from its self-service POS terminal 130 and the product registration file 167, specifically the first quantity (the number of products placed in the basket) and the second quantity (the number of products registered in the product registration file 167). The edge gateway 20 outputs this information to the self-service POS terminal 130 only if the first quantity and the second quantity do not match.

[0188] The notification control unit 238 provides notification by illuminating or flashing the warning light 190, provided that the product quantity comparison information acquisition unit 237 has acquired the first quantity and the second quantity from the edge gateway 20.

[0189] The payment method acquisition unit 239 acquires the payment method instructed by the customer by detecting, using the operation device 183, whether one of the selection buttons for selecting a payment method displayed on the display device 182 of the self-service POS terminal 130 has been pressed.

[0190] The payment processing unit 240 performs the payment using the payment method acquired by the payment method acquisition unit 239. If cash payment is selected, the payment processing unit 240 receives the cash using the change dispenser 188 provided by the self-service POS terminal 130. If cashless payment is selected, the payment processing unit 240 reads the customer's credit card registration information or the amount charged to the customer's electronic money card using the card reader 186 provided by the self-service POS terminal 130, according to the payment method, and sends the payment to an external server (not shown) that performs payment processing.

[0191] The payment completion information display unit 241 displays information indicating that the payment has been completed, along with the transaction details, on the display device 182 of the self-service POS terminal 130.

[0192] The receipt issuing unit 242 issues a receipt with the payment details printed on it from the receipt printer 187 installed in the self-service POS terminal 130.

[0193] (Process flow performed by a self-service POS terminal) Figure 20 illustrates the processing flow of the self-service POS terminal 130. Figure 20 is a flowchart showing an example of the processing flow of a self-service POS terminal.

[0194] The product registration unit 232 receives an instruction that product registration has started when the operation device 183 detects that the "Start Shopping" button displayed on the display device 182 has been pressed (step S61).

[0195] The shooting control unit 233 instructs the camera 189 to start shooting video (step S62).

[0196] The video output unit 234 associates the video captured by the camera 189 with the identification number of the self-service POS terminal 130 and outputs it to the edge gateway 20 (step S63).

[0197] The product code acquisition unit 231 determines whether a product code has been acquired from the code symbol read by the scanner 184 or the hand scanner 185 (step S64). If it is determined that a product code has been acquired (step S64: Yes), the process proceeds to step S65. On the other hand, if it is not determined that a product code has been acquired (step S64: No), the process returns to step S63.

[0198] In step S64, if it is determined that a product code has been obtained, the product registration unit 232 registers the product with the product code obtained by the product code acquisition unit 231 in the product registration file 167 (step S65).

[0199] The product registration completion instruction acquisition unit 235 determines whether there is an instruction that product registration has been completed (step S66). If it is determined that there is an instruction that product registration has been completed (step S66: Yes), the process proceeds to step S67. On the other hand, if it is not determined that there is an instruction that product registration has been completed (step S66: No), the process returns to step S63. The product registration completion instruction acquisition unit 235 determines that there is an instruction that product registration has been completed when the operation device 183 detects that the checkout button displayed on the display device 182 has been pressed.

[0200] In step S66, if it is determined that an instruction has been received indicating that product registration is complete, the shooting control unit 233 gives an instruction to the camera 189 to end shooting (step S67). Although not shown in the flowchart of Figure 20, at this time the video output unit 234 stops outputting video to the edge gateway 20.

[0201] The product registration information output unit 236 outputs the product registration file 167 to the edge gateway 20 (step S68).

[0202] The product count comparison information acquisition unit 237 determines whether it has acquired product count comparison information from the edge gateway 20 (step S69). If it is determined that product count comparison information has been acquired (step S69: Yes), the process proceeds to step S70. On the other hand, if it is not determined that product count comparison information has been acquired (step S69: No), the process proceeds to step S71.

[0203] In step S69, if it is determined that the product count comparison information has been acquired, the notification control unit 238 provides notification by illuminating or flashing the warning light 190 (step S70). After that, the self-service POS terminal 130 terminates processing.

[0204] On the other hand, if it is determined in step S69 that the product number comparison information has not been obtained, the payment method acquisition unit 239 acquires the payment method instructed by the customer by detecting with the operation device 183 whether one of the selection buttons for selecting a payment method displayed on the display device 182 of the self-service POS terminal 130 has been pressed (step S71).

[0205] The settlement processing unit 240 performs settlement processing according to the settlement method obtained in step S71 (step S72).

[0206] The payment completion information display unit 241 displays information indicating that the payment has been completed, along with the transaction details, on the display device 182 of the self-service POS terminal 130 (step S73).

[0207] The receipt issuing unit 242 issues a receipt with the payment details printed on it from the receipt printer 187 installed in the self-service POS terminal 130 (step S74). After that, the self-service POS terminal 130 terminates processing.

[0208] (Effects of the embodiment) As explained above, in the product sales data processing system of the second embodiment, if the first quantity and the second quantity do not match, the first quantity and the second quantity are notified to the self-service POS terminal 130 (terminal device) where the customer registers and pays for the products they purchase. Therefore, without making significant modifications to conventional self-service POS terminals, the self-service POS terminal can reliably detect and report fraudulent activity, such as when a customer places products in a basket or bag without reading the code symbol with the scanner 184 or hand scanner 185.

[0209] Although embodiments of the present invention have been described above, these embodiments are illustrative and are not intended to limit the scope of the invention. This novel embodiment can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0210] 1. Product Sales Data Processing System 5 Shopping cart 7. Basket (storage area) 8 access points 10. Shopping cart terminal (product registration device) 18 Scanners 19,189 Cameras (imaging devices) 20. Edge Gateway (Product Registration Support Device) 30 Store Servers 40 Mobile devices (terminal devices) 50 Accounting machines (accounting devices) 61 Product Registration Start Instruction Acquisition Unit 62,233 Imaging Control Unit 63,234 Video output section 64,231 Product Code Acquisition Section 65,232 Product Registration Department 66,235 Product registration completion instruction acquisition department 67,236 Product Registration Information Output Unit 68 Transaction ID Acquisition Section 69 Settlement Instruction Acquisition Department 70 Transaction ID display section 71 Video Acquisition Unit 72. Product Counting Department (Counting Department) 73. Product Registration Information Acquisition Unit (Product Count Acquisition Unit) 74 Product number comparison section 75 Product Registration Information Output Unit 76,91 Transaction ID Acquisition Section 77 Transaction ID Output Section 78. Product Quantity Comparison Information Output Unit 81,237 Product number comparison information acquisition department 82,238 Notification Control Unit 83 Product number comparison information display section 92 Product Registration Information Acquisition Department 93,239 Payment Method Acquisition Department 94,240 Payment Processing Unit 95,241 Payment completion information display section 96,242 Receipt Issuance Department 130 Self-service POS terminal (terminal device) N In-store network [Prior art documents] [Patent Documents]

[0211] [Patent Document 1] Japanese Patent Publication No. 2014-235530

Claims

1. A product registration support device that assists the operation of a product registration device that registers a product by reading information that uniquely identifies the product to be purchased, A video acquisition unit acquires an image captured by a camera that photographs the storage compartment containing the product from which the aforementioned information was read, and an identification number that identifies the storage compartment. A counting unit counts a first number, which is the number of products placed in the storage unit, by detecting from the video acquired by the video acquisition unit that an object has been separated from the customer's hand, whose skeleton has been estimated. The aforementioned product registration device includes a product quantity acquisition unit that acquires a second quantity which is the number of products read up to the completion of registration, The system includes a product quantity comparison information output unit that outputs the identification number, the first quantity, and the second quantity to a terminal device for notification, provided that the first quantity and the second quantity do not match. Product registration support device.

2. The aforementioned video acquisition unit, From the time the product registration device is instructed to start product registration until the time the product registration is completed, the camera captures images from that period. Product registration support device according to claim 1.

3. The aforementioned terminal device is a portable terminal carried by a store employee. Product registration support device according to claim 1 or claim 2.

4. The aforementioned terminal device is a self-service POS terminal that allows customers to register and pay for the products they purchase themselves. Product registration support device according to claim 1 or claim 2.

5. The device further comprises a transaction ID output unit that outputs a transaction ID to the product registration device that uniquely identifies a transaction, provided that the first number and the second number match. Product registration support device according to claim 1 or claim 2.

6. A scanner installed in the shopping cart that reads a code symbol containing the product's identification number attached to the product, A camera installed in the shopping cart, which captures images of the storage compartment containing the product whose code symbol has been read, A product registration support device that acquires the aforementioned video and the identification number of the shopping cart, detects from the video that an object has been released from the customer's hand (whose skeletal structure is estimated), counts a first number which is the number of products placed in the storage compartment, acquires a second number which is the number of products read by the scanner by the time registration is complete, and outputs the identification number, the first number, and the second number, provided that the first number and the second number do not match. A terminal device that is held by a store employee and provides notification on the condition that it has obtained the identification number, the first quantity, and the second quantity output by the product registration support device, A product registration device for registering products scanned by the aforementioned scanner, The product registration device comprises an accounting device that performs payment processing for products registered by the product registration device. Product registration support system.

7. A computer that controls a product registration support device that assists the operation of a product registration device that registers a product by reading information that uniquely identifies the product to be purchased, A video acquisition unit acquires an image captured by a camera that images the storage compartment containing the product from which the aforementioned information was read, and an identification number that identifies the storage compartment. A counting unit counts a first number, which is the number of products placed in the storage unit, by detecting from the video acquired by the video acquisition unit that an object has been separated from the customer's hand, whose skeleton has been estimated. The aforementioned product registration device includes an acquisition unit that acquires a second number, which is the number of products read up to the completion of registration, A program that functions as a product quantity comparison information output unit that outputs the identification number, the first quantity, and the second quantity to a terminal device for notification, provided that the first quantity and the second quantity do not match.

Images