Monitoring device and program

Abstract

To enable monitoring of an occurrence state of losing an article in a settlement device corresponding to a full-self service.SOLUTION: A monitoring device includes storage means, discovery means, and notification means. The storage means stores a reference image obtained by photographing a reference area including at least first and second commodity placement tables of a settlement device when the settlement device is in an idling state, the settlement device including the first commodity placement table to place a purchased commodity before registered by a customer and a second commodity placement table being a location to place a purchased commodity after registered by the customer. The discovery means defines the image obtained by photographing the reference area when the customer leaves the settlement device after terminating a settlement in the settlement device as a comparison image so as to compare the image with the reference image, and then, discovers a lost article left on the first and second commodity placement tables of the settlement device. The notification means notifies a customer of the discovery when the lost article left in the settlement device is discovered.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a monitoring device and a program. [Background technology]

[0002] In recent years, self-service point-of-sale (POS) terminals have been gaining attention in supermarkets and other retail stores from the perspectives of reducing labor costs and preventing the spread of infectious diseases. Self-service POS terminals are fully self-service payment devices that allow customers to perform all operations themselves, from registering purchased items to paying. Technologies have been developed for such fully self-service payment devices that can prevent fraudulent behavior by customers.

[0003] On the other hand, with fully self-service payment devices, new problems have arisen because customers operate the POS terminal themselves. One of these is the problem of lost property. When customers operate a self-service POS terminal, they sometimes temporarily place their personal belongings (umbrellas, bags, wallets, etc.) or registered products in any position on the terminal that does not interfere with their operation, and then forget about them. When such lost property is found at a self-service POS terminal, an employee (attendant) must quickly secure and process it, but often the lost property goes unnoticed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 7-141569 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments of the present invention is to provide a monitoring device that can monitor the occurrence of lost items in a fully self-service settlement device. [Means for solving the problem]

[0006] In one embodiment, the monitoring device comprises: a storage means; A detection means; The system includes a detecting means and a notifying means. The storage means stores a reference image obtained by capturing a reference area including at least the first and second product placement tables of the payment device when the payment device is in an idle state. The reference image includes a first product placement table on which a customer places purchased products before registration and a second product placement table on which a customer places purchased products after registration. Detection Method After completing the payment at the payment device The detecting means detects when the customer leaves the payment device. The customer left the payment device It was detected that The lost item left on the first and second product placement tables of the settlement device is discovered by comparing an image of the reference area photographed at that time with the reference image as a comparison image. The notification means notifies the fact that the lost item left at the settlement device has been discovered. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing a fully self-service checkout system. [Figure 2] FIG. 2 is a diagram showing the external configuration of the self-service POS terminal. [Figure 3] FIG. 3 is a block diagram showing the main circuit configuration of the monitoring device according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing an example of a main information processing procedure executed by the processor of the monitoring device in accordance with the monitoring program. [Figure 5] FIG. 5 is a flowchart showing an example of a procedure of main information processing executed by the processor of the monitoring device in accordance with the monitoring program. [Figure 6] FIG. 6 is a flowchart showing an example of a procedure for main information processing executed by the processor of the monitoring device in accordance with the monitoring program. [Figure 7] FIG. 7 is a schematic diagram showing the reference area. [Figure 8] FIG. 8 is a schematic diagram showing a behavior detection area. [Figure 9] FIG. 9 is a schematic diagram showing an article detection area. [Figure 10] FIG. 10 is a schematic diagram showing an example of a display of a warning screen on a self-service POS terminal. [Figure 11] FIG. 11 is a schematic diagram showing a lost-item detection area. [Figure 12] FIG. 12 is a schematic diagram showing an example of a display of a report screen on the attendant terminal. [Figure 13] FIG. 13 is a schematic diagram showing an example of a warning screen displayed on a self-service POS terminal. [Figure 14] FIG. 14 is a schematic diagram showing an example of a warning screen displayed on the attendant terminal. [Figure 15] FIG. 15 is a flowchart showing an example of a main information processing procedure executed by a processor of a monitoring device according to a modification in accordance with a monitoring program. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, the embodiments will be described with reference to the drawings.

[0009] [Checkout system explanation] 1 is a schematic diagram showing a configuration of a full self-service checkout system 100. The checkout system 100 includes a plurality of self-service POS terminals 10, an attendant terminal 20, and a store server 30. The checkout system 100 connects each of the self-service POS terminals 10, the attendant terminal 20, and the store server 30 via a communication network 40 such as a LAN (Local Area Network). The store server 30 provides various services related to the checkout system 100 to each of the self-service POS terminals 10 and the attendant terminals 20 connected via the communication network 40.

[0010] The multiple self-service POS terminals 10 are installed at the checkout area of ​​the store, and are operated by customers who have finished shopping. Customers who place their purchases in a shopping basket or shopping cart in a sales area where products are displayed and proceed to the checkout area operate the self-service POS terminal 10 to pay for the purchased products, that is, to settle the transaction from product registration to payment, all by themselves. The self-service POS terminal 10 is an example of a fully self-service payment device. The payment device can also be referred to as a payment terminal, a payment device, an accounting terminal, a product registration device, a product registration terminal, etc.

[0011] The attendant terminal 20 is installed at a monitoring station near the exit of the checkout area. The attendant terminal 20 displays a list of purchased items registered for each self-service POS terminal 10 on a display device. The attendant terminal 20 also reports error information if an error occurs at the self-service POS terminal 10. An employee at the monitoring station, i.e., an attendant, monitors the checkout area based on the information displayed and reported on the attendant terminal 20. While FIG. 1 shows one attendant terminal 20, multiple attendant terminals 20 may be connected to the communication network 40. At least one of the attendant terminals 20 may be installed in a location other than the monitoring station, such as a back office on another floor of the same store or a help desk. Even attendant terminals 20 located far from the self-service POS terminal 10 can report error information from the self-service POS terminal 10 via the communication network 40. Therefore, even employees who cannot directly check the self-service POS terminal 10 can monitor the self-service POS terminal 10.

[0012] Each self-service POS terminal 10 includes a monitoring device 50. The monitoring device 50 may be located either inside or outside the self-service POS terminal 10. The monitoring device 50 has the function of monitoring whether a customer operating the self-service POS terminal 10, i.e., a person making a payment, has left behind or lost an item. The monitoring device 50 is connected to a camera 60 and a motion sensor 70 installed near the self-service POS terminal 10. The motion sensor 70 may be installed in the self-service POS terminal 10, and the monitoring device 50 may be configured to receive a detection output from the motion sensor 70 of the self-service POS terminal 10. The monitoring device 50 activates the camera 60 to capture an image of the self-service POS terminal 10 in response to detection of a person making a payment by the motion sensor 70. The monitoring device 50 monitors whether an item has been lost at the self-service POS terminal 10 based on information such as images captured by the camera 60, and issues an alert when an item has been lost. The monitoring device 50 may also issue an alert when an item is likely to be lost. The alert is sent to either or both of the self-service POS terminal 10 and the attendant terminal 20. Each monitoring device 50 is connected to the attendant terminal 20 via the communication network 40 and can output an alert signal to the attendant terminal 20. Each monitoring device 50 may also be connected to the attendant terminal 20 without going through the communication network 40.

[0013] The monitoring device 50 may also have a function for monitoring fraudulent behavior of the settlement person. Specifically, the monitoring device 50 monitors the settlement person who operates the self-service POS terminal 10 to settle a transaction based on information such as images captured by a camera 60 installed near the self-service POS terminal 10, and issues an alert if the settlement person engages in fraudulent behavior. Fraudulent behavior is, for example, bagging purchased items without registering them in the self-service POS terminal 10. Fraudulent behavior is, for example, leaving the checkout area without paying for the purchased items.

[0014] The store server 30 manages a product database. The product database is a collection of product records created for each product sold in the store. The product records include data such as product code, product name, and price. The product code is a unique code assigned to each product to identify it individually. Each product is affixed with a barcode that represents the product code of that product. The product name and price are the unique name of the product identified by the product code and the sales price per item.

[0015] [Self-POS terminal explanation] FIG. 2 is a diagram showing the external configuration of the self-service POS terminal 10. The self-service POS terminal 10 includes a main body 101, and a first product placement table 102 and a second product placement table 103, which are arranged on the left and right sides of the main body 101. The first product placement table 102, which is arranged on the right side as you face the main body 101, is a table on which a customer who is making a payment places purchased products before registration. Purchased products before registration are usually placed in a shopping basket and placed on the first product placement table 102. Purchased products before registration may also be placed in a shopping cart and placed in front of the first product placement table 102. Note that the installation positions of the first product placement table 102 and the second product placement table 103 may be reversed.

[0016] The second product placement table 103, located on the left side of the main body 101, is a table on which the payment person places purchased items after registering them. After registration, the purchased items are placed in a container placed on the second product placement table 103, such as a shopping basket, a plastic bag, or a personal bag. After registration, the purchased items may also be placed directly on the second product placement table 103. In FIG. 2, the second product placement table 103 has a temporary placement table 104 attached to its upper part via a support pillar. The temporary placement table 104 is, for example, a table on which items are temporarily placed before being placed in a plastic bag or the like. The temporary placement table 104 has two holding arms 105 spaced apart on the left and right. Using these holding arms 105, the payment person can place an open plastic bag, personal bag, or the like on the second product placement table 103.

[0017] The main body 101 includes a touch panel 11 as an input / output device, a card reader 12, a printer 13, a scanner 14, a hand scanner 15, a cash processing machine 16, a patrol lamp 17, a light emitting unit 18, a speaker 19, and the like.

[0018] The touch panel 11 includes a display as a display device and a touch sensor as an input device. The display shows various screen information such as a checkout start screen, a product registration screen, a payment method selection screen, and a checkout end screen. The checkout start screen is a screen that accepts the start of payment for a transaction. A start key is provided on the checkout start screen to command the start of payment for a transaction. A payer who wants to start registering purchased products touches the start key. When the touch sensor detects that the start key has been touched, the controller of the self-service POS terminal 10 switches the display screen from the checkout start screen to the product registration screen.

[0019] The product registration screen displays detailed information such as the product name, price, and total amount of the registered purchased products. A checkout key is provided on the product registration screen to instruct the user to proceed to payment for the registered purchased products. After registering the purchased products, the user touches the payment key. When the touch sensor detects that the checkout key has been touched, the controller of the self-service POS terminal 10 switches the display screen from the product registration screen to a payment method selection screen.

[0020] The payment method selection screen is a screen that accepts the selection input of a payment method, such as cash payment, credit card payment, electronic money payment, or code payment. The payment method selection screen has a cash key for issuing a cash payment command, a credit key for issuing a credit card payment command, an electronic money key for issuing an electronic money payment command, and a code payment key for issuing a code payment command. The person making the payment touches the key for the desired payment method. For example, when the touch sensor detects that the cash key has been touched, the controller of the self-service POS terminal 10 executes a cash payment process and switches the display screen from the payment method selection screen to a cash transaction completion screen. When the touch sensor detects that the credit key has been touched, the controller of the self-service POS terminal 10 executes a credit card payment process and switches the display screen from the payment method selection screen to a credit card transaction completion screen. When the touch sensor detects that the electronic money key has been touched, the controller of the self-service POS terminal 10 executes an electronic money card payment process and switches the display screen from the payment method selection screen to an electronic money card transaction completion screen. When the touch sensor detects that the code payment key has been touched, the controller of the self-service POS terminal 10 executes the payment process using the code for code payment and switches the display screen from the payment method selection screen to the transaction completion screen using code payment.

[0021] For example, a cash transaction completion screen is a screen that displays the total transaction amount, the amount of cash received, the amount of change, etc. For example, a cash settlement process is a process in which the total transaction amount is subtracted from the amount received to calculate the change amount, and a receipt is issued to complete the transaction. Since this type of cash settlement process is well known, a detailed description will be omitted. Furthermore, since payment processes using credit cards, electronic money cards, or code payments are also well known, a description will be omitted here.

[0022] The card reader 12 is a device that reads data recorded on a card medium such as a credit card, electronic money card, or point card. If the card medium is a magnetic card, the card reader 12 is a magnetic card reader. If the card medium is an IC card, the card reader 12 is an IC card reader. The self-service POS terminal 10 may be equipped with either a magnetic card reader or an IC card reader as the card reader 12, or may be equipped with both. The card reader 12 may also be a card reader / writer that has a function for writing data to a card.

[0023] The printer 13 is a device that prints data related to purchase receipts, credit card slips, etc. onto receipt paper. The receipt paper on which various data has been printed by the printer 13 is cut by a cutter and issued from the receipt issuing slot. The printer 13 may be implemented as, for example, a thermal printer or a dot-matrix printer.

[0024] The scanner 14 and the hand scanner 15 are used to read information identifying a product. In this embodiment, the scanner 14 and the hand scanner 15 read barcodes attached to products as information identifying the products. The scanner 14 optically reads the barcode of a product held up against a reading window (glass window) so that the barcode on the product faces the scanner 14. The hand scanner 15 is handheld and operated by the person making the payment, and optically reads the barcode when brought close to the barcode on the product. The scanner 14 and the hand scanner 15 may be used to read information identifying a product, and may also be used to read information other than a barcode, such as a two-dimensional code.

[0025] The cash processing machine 16 processes cash. The cash processing machine 16 has a banknote input port 161, a banknote discharge port 162, a coin input port 163, and a coin discharge port 164. The cash processing machine 16 processes banknotes inserted into the banknote input port 161. The cash processing machine 16 discharges banknotes as change from the banknote discharge port 162. The cash processing machine 16 accepts and processes coins inserted into the coin input port 163. The cash processing machine 16 discharges coins as change from the coin discharge port 164.

[0026] The patrol lamp 17 is attached to the tip of a vertically extending pole so that it can be seen from a distance. The patrol lamp 17 includes a light-emitting member and a cylinder. The light-emitting member is attached inside the cylinder so that it can rotate coaxially with the cylinder. When the patrol lamp 17 is activated, it can rotate while emitting light from the light-emitting member. The patrol lamp 17 is used to display an alert. For example, the patrol lamp 17 lights up the light-emitting member when an item is registered, allowing the person making the payment or an employee at the attendant terminal 20 to visually confirm whether the item has been scanned properly.

[0027] The light-emitting unit 18 is disposed on the lower part of the patrol lamp 17 on the pole. Like the patrol lamp 17, the light-emitting unit 18 also includes a light-emitting member and a cylinder. The light-emitting member can selectively emit light in multiple colors. Alternatively, the light-emitting unit 18 is provided with multiple light-emitting members of different light colors. Like the patrol lamp 17, the light-emitting unit 18 is also intended to display alerts. The light-emitting unit 18 can display alerts such as an error, the detection of fraudulent behavior by the payment person, etc., by the light-emitting color of the light-emitting member or by the light-emitting color and its flashing state.

[0028] The speaker 19 outputs a warning sound as an alert or a voice message or the like.

[0029] A camera 60 serving as an imaging device is attached to the upper part of the main body 101. The camera 60 photographs the self-service POS terminal 10 and the person operating it. As shown in FIG. 2, the display of the touch panel 11, the card insertion slot of the card reader 12, the receipt issuing slot of the printer 13, the reading window of the scanner 14, and the bill insertion slot 161, bill ejection slot 162, coin insertion slot 163, and coin ejection slot 164 of the cash processing machine 16 are arranged on one side (hereinafter referred to as the front) of the main body 101. Therefore, the person operating the self-service POS terminal 10 stands facing the front of the main body 101. The camera 60 photographs almost the entire body of the person operating the self-service POS terminal 10 from above. Therefore, the head, face, shoulders, arms, chest, abdomen, etc. of the person operating the self-service POS terminal 10 are photographed by the camera 60.

[0030] A human presence sensor 70 is attached to the front of the main body 101 of the self-service POS terminal 10. The human presence sensor 70 transmits a detection output to the monitoring device 50 while a person making a payment is present within the detection range on the front side of the self-service POS terminal 10. The human presence sensor 70 is also configured to transmit the detection output to the self-service POS terminal 10, so that the self-service POS terminal 10 transitions to an idle state based on the detection output while a person is not present. The idle state is a standby state in which power is saved by, for example, turning off the display of the touch panel 11. When the self-service POS terminal 10 recognizes the presence of a person making a payment based on the detection output of the human presence sensor 70, it can play audio guidance through the speaker 19 to prompt the person to perform the first operation.

[0031] [Configuration of monitoring device] 3 is a block diagram showing the main circuit configuration of the monitoring device 50. The monitoring device 50 includes a processor 51, a main memory 52, an auxiliary storage device 53, a camera interface 54, a sensor interface 55, a terminal interface 56, a network interface 57, and a system transmission path 58. The system transmission path 58 includes an address bus, a data bus, a control signal line, etc. The system transmission path 58 connects the processor 51 to each of the other components and transmits data signals exchanged between them.

[0032] The monitoring device 50 is configured as a computer by connecting a processor 51, a main memory 52, an auxiliary storage device 53, a camera interface 54, a sensor interface 55, a terminal interface 56, and a network interface 57 via a system transmission line 58.

[0033] The processor 51 corresponds to the central part of the computer. The processor 51 controls each part to realize various functions of the monitoring device 50 in accordance with an operating system or application programs. The processor 51 is, for example, a CPU (Central Processing Unit). The processor 51 may be a multi-core / multi-thread processor, and can execute multiple processes in parallel.

[0034] The main memory 52 corresponds to the main storage portion of the computer. The main memory 52 includes a nonvolatile memory area and a volatile memory area. The main memory 52 stores an operating system or application programs in the nonvolatile memory area. The main memory 52 may store data required for the processor 51 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 52 uses the volatile memory area as a work area where data is rewritten by the processor 51 as appropriate. The nonvolatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0035] The auxiliary storage device 53 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disk Drive), or an SSD (Solid State Drive) can be the auxiliary storage device 53. The auxiliary storage device 53 stores data used by the processor 51 when performing various processes, data created by the processes in the processor 51, etc. The auxiliary storage device 53 may also store the application programs described above.

[0036] The camera interface 54 is connected to the camera 60. The camera interface 54 outputs a control signal to the camera 60 to control the driving of the camera 60. The camera 60 performs a photographing operation in accordance with this control signal. The camera interface 54 captures image data photographed by the camera 60.

[0037] The sensor interface 55 connects to the human presence sensor 70. The sensor interface 55 receives the sensing output from the human presence sensor 70.

[0038] The terminal interface 56 connects to the self-service POS terminal 10. The terminal interface 56 communicates data with the self-service POS terminal 10. The terminal interface 56 receives operation input data and the like input by the user into the self-service POS terminal 10. The operation input data includes operation input data related to starting use, operation input data related to product registration, and operation input data related to a payment declaration. The operation input data related to starting use is data output in response to the user pressing the start key on the payment start screen. The operation input data related to product registration is data output in response to the user holding the barcode of a product over the reading window of the scanner 14 or holding the reading part of the hand scanner 15 over the barcode. The data includes product record data including the product code represented by the barcode scanned by the scanner 14 or the hand scanner 15. The operation input data related to a payment declaration is data output in response to the user pressing the payment key on the product registration screen.

[0039] The terminal interface 56 also takes in control operation input data and the like input by an employee to the self-service POS terminal 10. The control operation input data includes operation input data related to starting the operation of the self-service POS terminal 10, operation input data related to unlocking, and the like. The operation input data related to unlocking is data that is output in response to an employee performing a predetermined operation to unlock the self-service POS terminal 10 that has been locked due to the occurrence of an error, the detection of fraudulent behavior, or the like.

[0040] The network interface 57 connects to the attendant terminal 20 via the communication network 40. The network interface 57 performs data communication with the attendant terminal 20.

[0041] In the monitoring device 50 configured as above, part of the volatile memory area in the main memory 52 is used as an idle image memory 521 and an active image memory 522. The uses of the idle image memory 521 and the active image memory 522 will be made clear in the explanation of operation given below.

[0042] The processor 51 has at least the functions of a behavior detection unit 511, a discovery unit 512, an alarm unit 513, and a prohibition unit 514. The behavior detection unit 511 analyzes the behavior of the settlement person based on images captured by the camera 60 and detects fraudulent behavior that the settlement person is monitoring. The discovery unit 512 discovers items lost by the settlement person based on images captured by the camera 60. The discovery unit 512 can be referred to as a discovery means. The detailed function of the discovery unit 512 will be explained in the operation description below. The notification unit 513 provides an alert when a lost item is discovered. The notification unit 513 can be referred to as an alarm means. The detailed function of the notification unit 513 will also be explained in the operation description below. The prohibition unit 514 has a function of prohibiting the self-service POS terminal 10 from registering items to be purchased by the next customer when the discovered lost item is an item other than an empty shopping basket. The prohibition unit 514 can be referred to as a prohibition means. The detailed function of the prohibition unit 514 will also be made clear in the explanation of operation given later.

[0043] The functions of the behavior detection unit 511, the discovery unit 512, the notification unit 513, and the prohibition unit 514 are realized by information processing that the processor 51 executes in accordance with a monitoring program.

[0044] The monitoring program is a type of application program stored in the main memory 52 or the auxiliary storage device 53. There are no particular limitations on the method for installing the monitoring program in the main memory 52 or the auxiliary storage device 53. The settlor monitoring program can be recorded on a removable recording medium, or can be distributed by communication via a network and installed in the main memory 52 or the auxiliary storage device 53. The recording medium can be in any form, such as a CD-ROM or memory card, as long as it can store the program and is readable by the device.

[0045] In addition, some of the functions of the behavior detection unit 511, the discovery unit 512, the notification unit 513, and the prohibition unit 514 may be realized in various other forms, including integrated circuits such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), an FPGA (Field-programmable gate array), and a DSP (Digital Signal processor).

[0046] [Explanation of the monitoring device's function] Next, the functions of the processor 51 of the monitoring device 50 as the behavior detection unit 511, the discovery unit 512, the notification unit 513, and the prohibition unit 514 will be described with reference to the flowcharts in Figures 4 to 6. Figures 4 to 6 are flowcharts showing the main information processing procedures executed by the processor 51 of the monitoring device 50 in accordance with the monitoring program.

[0047] In ACT1, the processor 51 determines whether there is a detection output from the human presence sensor 70. This is an operation to confirm that there is no detection output from the human presence sensor 70, that is, that an employee or a settlement person is not facing the self-service POS terminal 10. If the processor 51 confirms that there is no detection output from the human presence sensor 70, it proceeds to ACT2.

[0048] Processor 51 outputs a start signal to camera 60 in ACT2. Camera 60 starts capturing images upon receiving the start signal. Processor 51 captures an idle image captured by camera 60 in ACT3 and stores it in idle image memory 521 of main memory 52. ​​Processor 51 then outputs a stop signal to camera 60 in ACT4. Camera 60 stops capturing images upon receiving the stop signal.

[0049] 7 is a schematic diagram showing a reference area AR included in an idol image. The reference area AR is an area including the entire self-service POS terminal 10, including the first product placement table 102 and the second product placement table 103 of the self-service POS terminal 10, and a payment person who is not present at this time. The idol image, which is an image of the reference area AR taken when no payment person is present, serves as a reference image when detecting lost items.

[0050] Thereafter, in ACT5, the processor 51 waits for a detection output to be transmitted from the human presence sensor 70. That is, the processor 51 waits for a settlement person to approach the self-service POS terminal 10. If a detection output is transmitted from the human presence sensor 70, the processor 51 proceeds to ACT6.

[0051] In ACT6, processor 51 outputs a start signal to camera 60. Camera 60 starts capturing images upon receiving the start signal. In ACT7, processor 51 captures an image during operation captured by camera 60 and stores it in image-in-operation memory 522 in main memory 52.

[0052] 8 is a schematic diagram showing a behavior detection area AD included in the image during operation. The behavior detection area AD corresponds to the reference area AR and is an area including the entire self-service POS terminal 10 including the first product placement table 102 and the second product placement table 103 of the self-service POS terminal 10, and a not-shown payer facing the front of the self-service POS terminal 10.

[0053] Then, in ACT 8, processor 51 detects the behavior of the person making the payment based on the operating images stored in operating image memory 522, and determines whether or not there has been any behavior to be monitored in ACT 9. If there is no behavior to be monitored, processor 51 proceeds to ACT 10. In ACT 10, processor 51 checks whether the operation input data acquired from self-service POS terminal 10 is operation input data related to a payment declaration, thereby determining whether or not to start the payment. If the payment is not to be started, processor 51 returns to ACT 7 and repeats the above operations.

[0054] In the behavior detection of ACT8, the processor 51 recognizes the behavior of the person making the payment who is facing the self-service POS terminal 10 based on the images of the person in motion. Specifically, the processor 51 estimates the person's skeleton from the images of the person in motion. Skeleton estimation is performed using AI technology such as deep learning. The processor 51 obtains hand movements, etc. from consecutive images of the person in motion by, for example, skeletal estimation. Because multiple images of the person in motion are used in this manner, an area capable of storing multiple images is secured in the main memory 52 as the image memory 522 of the person in motion. The processor 51 then recognizes the behavior of the person taking out, registering, packing, or leaving from the obtained hand movements, etc. Note that the processor 51 may also recognize the behavior of the person making the payment by motion detection or the like that detects the operator's movements.

[0055] A take-out action is an action in which a payer takes out an item from a shopping basket or shopping cart placed on the first item placement table 102. For example, an action in which a payer takes out an item from the first item placement table 102 before the item is registered and puts it in a customer's pocket without putting it in a bag is also a take-out action. A registration action is an action in which a payer holds an item over the reading window of the scanner 14 to have the scanner 14 read the barcode attached to the item. A bagging action is an action in which a payer places an item in a container prepared on the second item placement table 103. Note that an action in which a payer places an item on the second item placement table 103 or the temporary placement table 104 without putting it in a container is also a type of bagging action. An exit action is an action in which a payer leaves in front of the self-service POS terminal 10 to exit the checkout area.

[0056] In ACT9, the processor 51 determines whether the person making the payment has engaged in fraudulent behavior based on the detected behavior and the operation input data from the self-service POS terminal 10. Fraudulent behavior includes, for example, bagging purchased items without registering them in the self-service POS terminal 10, leaving the checkout area without paying for the purchased items, etc.

[0057] If it is determined in ACT9 that a monitored behavior has occurred, the processor 51 outputs a warning signal to the attendant terminal 20 in ACT11. The warning signal includes identification information for identifying the self-service POS terminal 10 and information indicating the monitored behavior. Upon receiving this warning signal, the attendant terminal 20 displays or outputs a warning message to alert the employee, such as, for example, "There is a missed registration in lane No. X" or "There is a customer attempting to exit lane No. X without paying." Although not specifically illustrated, the processor 51 may also output a warning signal to the self-service POS terminal 10, so that a warning is displayed on the self-service POS terminal 10. This allows the customer to rescan a product if, for example, a product is not registered due to a scanning error. After outputting the warning signal in ACT11, the processor 51 returns to ACT7 and repeats the above operations.

[0058] When the payment person completes the registration of the purchased items at the self-service POS terminal 10 and performs a checkout start operation to instruct the payment to begin, operation input data relating to the payment declaration is sent from the self-service POS terminal 10 to the monitoring device 50. Upon receiving this operation input data relating to the payment declaration, the processor 51 determines in ACT10 that the payment should begin, and proceeds to ACT12.

[0059] The processor 51 captures an image during operation taken by the camera 60 as ACT12, and stores the image during operation in the image memory 522 of the main memory 52.

[0060] Then, processor 51 executes a first comparison process as ACT 13. In this first comparison process, processor 51 uses an image in the item detection area of ​​the operating image as a comparison image and compares it with the idle image, which is a reference image stored in idle image memory 521. Hereinafter, for convenience, the first comparison process will also be referred to as the first comparison.

[0061] 9 is a schematic diagram showing an item detection area AO included in the image during operation. The item detection area AO is a comparison area that does not include the second item placement table 103 of the self-service POS terminal 10, but includes at least the first item placement table 102.

[0062] In ACT14, the processor 51 determines whether or not an item other than an empty shopping basket was present in the item detection area AO as a result of the first comparison. Specifically, the processor 51 determines that an item exists in the portion where the images do not match in the first comparison, and further analyzes the image of the item to confirm whether or not the item is an empty shopping basket. The external shape of an empty shopping basket is constant. Therefore, the processor 51 uses pattern recognition technology using external shape data of an empty shopping basket to confirm whether or not the item is an empty shopping basket. If the processor 51 determines that the item is an item other than an empty shopping basket, that is, if the processor 51 finds an item remaining in the portion of the self-service POS terminal 10 that includes at least the first product tray 102 but excludes the second product tray 103, the processor 51 proceeds to ACT15.

[0063] The processor 51 outputs an attention-calling command to the self-service POS terminal 10 as ACT15. Upon receiving this attention-calling command, the self-service POS terminal 10 displays an attention-calling screen on the display of the touch panel 11. At this time, a warning sound may be emitted from the speaker 19.

[0064] 10 is an example of the attention screen Sa1. The attention screen Sa1 displays an attention message Ma1 and a "Confirm" key Ka. The attention message Ma1 is a message written in a gentle manner, such as "Please be careful not to leave anything behind."

[0065] The processor 51 checks whether or not a confirmation operation has been performed by the settlor in ACT 16. If no confirmation operation has been performed, the processor 51 returns to ACT 12 and repeats the operations from ACT 12 to ACT 16. If a confirmation operation has been performed by the settlor, the processor 51 proceeds to ACT 17.

[0066] In response to the attention message Ma1 on the attention screen Sa1, the settlor checks the items remaining on the self-service POS terminal 10 and touches the "Confirm" key Ka. In response to touching the "Confirm" key Ka, the self-service POS terminal 10 transmits operation input data relating to the operation of the "Confirm" key Ka to the monitoring device 50 and erases the attention screen Sa1 on the display. In ACT16, the processor 51 checks that a confirmation operation has been performed from the self-service POS terminal 10 using the operation input data, and proceeds to ACT17.

[0067] Alternatively, the settlor removes the item from the self-service POS terminal 10 in response to the attention message Ma1 on the attention screen Sa1. As a result, the processor 51 determines in ACT14 as a result of the first comparison that there is nothing in the item detection area AO or that there is only an empty shopping basket. Based on this determination, the processor 51 proceeds to ACT17. In this case, although not specifically shown in the figure, the processor 51 transmits an attention output cancellation command to erase the attention screen Sa1 of the self-service POS terminal 10.

[0068] In ACT 17, the processor 51 captures an in-motion image captured by the camera 60 and stores it in the in-motion image memory 522 of the main memory 52. ​​Then, in ACT 18, the processor 51 detects the behavior of the person making the payment based on the in-motion image stored in the in-motion image memory 522, and determines whether or not there was any behavior to be monitored in ACT 19. If there is no behavior to be monitored, the processor 51 determines whether or not the payment has been completed based on the status data from the self-service POS terminal 10 in ACT 20. If the payment has not been completed, the processor 51 returns to ACT 17 and repeats the above operations. Note that the behavior detection in ACT 18 and the determination of the behavior to be monitored in ACT 19 are the same as those in ACT 8 and ACT 9 described above.

[0069] If it is determined in ACT19 that a behavior to be monitored has occurred, the processor 51 outputs a warning signal to the attendant terminal 20 in ACT21. After outputting the warning signal in ACT21, the processor 51 returns to ACT17 and repeats the above operations.

[0070] If it is determined in ACT20 that the payment has been completed, the processor 51 proceeds to ACT22. In ACT22, the processor 51 determines whether there is a detection output from the human presence sensor 70. This is an operation to confirm that there is no detection output from the human presence sensor 70, in other words, that the person making the payment has moved away from in front of the self-service POS terminal 10. If the processor 51 confirms that there is no detection output from the human presence sensor 70, it proceeds to ACT23.

[0071] The processor 51 captures the operating image captured by the camera 60 as ACT23 and stores it in the operating image memory 522 of the main memory 52. ​​In other words, the operating image stored in the operating image memory 522 is an image captured at the time when the human sensor 70 no longer outputs a detection output and the settlement person moves away from the self-service POS terminal 10.

[0072] Then, processor 51 executes a second comparison process as ACT 24. In this second comparison process, processor 51 uses an image in the lost item detection area among the operating images as a comparison image and compares it with the idle image, which is a reference image, stored in idle image memory 521. Hereinafter, the second comparison process will also be referred to as the second comparison for convenience.

[0073] 11 is a schematic diagram showing a lost-item detection area AL included in the image during operation. The lost-item detection area AL corresponds to the reference area AR and is a comparison area including the entire self-service POS terminal 10, including the first product placement table 102 and the second product placement table 103 of the self-service POS terminal 10.

[0074] In ACT 25, processor 51 determines whether any item was found in lost property detection area AL as a result of the second comparison. Specifically, processor 51 determines that any part of the image that does not match in the second comparison indicates the presence of an item. If no such item is found, processor 51 returns to ACT 5 and proceeds to operation for a new settler.

[0075] On the other hand, if any item is found, processor 51 proceeds to ACT26. Processor 51 determines whether the item found in ACT26 is an item other than an empty shopping basket. As described above, processor 51 can determine whether the item is an empty shopping basket by using pattern recognition technology that uses external shape data of an empty shopping basket. If the item is an empty shopping basket, processor 51 proceeds to ACT27.

[0076] The processor 51 outputs a report signal to the attendant terminal 20 in ACT27. The report signal includes identification information that identifies the self-service POS terminal 10 and information that indicates that an empty shopping basket has been left behind. In this case, a message may be displayed on the display of the touch panel 11 or spoken from the speaker 19 to request the settlement person, who has moved away from the front of the self-service POS terminal 10 but has not yet gone far from the self-service POS terminal 10, to put away the empty shopping basket. Thereafter, the processor 51 returns to ACT5 and proceeds to an operation for a new settlement person.

[0077] Upon receiving the report signal, the attendant terminal 20 displays a report screen. FIG. 12 shows an example of the report screen Sb1. The report screen Sb1 displays a report message Mb1 and a "Confirm" key Kb. The empty shopping basket may be cleared by the person who returned, by the next person who cleared it, or by another person stacking it on top of the other. Therefore, the report message Mb1 may be a message to alert the employee, such as "There is an empty shopping basket left in lane No. X," rather than necessarily instructing the employee to clear the empty shopping basket. Upon receiving the report message Mb1, the employee can operate the "Confirm" key Kb to confirm that they have confirmed the notification. Furthermore, if the employee is in a position to clear the empty shopping basket, they can go to the self-service POS terminal 10 and clear the empty shopping basket.

[0078] Furthermore, if it is determined in ACT26 that an item other than an empty shopping basket has been found, or that both an empty shopping basket and other items have been found, the processor 51 proceeds to ACT28. The processor 51 outputs an alarm command to the self-service POS terminal 10 in ACT28. Furthermore, the processor 51 outputs an operation prohibition command to the self-service POS terminal 10 in ACT29.

[0079] In response to receiving the alarm command, the self-checkout POS terminal 10 displays a warning screen on the display of the touch panel 11. At this time, a warning sound may also be emitted from the speaker 19. Furthermore, in response to receiving the operation prohibition command, the self-checkout POS terminal 10 transitions to a locked state in which no operation by the payer is accepted. In this way, the monitoring device 50 can prohibit the next customer from registering the purchased items.

[0080] 13 is an example of the warning screen Sa2. A warning message Ma2 is displayed on the warning screen Sa2. This warning message Ma2 can include both a message for the settlement person who has moved away from the front of the self-service POS terminal 10 but has not yet gone far from the self-service POS terminal 10, and a message for the next settlement person who was about to use the self-service POS terminal 10. The former may be, for example, "You left something behind," and the latter may be, for example, "An employee will be on his way now. Please wait a moment."

[0081] Furthermore, the processor 51 outputs a warning signal as ACT30 to the attendant terminal 20. The warning signal includes identification information for identifying the self-service POS terminal 10 and information indicating that a lost item has been found.

[0082] Thereafter, processor 51 outputs a stop signal as ACT31 to camera 60. Camera 60 receives this stop signal and ends shooting.

[0083] Then, the processor 51 waits for a release signal to be transmitted from the self-checkout POS terminal 10 in ACT32.

[0084] Upon receiving the warning signal, the attendant terminal 20 displays a warning screen and sounds a warning sound. FIG. 14 is an example of the warning screen Sb2. The warning screen Sb2 displays a warning message Mb2 and a "Confirm" key Kb. The warning message Mb2 may instruct an employee to retrieve the lost item and unlock the self-service POS terminal 10, for example, by saying, "A customer has left an item in lane No. X. Please retrieve the item immediately and unlock the POS terminal." Upon receiving this warning message Mb2, the employee can operate the "Confirm" key Kb, go to the corresponding self-service POS terminal 10, and retrieve the lost item.

[0085] When an employee performs an operation to unlock the self-service POS terminal 10, an unlock signal is transmitted from the self-service POS terminal 10 to the monitoring device 50. When reception of this unlock signal is confirmed in ACT 32, the processor 51 returns to ACT 5 and proceeds to an operation for a new settler.

[0086] Here, the processor 51 realizes the function of a behavior detection unit 511 by processing ACT7 to ACT9 in Fig. 4 and further processing ACT17 to ACT19 in Fig. 5. The processor 51 realizes the function of a discovery unit 512 by processing ACT12 to ACT14 in Fig. 5 and further processing ACT23 to ACT26 in Fig. 6. The processor 51 realizes the function of a notification unit 513 by processing ACT15 in Fig. 5 and further processing ACT27, ACT28, and ACT30 in Fig. 6. The processor 51 realizes the function of a prohibition unit 514 by processing ACT29 in Fig. 6.

[0087] [Effects of surveillance devices] As described above in detail, the monitoring device 50 having the function of the discovery unit 512 can monitor the occurrence of items lost by a payment person by comparing images taken of the self-service POS terminal 10 before and after the payment person uses the self-service POS terminal 10, which is a fully self-service payment device. For example, even if the payment person leaves behind personal belongings such as an umbrella, bag, wallet, or purchased items on the first product stand 102 or the second product stand 103 and moves away from the self-service POS terminal 10, the monitoring device 50 can immediately discover the lost items.

[0088] When the monitoring device 50 having the function of the notification unit 513 finds a lost item, it outputs a warning signal to the attendant terminal 20, thereby displaying a warning message Mb2 informing the attendant that a lost item has been found on the attendant terminal 20. This allows an employee to quickly retrieve the lost item.

[0089] In addition to sending this warning output to the attendant terminal 20, the monitoring device 50 can also output an alarm command to the self-service POS terminal 10, causing the self-service POS terminal 10 to display a warning message Ma2 on the display of the touch panel 11 and to emit a warning sound from the speaker 19. This makes it possible to notify a payer who has moved away from the front of the self-service POS terminal 10 but has not yet gone far from the self-service POS terminal 10 that they have left something behind.

[0090] Furthermore, when a lost item is discovered, the monitoring device 50 outputs an operation prohibition command to the self-service POS terminal 10, thereby causing the self-service POS terminal 10 to transition to a locked state in which operation by the payment person is not accepted. This makes it possible to prohibit the next customer from registering a purchased item until the lost item is recovered. For example, if an item is left behind and it is located on the second product tray 103, it is likely to be an item that has already been purchased, and if it is located on the first product tray 102, it is likely to be an item that has not been purchased. The employee must store the purchased item until the payment person comes to pick it up, and if it is an item that has not been purchased, it must return it to the product shelf. Because the self-service POS terminal 10 is in a registration operation prohibition state and no operation by the next customer is being performed, the employee can immediately confirm which item was left behind.

[0091] If an empty shopping basket is found that is not the personal property of the payer, the monitoring device 50 can output a report signal to the attendant terminal 20, thereby informing the attendant terminal 20 that an empty shopping basket has been left behind. This makes it possible to have an employee go and put away the empty shopping basket if the situation allows.

[0092] Also, for example, when the person making the payment has finished registering the purchased items in the shopping basket and declares that they are ready to pay, the monitoring device 50 can detect any items left in the part of the self-service POS terminal 10 that excludes the second product tray 103 and includes at least the first product tray 102 by comparing the image taken before the person made the payment with an image that includes at least the first product tray 102 but does not include the second product tray 103.

[0093] Then, if the discovered item is an item other than an empty shopping basket, the monitoring device 50 outputs an attention command to the self-service POS terminal 10, causing an attention message Ma1 to be displayed on the self-service POS terminal 10. This makes it possible to remind the payer not to leave anything behind. In addition, since the attention message Ma1 is displayed when there are unregistered purchased items remaining in the shopping basket, it can also serve as a reminder to the payer not to forget to register them.

[0094] It should be noted that no particular warning is issued if the item left in the portion of the self-service POS terminal 10 excluding the second product placement table 103 and including at least the first product placement table 102 is an empty shopping basket. For example, in this case, the settlement person may be asked to put away the empty shopping basket.

[0095] [Variations] In the above embodiment, when a lost item is discovered and the self-service POS terminal 10 is locked, the monitoring device 50 waits for an employee to unlock the self-service POS terminal 10 before proceeding to the next payment customer. A scenario is envisioned in which the payment customer realizes their lost item due to a warning screen on the display of the self-service POS terminal 10 or a warning sound from the speaker 19 in response to the alarm command output in ACT28 of Fig. 6, and returns to retrieve the lost item. In such a case, if the monitoring device 50 can proceed to the next payment customer's operation without waiting for an employee to unlock the self-service POS terminal 10, this can reduce the frustration of the next payment customer waiting for the lock to be released.

[0096] 15 is a flowchart showing an example of a main information processing procedure executed by processor 51 of monitoring device 50 according to a modification in accordance with a monitoring program to enable such an operation. After outputting a warning signal to attendant terminal 20 as ACT 30, processor 51 captures an operating image captured by camera 60 as ACT 41 and stores the image in operating image memory 522 of main memory 52.

[0097] Then, in ACT 42, processor 51 executes a second comparison similar to ACT 24. That is, processor 51 compares the image in the lost item detection area AL among the operating images as a comparison image with the idol image, which is the reference image stored in idle image memory 521.

[0098] In ACT 43, the processor 51 determines whether or not any item was found in the lost item detection area AL as a result of the second comparison. That is, it determines whether or not the lost item found in ACT 25 is still there. If an item is found in the lost item detection area AL, the processor 51 returns to ACT 41.

[0099] When the lost item is retrieved by the settler or employee who left it behind, the processor 51 proceeds from ACT43 to ACT44. In ACT44, the processor 51 outputs an idle transition command to the self-service POS terminal 10. In response to receiving the idle transition command, the self-service POS terminal 10 releases the lock and transitions to the idle state.

[0100] At this time, the processor 51 may be configured to confirm whether the person who retrieved the lost item is the person who made the payment or an employee, and if the lost item is retrieved by someone other than the person who left it, to issue an alarm to the self-service POS terminal 10 and the attendant terminal 20. Whether the person is the person who made the payment and left the item can be determined, for example, by comparing the image of the person who retrieved the lost item acquired in ACT41 with the image of the person who made the payment and left the item stored in the active image memory 522. Whether the person is an employee can be determined, for example, by comparing the clothing of the person who retrieved the lost item in the image acquired in ACT41 with images of clothing of employees registered in advance.

[0101] After outputting an idle transition command to the self-service POS terminal 10 in ACT44, the processor 51 proceeds to ACT45. In ACT45, the processor 51 outputs a stop signal to the camera 60. Upon receiving this stop signal, the camera 60 stops capturing images. Thereafter, the processor 51 returns to ACT5 and proceeds to an operation for a new settler.

[0102] In this way, the monitoring device 50 can move on to the next operation for the person making the settlement, triggered by the recovery of the lost item.

[0103] The procedures and contents of the flowcharts shown in the embodiment and modified examples are examples. The procedures and contents can be changed as appropriate as long as similar effects can be achieved. Furthermore, the screen examples shown in the embodiment are also examples. The layout, display items, etc. can be changed as appropriate as long as similar effects can be achieved.

[0104] In the above embodiment, an example has been given in which each self-service POS terminal 10 is equipped with a monitoring device 50. One monitoring device 50 may be connected to two or more self-service POS terminals 10, and the processor 51 of the monitoring device 50 may have the functions of a behavior detection unit 511, a discovery unit 512, and a notification unit 513 for each of the two or more self-service POS terminals 10.

[0105] In the above embodiment, the lost property is described as the personal property of the customer who made the payment, but the lost property is not limited to this. For example, the printer 13 of the self-service POS terminal 10 may issue point tickets or coupons in addition to receipts or invoices for transactions. If the monitoring device 50 detects that such tickets have been left in the printer 13, it may treat them as lost property and display a warning. Furthermore, the banknote discharge port 162 and the coin discharge port 164, which are the output sections of the cash processing machine 16 of the self-service POS terminal 10, return change or unrecognized currency. Therefore, even if the monitoring device 50 detects that currency or the like has been left in the banknote discharge port 162 or the coin discharge port 164, it may treat them as lost property and display a warning.

[0106] In the above embodiment, it has been explained that the monitoring device 50 may be located either inside or outside the self-service POS terminal 10. In another embodiment, the self-service POS terminal 10 may incorporate the function of the monitoring device 50.

[0107] Furthermore, in the above embodiment, the self-service POS terminal 10, which is a fully self-service payment device, is mentioned, but the operations performed by the payer after moving the shopping basket in a semi-self-service POS terminal, which is a semi-self-service payment device, are the same as those in the self-service POS terminal 10. Therefore, by applying the operations subsequent to the payment process in the monitoring device 50, the same effects can be expected for semi-self-service payment devices.

[0108] Although the present invention has been described with reference to certain embodiments and variations thereof, these embodiments and variations are presented as examples and are not intended to limit the scope of the invention. These novel embodiments and variations may be implemented in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. These embodiments and variations thereof are included within the scope of the invention and the scope of the inventions and their equivalents as set forth in the claims. The inventions described in the original claims of this application are set forth below. [1] A payment device having a first product placement table on which a customer places purchased products before registration and a second product placement table on which the customer places purchased products after registration, when the payment device is in an idle state, includes a storage means for storing a reference image of a reference area including at least the first and second product placement tables of the payment device; a detection means for detecting lost items left on the first and second product placement tables of the payment device by comparing an image of the reference area taken at the time when the customer leaves the payment device after completing payment at the payment device with the reference image; a notification means for notifying the user when the lost item left at the settlement device is found; A monitoring device comprising: [2] The finding means determines, based on the comparison image, whether the lost item is a shopping basket that was left empty to contain the purchased items before the registration; The monitoring device according to [1], wherein the notification means issues a notification depending on whether the lost item is an empty shopping basket. [3] The monitoring device described in [2] further comprises a prohibition means for prohibiting the next customer from registering purchased items in the settlement device when the lost item is an item other than the empty shopping basket. [4] The discovery means: an image of a comparison area of ​​the payment device that includes at least the first product placement tray but does not include the second product placement tray, among images of the reference area taken at the time when the customer has completed registration of the purchased products at the payment device and performed a checkout start operation to instruct the customer to carry out the checkout, is taken as the comparison image, and the image is compared with the reference image to discover any articles left in the part of the payment device that excludes the second product placement tray but includes at least the first product placement tray; determining whether the item is an empty shopping basket that was left behind to contain the purchased items before the registration based on the comparison image; The monitoring device according to [1], wherein the notification means determines the content of the notification and notifies the user based on whether the item is an empty shopping basket or not. [5] The notification means If the item is an empty shopping basket, no notification is made. The monitoring device described in [4] above issues an alert to alert the customer if the item is anything other than an empty shopping basket. [6] A computer that communicates with a payment device, a storage means for storing a reference image of a reference area including at least the first and second product placement tables of the payment device when the payment device is in an idle state, the reference area including the first and second product placement tables on which a customer places purchased products before registration and a second product placement table on which the customer places purchased products after registration; a detection means for detecting lost items left on the first and second product placement tables of the payment device by comparing an image of the reference area taken at the time when the customer leaves the payment device after completing payment at the payment device as a comparison image with the reference image; A program for functioning as a notification means for notifying a user when the lost item left at the settlement device is found. [Explanation of symbols]

[0109] 10...self-service POS terminal, 11...touch panel, 12...card reader, 13...printer, 14...scanner, 15...hand scanner, 16...cash processing machine, 17...patrol lamp, 18...light emitting unit, 19...speaker, 20...attendant terminal, 30...store server, 40...communication network, 50...monitoring device, 51...processor, 52...main memory, 53...auxiliary storage device, 54...camera interface, 55...sensor interface, 56...terminal interface, 57...network interface, 58...system transmission path, 60...camera, 70...human presence sensor, 100...checkout system, 101...main body, 102...first product placement table, 103...second product placement table, 104...temporary placement table, 105...holding arm, 161...banknote insertion slot, 162...banknote discharge slot, 163...coin insertion slot, 164...coin discharge slot, 511...behavior detection unit, 512...detection unit, 513...alarm unit, 514...prohibition unit, 521...idle image memory, 522...operating image memory, AD...behavior detection area, AL...lost item detection area, AO...item detection area, AR...reference area, Ka, Kb...'Confirm' key, Ma1...warning message, Ma2, Mb2...warning message, Mb1...report message, Sa1...warning screen, Sa2, Sb2...warning screen, Sb1...report screen

Claims

1. a storage means for storing a reference image of a reference area including at least the first and second product placement tables of the payment device when the payment device is in an idle state, the reference area including the first and second product placement tables and the first product placement table is provided with a first product placement table on which a customer places purchased products before registration and a second product placement table on which the customer places purchased products after registration; a detection means for detecting that the customer has left the payment device after completing the payment at the payment device; a detection means for detecting lost items left on the first and second product placement tables of the payment device by comparing an image of the reference area taken at the time when the detection means detects that the customer has left the payment device with the reference image as a comparison image; and a notification means for notifying the user when the lost item left at the settlement device is found; A monitoring device comprising:

2. The finding means determines, based on the comparison image, whether the lost item is a shopping basket that was left empty to contain the purchased items before the registration; 2. The monitoring device according to claim 1, wherein said notification means issues a notification when said lost item is an empty shopping basket.

3. 3. The monitoring device according to claim 2, further comprising prohibiting means for prohibiting the settlement device from registering any purchased items by a next customer when the lost item is an item other than the empty shopping basket.

4. 2. The monitoring device of claim 1, wherein the detection means detects items left in the part of the payment device that excludes the second product tray but includes at least the first product tray by comparing an image of the reference area taken at the time when the customer has completed registration of the purchased items at the payment device and performed a checkout start operation to instruct the customer to checkout, as the comparison image, in a comparison area of ​​the payment device that includes at least the first product tray.

5. The discovery means determining whether the item is an empty shopping basket that was left behind to contain the purchased items before the registration based on the comparison image; The notification means If the item is an empty shopping basket, no notification is made.

5. The monitoring device according to claim 4, wherein if the item is an item other than an empty shopping basket, an alarm is issued to call the customer's attention.

6. A computer that communicates with the payment device, a storage means for storing a reference image of a reference area including at least the first and second product placement tables of the settlement device when the settlement device is in an idle state, the reference area including the first and second product placement tables on which a customer places purchased products before registration and a second product placement table on which the customer places purchased products after registration; a detection means for detecting that the customer has left the settlement device after completing the settlement at the settlement device; a discovery means for discovering lost items left on the first and second product placement tables of the payment device by comparing an image of the reference area taken at the time when the detection means detects that the customer has left the payment device with the reference image as a comparison image; A program for functioning as a notification means for notifying a user when the lost item left at the settlement device is found.

Images