Monitoring device and its program, accounting system

Abstract

To provide a monitoring device that can monitor the occurrence of lost and found items in a full-self-supporting settlement device.SOLUTION: A monitoring device includes acquisition means, detection means, timing means, and reporting means. The acquisition means acquires an image of a photographed area including at least a placing unit of a settlement device having the placing unit on which a purchased article is placed. The detection means detects the article placed on the placing unit based on the image acquired by the acquisition means. The timing means times a time during which the article detected by the detection means is placed at the same location. When the time timed by the timing means exceeds a threshold value, the reporting means informs a user of the fact that there is a lost and found item.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a monitoring device, a program therefor, and an accounting system including the monitoring device. [Background technology]

[0002] In recent years, self-service point-of-sale (POS) terminals have been gaining attention in supermarkets and other retail stores from the perspectives of reducing labor costs and preventing the spread of infectious diseases. Self-service POS terminals are fully self-service payment devices that allow customers to perform all operations themselves, from registering purchased items to paying. Technologies have been developed for such fully self-service payment devices that can prevent fraudulent behavior by customers.

[0003] On the other hand, with fully self-service payment devices, new problems have arisen because customers operate the payment device themselves. One of these is the problem of lost property. When customers operate the payment device, they sometimes temporarily place personal belongings (umbrellas, bags, wallets, etc.) or registered products in any position on the payment device and then forget them. When such lost items are found on the payment device, an employee (attendant) must quickly retrieve them, but they often go unnoticed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 7-141569 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments of the present invention is to provide a monitoring device that can monitor the occurrence of lost items at a fully self-service payment device, and an accounting system that includes this monitoring device. [Means for solving the problem]

[0006] In one embodiment, the monitoring device includes an acquisition means, a detection means, a timing means, and a notification means. The acquisition means acquires an image of a photographed area of ​​a payment device having a placement section on which purchased items are placed, the placement section including at least the placement section. The detection means detects items placed on the placement section based on the image acquired by the acquisition means. The timing means measures the time that the items detected by the detection means are placed in the same location. The notification means notifies the user that a lost item has been found when the time measured by the timing means exceeds a threshold value. The monitoring device further includes a receiving means for receiving from the payment device a payment completion signal notifying completion of payment for the purchased items, and a payment start signal notifying start of payment for the purchased items from the payment device. The detecting means starts an operation of detecting the items using the payment completion signal as a trigger, and stops the detection operation if the items have not been detected at the time of receiving the payment start signal. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a transaction system according to one embodiment. [Figure 2] FIG. 2 is a schematic diagram showing the external configuration of the self-service POS terminal. [Figure 3] FIG. 3 is a block diagram showing the main circuit configuration of the monitoring device. [Figure 4] FIG. 4 is a schematic diagram showing the data structures of the first and second tables. [Figure 5] FIG. 5 is a schematic diagram showing an example of an image captured by a camera. [Figure 6] FIG. 6 is a schematic diagram showing the configuration of means required to realize the fraudulent behavior monitoring function. [Figure 7] FIG. 7 is a schematic diagram showing the configuration of means required to realize the lost-item monitoring function. [Figure 8] FIG. 8 is a flowchart showing the main steps of information processing executed by the processor in accordance with the fraudulent behavior monitoring program. [Figure 9] FIG. 9 is a flowchart showing the main steps of the information processing executed by the processor in accordance with the lost-item monitoring program. [Figure 10] FIG. 10 is a flowchart showing the main steps of the information processing executed by the processor in accordance with the lost-item monitoring program. [Figure 11] FIG. 11 is a flowchart showing the main steps of the information processing executed by the processor in accordance with the lost-item monitoring program. [Figure 12] FIG. 12 is a schematic diagram showing an example of a screen displayed on a display device of the attendant terminal. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of a checkout system including a monitoring device that can monitor the occurrence of lost items at a fully self-service payment device will be described with reference to the drawings.

[0009] [Explanation of accounting system] Figure 1 is a schematic diagram showing the configuration of a checkout system 100 according to this embodiment. The checkout system 100 includes multiple self-service POS terminals 10, an attendant terminal 20, and a store server 30. The checkout system 100 connects each self-service POS terminal 10, the attendant terminal 20, and the store server 30 via a communications network 40 such as a local area network (LAN).

[0010] The multiple self-service POS terminals 10 are installed at the checkout area of ​​the store, and are operated by customers who have finished shopping. Customers who place their purchases in a shopping basket or shopping cart in a sales area where products are displayed and proceed to the checkout area operate the self-service POS terminal 10 to pay for the purchased products, that is, to settle the transaction from product registration to payment, all by themselves. The self-service POS terminal 10 is an example of a fully self-service payment device. The payment device can also be referred to as a payment terminal, a payment device, an accounting terminal, a product registration device, a product registration terminal, etc.

[0011] The attendant terminal 20 is installed at a monitoring station near the exit of the checkout area. The attendant terminal 20 displays a list of purchased items registered for each self-service POS terminal 10 on a display device. The attendant terminal 20 also reports error information if an error occurs at the self-service POS terminal 10. An employee at the monitoring station, i.e., an attendant, monitors the checkout area based on the information displayed and reported on the attendant terminal 20. While FIG. 1 shows one attendant terminal 20, multiple attendant terminals 20 may be connected to the communication network 40. At least one of the attendant terminals 20 may be installed in a location other than the monitoring station, such as a back office on another floor of the same store or at a help desk. Even attendant terminals 20 located far from the self-service POS terminal 10 can report error information from the self-service POS terminal 10 via the communication network 40. Therefore, even employees who cannot directly check the self-service POS terminal 10 can monitor the self-service POS terminal 10.

[0012] The store server 30 provides various services related to the checkout system 100 to each self-service POS terminal 10 and attendant terminal 20 connected via a communication network 40. For example, the store server 30 manages a product database. The product database is a collection of product records created for each product sold in the store. The product records include data such as product codes, product names, and prices. The product codes are unique codes assigned to each product to identify each product. Each product is assigned a barcode that represents the product code of that product. The product names and prices are the unique name of the product identified by the product code and the sales price per unit. When a product code is entered into the self-service POS terminal 10, the store server 30 searches the product database and provides data such as the product name and price of the product record that includes that product code to the self-service POS terminal 10.

[0013] Each self-service POS terminal 10 includes a monitoring device 50. The monitoring device 50 may be located either externally or internally of the self-service POS terminal 10. Each monitoring device 50 is connected to the attendant terminal 20 via a communication network 40. Each monitoring device 50 may also be connected to the attendant terminal 20 without via the communication network 40.

[0014] The monitoring device 50 has the function of monitoring fraudulent behavior of the customer operating the self-service POS terminal 10, i.e., the payment person, and the occurrence of forgotten or lost items. The monitoring device 50 is connected to a camera 60 installed near the self-service POS terminal 10. The monitoring device 50 monitors fraudulent behavior of the payment person at the self-service POS terminal 10 and the occurrence of lost items based on information such as images captured by the camera 60. If fraudulent behavior of the payment person is detected, the monitoring device 50 issues a warning. An example of fraudulent behavior is bagging purchased items without registering them in the self-service POS terminal 10. An example of fraudulent behavior is leaving the checkout area without paying for the purchased items. The warning is issued to both or either the self-service POS terminal 10 and the attendant terminal 20. If a lost item is detected, the monitoring device 50 notifies the attendant terminal that a lost item has been found. The notification is sent to the attendant terminal. The notification may be sent to both the self POS terminal 10 and the attendant terminal 20.

[0015] [Self-POS terminal explanation] 2 is a schematic diagram showing the external configuration of the self-service POS terminal 10. The self-service POS terminal 10 includes a main body 101, and a first product placement table 102 and a second product placement table 103, which are arranged on the left and right sides of the main body 101. The first product placement table 102, which is arranged on the right side as viewed from the main body 101, is a table on which a customer who is making a payment places purchased products before registration. Purchased products before registration are usually placed in a shopping basket and placed on the first product placement table 102. Purchased products before registration may also be placed in a shopping cart and placed in front of or above the first product placement table 102.

[0016] The second product placement table 103, located on the left side of the main body 101, is a table on which the payment person places purchased items after registering them. After registration, the purchased items are placed in a container placed on the second product placement table 103, such as a shopping basket, a plastic bag, or a personal bag. After registration, the purchased items may also be placed directly on the second product placement table 103. In FIG. 2, the second product placement table 103 has a temporary placement table 104 attached to its upper part via a support pillar. The temporary placement table 104 is, for example, a table on which items are temporarily placed before being placed in a plastic bag or the like. The temporary placement table 104 has two holding arms 105 spaced apart on the left and right. Using these holding arms 105, the payment person can place an open plastic bag, personal bag, or the like on the second product placement table 103.

[0017] In this way, the first product placement table 102 and the second product placement table 103 function as placement sections for placing purchased products. Note that the mounting positions of the first product placement table 102 and the second product placement table 103 may be reversed.

[0018] The main body 101 includes a touch panel 11 as an input / output device, a card reader 12, a printer 13, a scanner 14, a hand scanner 15, a cash processing machine 16, a patrol lamp 17, a light emitting unit 18, a speaker 19, and the like.

[0019] The touch panel 11 includes a display as a display device and a touch sensor as an input device. The display displays various screen information such as a payment start screen, a product registration screen, a payment method selection screen, and a payment completion screen. The payment start screen is a screen that accepts the start of a transaction payment. A start key is provided on the payment start screen to command the start of a transaction payment. A payer who starts registering purchased products touches the start key. When the touch sensor detects that the start key has been touched, the controller of the self-service POS terminal 10 switches the display screen from the payment start screen to the product registration screen.

[0020] The product registration screen displays detailed information such as the product name, price, and total amount of the registered purchased products. A payment key is provided on the product registration screen to instruct the user to proceed to payment for the registered purchased products. After registering the purchased products, the user touches the payment key. When the touch sensor detects that the payment key has been touched, the controller of the self-service POS terminal 10 switches the display screen from the product registration screen to a payment method selection screen.

[0021] The payment method selection screen is a screen that accepts the selection input of a payment method, such as cash payment, credit card payment, electronic money payment, or code payment. The payment method selection screen has arranged thereon keys such as a cash key for issuing a cash payment command, a credit key for issuing a credit card payment command, an electronic money key for issuing an electronic money payment command, and a code payment key for issuing a code payment command. The person making the payment touches the key for the desired payment method. For example, when a touch sensor detects that the cash key has been touched, the controller of the self-service POS terminal 10 executes a cash payment process and switches the display screen from the payment method selection screen to a cash payment completion screen. When a touch sensor detects that the credit key has been touched, the controller of the self-service POS terminal 10 executes a credit card payment process and switches the display screen from the payment method selection screen to a credit card payment completion screen. When a touch sensor detects that the electronic money key has been touched, the controller of the self-service POS terminal 10 executes an electronic money card payment process and switches the display screen from the payment method selection screen to an electronic money card payment completion screen. When the touch sensor detects that the code payment key has been touched, the controller of the self-service POS terminal 10 executes the payment process using the code for code payment and switches the display screen from the payment method selection screen to the code payment payment completion screen.

[0022] For example, a cash payment completion screen is a screen that displays the total amount of the transaction, the amount of cash deposited, the amount of change, etc. For example, a cash payment process is a process in which the total amount of the transaction is subtracted from the amount deposited to calculate the change amount, and a receipt is issued to complete the transaction. Since this type of cash payment process is well known, a detailed description will be omitted. Furthermore, since payment processes using credit cards, electronic money cards, or code payments are also well known, a description will be omitted here.

[0023] The card reader 12 is a device that reads data recorded on a card medium such as a credit card, electronic money card, or point card. If the card medium is a magnetic card, the card reader 12 is a magnetic card reader. If the card medium is an IC card, the card reader 12 is an IC card reader. The self-service POS terminal 10 may be equipped with either a magnetic card reader or an IC card reader as the card reader 12, or may be equipped with both. The card reader 12 may also be a card reader / writer that has a function for writing data to a card.

[0024] The printer 13 is a device that prints data related to purchase receipts, credit card slips, etc. onto receipt paper. The receipt paper on which various data has been printed by the printer 13 is cut by a cutter and issued from the receipt issuing slot. The printer 13 may be implemented as, for example, a thermal printer or a dot-matrix printer.

[0025] The scanner 14 and the hand scanner 15 are used to read information identifying a product. In this embodiment, the scanner 14 and the hand scanner 15 read barcodes attached to products as information identifying the products. The scanner 14 optically reads the barcode of a product held up against a reading window (glass window) so that the barcode on the product faces the scanner 14. The hand scanner 15 is handheld and operated by the person making the payment, and optically reads the barcode when brought close to the barcode on the product. The scanner 14 and the hand scanner 15 may be used to read information identifying a product, and may also be used to read information other than a barcode, such as a two-dimensional code.

[0026] The cash processing machine 16 processes cash. The cash processing machine 16 has a banknote input port 161, a banknote discharge port 162, a coin input port 163, and a coin discharge port 164. The cash processing machine 16 processes banknotes inserted into the banknote input port 161. The cash processing machine 16 discharges banknotes as change from the banknote discharge port 162. The cash processing machine 16 accepts and processes coins inserted into the coin input port 163. The cash processing machine 16 discharges coins as change from the coin discharge port 164.

[0027] The patrol lamp 17 is attached to the tip of a vertically extending pole so that it can be seen from a distance. The patrol lamp 17 includes a light-emitting member and a cylinder. The light-emitting member is attached inside the cylinder so that it can rotate coaxially with the cylinder. When the patrol lamp 17 is activated, it can rotate while emitting light from the light-emitting member. The patrol lamp 17 is used to display an alert. For example, the patrol lamp 17 lights up the light-emitting member when an item is registered, allowing the person making the payment or an employee at the attendant terminal 20 to visually confirm whether the item has been scanned properly.

[0028] The light-emitting unit 18 is disposed on the lower part of the patrol lamp 17 on the pole. Like the patrol lamp 17, the light-emitting unit 18 also includes a light-emitting member and a cylinder. The light-emitting member can selectively emit light in multiple colors. Alternatively, the light-emitting unit 18 is provided with multiple light-emitting members of different light colors. Like the patrol lamp 17, the light-emitting unit 18 is also intended to display alerts. The light-emitting unit 18 can display alerts such as an error, the detection of fraudulent behavior by the payment person, etc., by the light-emitting color of the light-emitting member or by the light-emitting color and its flashing state.

[0029] The speaker 19 outputs a warning sound as an alert or a voice message or the like.

[0030] A camera 60 serving as an imaging device is attached to the upper part of the main body 101. The camera 60 photographs the self-service POS terminal 10 and the person operating it. As shown in FIG. 2 , the display of the touch panel 11, the card insertion slot of the card reader 12, the receipt issuing slot of the printer 13, the reading window of the scanner 14, and the bill insertion slot 161, bill ejection slot 162, coin insertion slot 163, and coin ejection slot 164 of the cash processing machine 16 are arranged on one surface (hereinafter referred to as the front) of the main body 101. Therefore, the person operating the self-service POS terminal 10 stands facing the front of the main body 101. The camera 60 photographs the self-service POS terminal 10 and the person standing in front of it from above. Therefore, the head, face, both shoulders, both arms, chest, abdomen, etc. of the person operating the self-service POS terminal 10 are photographed by the camera 60. The camera 60 also captures an image of the top surface of the self-service POS terminal 10, including the first product placement table 102 and the second product placement table 103, which are arranged on the left and right sides of the main body 101.

[0031] [Configuration of monitoring device] 3 is a block diagram showing the main circuit configuration of the monitoring device 50. The monitoring device 50 includes a processor 51, a main memory 52, an auxiliary storage device 53, a timer 54, a camera interface 55, a terminal interface 56, a network interface 57, and a system transmission path 58. The system transmission path 58 includes an address bus, a data bus, a control signal line, etc. The system transmission path 58 connects the processor 51 to each of the other components and transmits data signals exchanged between them.

[0032] The monitoring device 50 is configured as a computer by connecting a processor 51, a main memory 52, an auxiliary storage device 53, a timer 54, a camera interface 55, a terminal interface 56, and a network interface 57 via a system transmission line 58.

[0033] The processor 51 corresponds to the central part of the computer. The processor 51 controls each part to realize various functions of the monitoring device 50 in accordance with an operating system or an application program. The processor 51 is, for example, a CPU (Central Processing Unit). It is desirable that the processor 51 be a multi-core or multi-threaded processor capable of executing multiple processes in parallel.

[0034] The main memory 52 corresponds to the main storage portion of the computer. The main memory 52 includes a nonvolatile memory area and a volatile memory area. The main memory 52 stores an operating system or application programs in the nonvolatile memory area. The main memory 52 may store data required for the processor 51 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 52 uses the volatile memory area as a work area where data is rewritten by the processor 51 as appropriate. The nonvolatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0035] The auxiliary storage device 53 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disk Drive), or an SSD (Solid State Drive) can be the auxiliary storage device 53. The auxiliary storage device 53 stores data used by the processor 51 when performing various processes, data created by the processes in the processor 51, etc. The auxiliary storage device 53 may also store the application programs described above.

[0036] The timer 54 is a circuit that periodically outputs a timer interrupt signal. The timer interrupt signal is provided to the processor 51 via a system transmission line 58. In this embodiment, the timer 54 outputs the timer interrupt signal at one-second intervals.

[0037] The camera interface 55 connects to the camera 60. The camera interface 55 outputs a control signal to the camera 60 to control the driving of the camera 60. The camera 60 performs a photographing operation in accordance with this control signal. The camera interface 55 takes in image data photographed by the camera 60.

[0038] The terminal interface 56 connects to the self-service POS terminal 10. The terminal interface 56 communicates data with the self-service POS terminal 10. The terminal interface 56 receives operation input data and the like input by the payer to the self-service POS terminal 10. The operation input data includes operation input data related to the start of payment, operation input data related to product registration, operation input data related to payment declaration, and operation input data related to payment completion. The operation input data related to the start of payment is data output in response to the payer pressing the start key on the payment start screen. The operation input data related to product registration is data output in response to the payer holding the product barcode over the reading window of the scanner 14 or holding the reading unit of the hand scanner 15 over the barcode. The data includes product record data including the product code represented by the barcode scanned by the scanner 14 or the hand scanner 15. The operation input data related to payment declaration is data output in response to the payer pressing the payment key on the product registration screen. The operation input data related to the completion of payment is data that is output after the payment process using the payment method selected from the payment method selection screen has been completed and the payment completion screen has been displayed. The operation input data related to the completion of payment is output after a sufficient amount of time has passed since the customer completed the payment and left in front of the self-service POS terminal 10. The self-service POS terminal 10 may be provided with a human presence sensor that outputs a detection signal while the person making the payment is within the detection range on the front side of the self-service POS terminal 10, and the operation input data related to the completion of payment may be output when the detection signal is discontinued.

[0039] The network interface 57 connects to the attendant terminal 20 via the communication network 40. The network interface 57 performs data communication with the attendant terminal 20.

[0040] In the monitoring device 50 configured as described above, part of the volatile memory area in the main memory 52 is used as an area for an image memory 521, a first table 522, and a second table 523. The image memory 521 is an area for storing frame-by-frame image data captured via the camera interface 55 in chronological order. The first table 522 and the second table 523 are areas for storing data relating to items detected from images captured by the camera 60.

[0041] 4 is a schematic diagram showing the data structure of the first table 522 and the second table 523. Both the first table 522 and the second table 523 have a column for a record number, a column for an item ID, a column for location information, and a column for the time the item has been left. That is, both the first table 522 and the second table 523 are areas capable of storing multiple records, each of which includes a record number, an item ID, location information, and the time the item has been left. In the record number column, a series of numbers starting from "1" are written as the record number. The item ID, location information, and time the item has been left written in the other columns will be made clear in the explanation of the lost property monitoring function 512, which will be described later.

[0042] The monitoring device 50 also stores a fraudulent behavior monitoring program and a lost property monitoring program so that the processor 51 has a fraudulent behavior monitoring function 511 and a lost property monitoring function 512. The fraudulent behavior monitoring program and the lost property monitoring program are types of application programs stored in the main memory 52 or the auxiliary storage device 53. There are no particular limitations on the method for installing the fraudulent behavior monitoring program and the lost property monitoring program into the main memory 52 or the auxiliary storage device 53. The fraudulent behavior monitoring program and the lost property monitoring program can be recorded on a removable storage medium, or can be distributed via communication over a network and installed in the main memory 52 or the auxiliary storage device 53. The storage medium can be in any form, such as a CD-ROM or memory card, as long as it can store the programs and is readable by the device.

[0043] [Functional description of monitoring device] 6 is a schematic diagram showing the configuration of means required to realize the fraudulent behavior monitoring function 511. As shown in the figure, the fraudulent behavior monitoring function 511 is realized by an acquisition means 71, a behavior detection means 72, a determination means 73, and a warning means 74. The acquisition means 71 is a means for sequentially acquiring images in frame units stored in chronological order in the image memory 521. The behavior detection means 72 is a means for detecting the behavior of the payer from the images acquired by the acquisition means 71. The determination means 73 is a means for determining whether the behavior detected by the behavior detection means 72 is fraudulent or not. The warning means 74 is a means for issuing a warning when the determination means 73 determines that the behavior is fraudulent.

[0044] 7 is a schematic diagram showing the configuration of means required to realize the lost-item monitoring function 512. As shown in the figure, the lost-item monitoring function 512 is realized by a receiving means 81, an acquiring means 82, a detecting means 83, a timing means 84, and a notifying means 85. The receiving means 81 is a means for receiving a payment completion signal from the self-service POS terminal 10 notifying the completion of payment for purchased items. Operation input data related to the completion of payment input from the self-service POS terminal 10 to the terminal interface 56 can serve as the payment completion signal. The receiving means 81 is also a means for receiving a payment start signal from the self-service POS terminal 10 notifying the start of payment for purchased items. Operation input data related to the start of payment input from the self-service POS terminal 10 to the terminal interface 56 can serve as the payment start signal.

[0045] The acquisition means 82 is a means for sequentially acquiring images in frame units stored in chronological order in the image memory 521. In other words, the acquisition means 82 acquires images of a photographed area including at least the placement units (first product placement unit 102, second product placement unit 103) of the settlement device (self-POS terminal 10) having placement units on which purchased products are placed.

[0046] The detection means 83 is a means for detecting an item placed on the placement section based on the image acquired by the acquisition means 82. The item is typically a commercial product. The item may be the payment person's personal belongings (umbrella, bag, wallet, etc.) or a shopping basket. The detection means 83 starts detecting the item when triggered by the payment completion signal. If the detection means 83 has not detected an item when it receives the payment start signal, it stops the detection operation. If the detection means 83 has detected an item when it receives the payment start signal, it continues the detection operation.

[0047] The timing means 84 is a means for timing the time that an item detected by the detection means 83 remains in the same place. As described above, if the detection means 83 has detected an item at the time the payment start signal is received, the detection means 83 continues the detection operation. In this case, the timing means 84 times the time that the item that was detected immediately before the payment start signal is received remains in the same place. The timing means 84 does not time the time that an item that is detected after the payment start signal is received.

[0048] The notification means 85 is means for notifying that a lost item has been found when the time measured by the timing means 84 exceeds a threshold value. The threshold value is any time period sufficient to identify an item placed in the same location as lost. The notification means 85 notifies the attendant that a lost item has been found by outputting a notification signal to the attendant terminal 20. The notification means 85 may also notify the operator of the self-service POS terminal 10, usually a customer, that a lost item has been found by outputting a notification signal to the self-service POS terminal 10.

[0049] Next, the details of the fraudulent behavior monitoring function 511 will be explained using the flowchart of FIG. 8 is a flowchart showing the main steps of the information processing executed by the processor 51 in accordance with the fraudulent behavior monitoring program. Note that the steps and contents of the information processing described below are merely examples. The steps and contents can be modified as appropriate as long as similar operational effects can be achieved.

[0050] In ACT1, processor 51 is in an idle state and waits for a payment start signal. When operation input data related to the start of payment is received from self-service POS terminal 10 via terminal interface 56, processor 51 determines YES in ACT1 and proceeds to ACT2. In ACT2, processor 51 acquires image data from image memory 521. Since image memory 521 stores image data captured by camera 60 in frame-by-frame order in chronological order, processor 51 acquires image data captured after receiving operation input data related to the start of payment.

[0051] In ACT 3, the processor 51, having acquired the image data, detects the behavior of the person making the payment at the self-service POS terminal 10 based on the image data. Then, in ACT 4, the processor 51 determines whether the behavior is fraudulent. If the behavior is not fraudulent, the processor 51 determines NO in ACT 4 and proceeds to ACT 5. In ACT 5, the processor 51 checks whether a payment completion signal has been received.

[0052] If the payment completion signal has not been received, the processor 51 determines NO in ACT 5 and returns to ACT 2. That is, the processor 51 acquires the next image data from the image memory 521 in ACT 2, detects the behavior of the payer in ACT 3, and determines whether the behavior is fraudulent in ACT 4.

[0053] For example, if the payer's behavior is to bag the purchased items without registering them in the self-service POS terminal 10, or to leave the checkout area without paying for the purchased items, the processor 51 determines YES in ACT4 and proceeds to ACT6. In ACT6, the processor 51 issues a warning. The processor 51 outputs a warning signal to the self-service POS terminal 10 via the terminal interface 56. Alternatively, the processor 51 outputs a warning signal to the attendant terminal 20 via the network interface 57. Alternatively, the processor 51 may output a warning signal to both the self-service POS terminal 10 and the attendant terminal 20.

[0054] After issuing the warning, processor 51 proceeds to ACT 5. If the payment completion signal has not been received, processor 51 returns to ACT 2. Processor 51 repeats the process of acquiring the next image data from image memory 521 in ACT 2, detecting the behavior of the payer in ACT 3, and determining whether the behavior is fraudulent in ACT 4.

[0055] When operation input data relating to the completion of payment is received from the self-service POS terminal 10 via the terminal interface 56, the processor 51 determines YES in ACT5. The processor 51 ends the information processing of the procedure shown in the flowchart of FIG. 8 and returns to the idle state.

[0056] Here, the processor 51 configures acquisition means 71 by processing ACT2, configures behavior detection means 72 by processing ACT3, configures determination means 73 by processing ACT4, and configures warning means 74 by processing ACT6. Thus, when fraudulent behavior of a settlement person operating the self-service POS terminal 10 is recognized, a warning signal is sent to the self-service POS terminal 10 or the attendant terminal 20. This warning signal causes the light-emitting unit 18 in the self-service POS terminal 10 to emit light and the speaker 19 to emit a warning sound. The attendant terminal 20 notifies the attendant that fraudulent behavior of the settlement person operating the self-service POS terminal 10 has been recognized. As a result, a warning is issued in real time in response to the settlement person's fraudulent behavior, which can have a deterrent effect on fraud.

[0057] Next, the lost property monitoring function 512 will be described in detail with reference to the flowcharts of FIGS. 9 to 11 and the screen example of FIG. 9 to 11 are flow charts showing the main steps of the information processing executed by the processor 51 in accordance with the lost property monitoring program. FIG. 12 is a schematic diagram showing a screen displayed on the display device of the attendant terminal 20. Note that the steps and contents of the information processing described below are merely examples. The steps and contents can be modified as appropriate as long as similar effects can be achieved. The screen of FIG. 12 is also merely an example. It goes without saying that the arrangement of display elements on the screen, the content of text, etc. can be modified as appropriate.

[0058] The processor 51, which is in an idle state, waits for a payment start signal in ACT 11. When operation input data related to the start of payment is received from the self-service POS terminal 10 via the terminal interface 56, the processor 51 determines YES in ACT 11 and proceeds to ACT 12. In ACT 12, the processor 51 clears the first table 522 and the second table 523.

[0059] Thereafter, the processor 51 waits for a payment completion signal in ACT 13. When operation input data related to the completion of payment is received from the self-service POS terminal 10 via the terminal interface 56, the processor 51 determines YES in ACT 13 and proceeds to ACT 14. In ACT 14, the processor 51 acquires image data from the image memory 521. Since the image memory 521 stores image data in frames captured by the camera 60 in chronological order, the processor 51 acquires the image data captured after receiving the operation input data related to the completion of payment.

[0060] Having acquired the image data, the processor 51, as ACT15, analyzes the image data and searches for items placed on the first product placement table 102 or the second product placement table 103. For example, the processor 51 uses known object recognition technology such as pattern matching or deep learning to recognize the first product placement table 102 and the second product placement table 103 from the image, and further recognizes items such as products, personal belongings, shopping baskets, etc. that are presumed to be placed on the first product placement table 102 or the second product placement table 103.

[0061] The processor 51 determines whether or not an item has been detected in ACT16. If an item has not been detected, the processor 51 determines NO in ACT16 and proceeds to ACT17. The processor 51 checks whether or not a payment start signal has been received in ACT17. If a payment start signal has not been received, the processor 51 determines NO in ACT17 and returns to ACT14. That is, the processor 51 obtains the next image data from the image memory 521 in ACT14, searches for the item in ACT15, and determines whether or not the item has been detected in ACT16.

[0062] In this way, the processor 51, which has received operation input data related to the completion of payment from the self-service POS terminal 10, repeatedly executes the processes of ACT14, ACT15, and ACT16 until it receives operation input data related to the start of payment from the self-service POS terminal 10. Then, when operation input data related to the start of payment is received without detecting any goods, the processor 51 determines YES in ACT17 and returns to ACT12. That is, the processor 51 clears the first table 522 and the second table 523 and waits for a payment completion signal.

[0063] On the other hand, if an item placed on the first product placement table 102 or the second product placement table 103 is detected from the image captured by the camera 60, the processor 51 determines YES in ACT 16 and proceeds to ACT 21 in FIG. 10. There are two events in which an item is detected. One is when the item has already been placed on the first product placement table 102 or the second product placement table 103 at the time when the operation input data related to the completion of the payment is received. This applies when the person making the payment leaves an item or personal belongings on the first product placement table 102 or the second product placement table 103, or when the person making the payment does not remove their empty shopping basket from the first product placement table 102. The other is when an item is placed on the first product placement table 102 or the second product placement table 103 after the operation input data related to the completion of the payment is received. For example, this applies to a case where a child leaves a toy or the like on the first product placement stand 102 or the second product placement stand 103. In either case, the processor 51 proceeds to ACT21.

[0064] Processor 51 stores the number of items N in ACT21. The number of items N is the total number of items detected from the image as being placed on the first item placement table 102 or the second item placement table 103. Processor 51 assigns a unique item ID to each of the N items in ACT22. Processor 51 also detects the center position coordinates for each of the N items in ACT23. The processing of ACT22 and ACT23 will now be described in detail with reference to FIG. 5.

[0065] FIG. 5 is a schematic diagram showing an example of an image AR captured by the camera 60. All display elements unnecessary for explanation have been omitted. The image AR shows a first product placement table 102 and a second product placement table 103. A first item A is placed on the first product placement table 102. A second item B is placed on the second product placement table 103. The first item A is an empty shopping basket. The second item B is a registered product. In other words, it is assumed that the person making the payment does not clear away the shopping basket placed on the first product placement table 102, and also leaves one of the purchased products on the second product placement table 103.

[0066] 5 is acquired, the processor 51 assigns an item IDa to the first item A and an item IDb to the second item B in ACT22. In addition, the processor 51 detects the coordinates (Xa, Ya) of the center position of the first item A and the coordinates (Xb, Yb) of the center position of the second item B in a two-dimensional plane coordinate system with the lower left corner of the image AR as the origin (Xo, Yo), for example, in ACT23.

[0067] After completing the processing of ACT22 and ACT23, the processor 51 proceeds to ACT24. In ACT24, the processor 51 creates a first table 522. That is, the processor 51 writes the item IDs and location coordinates of the detected items (item A, item B) in the item ID column and location coordinate column of the first table 522 in ascending order of record number. The processor 51 also writes 0 seconds as the leaving time Tp in each of the leaving time columns.

[0068] After creating the first table 522, the processor 51 proceeds to ACT25. In ACT25, the processor 51 resets the addition counter n to "0". The addition counter n is built into the processor 51. Thereafter, in ACT26, the processor 51 counts up the addition counter n by "1". Then, in ACT27, the processor 51 retrieves the record with record number n from the first table 522. The "n" in the record number n is the count value of the addition counter n. The record includes an item ID, location information, and leaving time Tp. Hereinafter, the record retrieved in ACT27 will be referred to as the first record Rx.

[0069] After acquiring the first record Rx, the processor 51 searches the second table 523 in ACT28. In ACT29, the processor 51 checks whether a record including position coordinates that approximately match those of the first record Rx is stored in the second table 523. Here, an approximate match refers to, for example, a case where the length of a line segment connecting a first point identified by one position coordinate and a second point identified by the other position coordinate is equal to or less than a predetermined distance. The predetermined distance is, for example, an arbitrary value equal to or less than 1 cm. Hereinafter, a record including position coordinates that approximately match those of the first record Rx will be referred to as a second record Ry.

[0070] If the second record Ry is not stored in the second table 523, the processor 51 determines NO in ACT 29 and proceeds to ACT 30. In ACT 30, the processor 51 determines whether the counter n has exceeded the number of items N. If the counter n has not exceeded the number of items N, the processor 51 determines NO in ACT 30 and returns to ACT 26.

[0071] That is, the processor 51 counts up the up counter n by "1" in ACT26, obtains the first record Rx from the first table 522 in ACT27, and searches the second table 523 in ACT28.

[0072] Initially, the second table 523 is cleared. Therefore, in ACT 29, there is no record including position coordinates that substantially match the position coordinates of the first record Rx, i.e., the second record Ry, stored in the second table 523. Therefore, the processor 51 repeats the processing of ACT 26 to ACT 30 until the counter n exceeds the number of items N.

[0073] When the counter n exceeds the number of items N, the processor 51 determines YES in ACT 30 and proceeds to ACT 33. In ACT 33, the processor 51 checks whether a record in which the standing time Tp exceeds the threshold value Tmax is stored in the first table 522. The threshold value Tmax is, for example, an arbitrary value equal to or greater than 5.

[0074] If there is not a single record stored in the first table 522 whose idle time Tp exceeds the threshold value Tmax, the processor 51 determines NO in ACT 33 and proceeds to ACT 34. In ACT 34, the processor 51 copies all records in the first table 522 to the second table 523. Once copying is complete, the processor 51 clears the first table 522 in ACT 35. Thereafter, the processor 51 proceeds to ACT 41 in FIG. 11.

[0075] In ACT41, the processor 51 checks whether or not a payment start signal has been received. If a payment start signal has not been received, the processor 51 determines NO in ACT41 and proceeds to ACT42. In ACT42, the processor 51 checks whether or not a timer interrupt signal has been received. If a timer interrupt signal has not been received, the processor 51 determines NO in ACT42 and returns to ACT41. In this way, the processor 51 waits to receive a payment start signal or a timer interrupt signal in ACT41 and ACT42.

[0076] When a timer interrupt signal is received in the standby state in ACT41 and ACT42, the processor 51 determines YES in ACT42 and proceeds to ACT43. The processor 51 executes the same processing as ACT14 to ACT16 shown in FIG. 9 in ACT43 to ACT45. That is, the processor 51 acquires the next image data from the image memory 521 in ACT43. The processor 51 analyzes the image data in ACT44 and searches for an item placed on the first product placement table 102 or the second product placement table 103. The processor 51 determines whether or not an item has been detected in ACT45.

[0077] If no article is detected, for example, if article A and article B shown in Fig. 5 are removed from the first product placement table 102 and the second product placement table 103, the processor 51 determines NO in ACT 45. The processor 51 ends the information processing of the procedures shown in the flowcharts of Figs. 9 to 11 and returns to the idle state.

[0078] On the other hand, if an item has been detected, for example, if item A and item B are still placed on the first item placement table 102 and the second item placement table 103, the processor 51 determines YES in ACT 45. The processor 51 returns to ACT 21 in FIG. 10. The processor 51 executes the processing from ACT 21 onwards in the same manner as described above. That is, the processor 51 stores the number of items N in ACT 21. The processor 51 also assigns unique item IDs to each of the N items in ACT 22, and detects the central position coordinates in ACT 23. The processor 51 then creates a first table 522 in ACT 24.

[0079] The processor 51 resets the addition counter n to "0" in ACT 25, and then counts up the addition counter n by "1" in ACT 26. Then, the processor 51 obtains the first record Rx from the first table 522 in ACT 27, and searches the second table 523 in ACT 28.

[0080] This time, the records in the first table 522 are copied to the second table 523 in the processing of ACT34, so if an item is detected at the same position in the image acquired last time and the image acquired this time, a record containing position coordinates that approximately match the position coordinates of the first record Rx, i.e., the second record Ry, will be stored in the second table 523.

[0081] If the second record Ry is stored in the second table 523, the processor 51 determines YES in ACT29 and proceeds to ACT31. The processor 51 detects the leaving time Tq of the second record Ry in ACT31. Then, the processor 51 adds "1" to the leaving time Tq of the second record Ry in ACT32, and adds the result to the leaving time Tq of the first record Rx. As a result, for example, if the leaving time Tp and the leaving time Tq are both "0", the leaving time Tp becomes "1". The processor 51 then proceeds to ACT30 and determines whether the counter n has exceeded the number of items N.

[0082] On the other hand, if the second record Ry is not stored in the second table 523, the processor 51 determines NO in ACT29, proceeds to ACT30, and determines whether the counter n has exceeded the number of items N.

[0083] If the counter n does not exceed the number of items N, the processor 51 returns to ACT 26. The processor 51 repeatedly executes the processes of ACT 26 to ACT 32 until the counter n exceeds the number of items N.

[0084] Thus, the leaving time Tp of the first record Rx for which the second record Ry, whose position coordinates are approximately the same, is stored in the second table 523, is incremented by "1." The leaving time Tp of the first record Rx for which the second record Ry, whose position coordinates are approximately the same, is not stored in the second table 523 remains at "0."

[0085] If the counter n exceeds the number of items N, the processor 51 checks in ACT33 whether the first table 522 stores any records whose abandoned time Tp exceeds the threshold value Tmax. If the first table 522 does not store any records whose abandoned time Tp exceeds the threshold value Tmax, the processor 51 copies all records in the first table 522 to the second table 523 in ACT34. The processor 51 also clears the first table 522 in ACT35. Thereafter, the processor 51 enters a standby state in ACT41 and ACT42, that is, waits for the receipt of a payment start signal or a timer interrupt signal. Then, if the timer interrupt signal is received again, the processor 51 executes the processes in ACT43 to ACT45, and if an item is detected, executes the processes in ACT21 to ACT35 in the same manner as described above.

[0086] In this way, if the processes of ACT21 to ACT35 and ACT41 to ACT45 are repeatedly executed without inputting a settlement start signal, there is a possibility that a record whose idle time Tp exceeds the threshold value Tmax will be detected from the first table 522.

[0087] When a record in which the abandoned time Tp exceeds the threshold value Tmax is detected from the first table 522, the processor 51 determines YES in ACT 33 and proceeds to ACT 36. The processor 51 performs an annunciation operation in ACT 36. For example, the processor 51 outputs an annunciation signal to the attendant terminal 20 via the network interface 57.

[0088] As shown in FIG. 12, the attendant terminal 20 that has received the notification signal displays an image 91 to notify the attendant that there is a lost item on a list of purchased items registered in the self-service POS terminal 10 to which the monitoring device 50 that output the notification signal is connected. After checking image 91, the attendant goes to the checkout area and collects the abandoned items A and B. The attendant then returns to the monitoring area and presses the [Confirm] button. When the [Confirm] button is pressed, a confirmation signal is output from the attendant terminal 20 to the monitoring device 50. The attendant may also collect the items after pressing the [Confirm] button.

[0089] The processor 51 of the monitoring device 50 that output the notification signal in Act 36 waits for a confirmation signal in Act 37. When the confirmation signal is received from the attendant terminal 20, the processor 51 determines YES in Act 37. The processor 51 ends the information processing of the procedures shown in the flowcharts of Figures 9 to 11 and returns to the idle state.

[0090] Here, the processor 51 constitutes a receiving means 81 by processing ACT13, constitutes an acquiring means 82 by processing ACT14 and ACT43, constitutes a detecting means 83 by processing ACT15 and ACT16 and ACT44 and ACT45, constitutes a timing means 84 by processing ACT27 to ACT32, and constitutes an alarm means 85 by processing ACT33 and ACT36.

[0091] Thus, if a person who has completed payment at the self-service POS terminal 10 leaves an item on the first product placement table 102 or the second product placement table 103 and leaves the area, the attendant is notified that an item has been lost. Therefore, the attendant can monitor the occurrence of lost items left at the self-service POS terminal 10 at the monitoring station. When the attendant confirms the lost item, he or she can quickly retrieve it, thereby minimizing the situation where the next customer is unable to operate the self-service POS terminal 10 because of the lost item.

[0092] Meanwhile, the self-service POS terminal 10 where the previous customer has completed payment displays the payment start screen regardless of whether the goods are left behind. Therefore, it is possible that the next customer will start payment before the attendant collects the goods. When the next customer touches the start key on the payment start screen, operation input data related to the start of payment, i.e., a payment start signal, is input from the self-service POS terminal 10 to the terminal interface 56. At this time, the processor 51 of the monitoring device 50 is in a standby state for ACT41 and ACT42 in FIG. 11.

[0093] When a payment start signal is received in the standby state of ACT41 and ACT42, processor 51 proceeds from ACT41 to ACT46. In ACT46, processor 51 checks whether a record with a leaving time Tq of "1" or more is stored in second table 523. A record with a leaving time Tq of "1" or more is a record related to an item that was left on first product placement table 102 or second product placement table 103 before the next customer started payment. Hereinafter, a record with a leaving time Tq of "1" or more will be referred to as third record Rz.

[0094] If the third record Rz is not stored in the second table 523, the processor 51 determines NO in ACT46 and returns to ACT12 in Fig. 9. The processor 51 executes the processes from ACT12 onwards in the same manner as described above.

[0095] On the other hand, if the third record Rz is stored in the second table 523, the processor 51 determines YES in ACT 46 and proceeds to ACT 47. The processor 51 acquires the position coordinates (X, Y) of the third record Rz in ACT 47. Incidentally, if multiple third records Rz are stored, the processor 51 acquires the position coordinates (X, Y) of each of them.

[0096] In ACT48, the processor 51 waits for a timer interrupt signal. When the processor 51 receives the timer interrupt signal, it determines YES in ACT48 and proceeds to ACT49. In ACT49 to ACT51, the processor 51 executes the same processing as ACT14 to ACT16 shown in FIG. 9. That is, in ACT49, the processor 51 acquires the next image data from the image memory 521. In ACT50, the processor 51 analyzes the image data and searches for an item placed on the first product placement table 102 or the second product placement table 103. In ACT51, the processor 51 determines whether or not an item has been detected. If an item has not been detected, the processor 51 determines NO in ACT51 and returns to ACT12 in FIG. 9. The processor 51 executes the processing from ACT12 onwards in the same manner as described above.

[0097] If an article is detected, the processor 51 determines YES in ACT 51 and proceeds to ACT 52. In ACT 52, the processor 51 detects the position coordinates of the center of the detected article, similar to the processing in ACT 23 of Fig. 10. In this case, if multiple articles are detected, the processor 51 detects the position coordinates of the center of each of the articles.

[0098] The processor 51 checks whether position coordinates that approximately match the position coordinates (X, Y) of the third record Rz have been detected in the processing of ACT 52 in ACT 53. In this case, as in the processing of ACT 29 in Fig. 10, it is considered that there is an approximate match when the length of the line segment connecting the first point specified by one position coordinate and the second point specified by the other position coordinate is equal to or shorter than a predetermined distance.

[0099] If the processing in ACT 52 does not detect position coordinates that approximately match the position coordinates (X, Y) of the third record Rz, the processor 51 determines NO in ACT 53 and returns to ACT 12 in Fig. 9. The processor 51 executes the processing from ACT 12 onwards in the same manner as described above.

[0100] If position coordinates that approximately match the position coordinates (X, Y) of the third record Rz are detected in the processing of ACT 52, the processor 51 determines YES in ACT 53 and proceeds to ACT 54. In ACT 54, the processor 51 adds "1" to the standing time Tq of all third records Rz for which approximately matching position coordinates have been detected.

[0101] In ACT 55, the processor 51 checks whether the idle time Tq has exceeded the threshold value Tmax. If the idle time Tq has not exceeded the threshold value Tmax, the processor 51 returns to ACT 48. The processor 51 executes the processes from ACT 48 onward in the same manner as described above. That is, when the processor 51 receives a timer interrupt signal, it acquires the next image data from the image memory 521 in ACT 49, analyzes the image data in ACT 50, and searches for an item placed on the first product placement table 102 or the second product placement table 103. If an item is detected as a result, the processor 51 detects the position coordinates of the center of the detected item in ACT 52, and checks in ACT 53 whether position coordinates that approximately match the position coordinates (X, Y) of the third record Rz have been detected. If position coordinates that approximately match the position coordinates (X, Y) of the third record Rz have been detected, the processor 51 adds "1" to the idle time Tq of all third records Rz for which approximately matching position coordinates have been detected in ACT 54.

[0102] Thus, when the leaving time Tq exceeds the threshold value Tmax, the processor 51 determines YES in ACT 55 and proceeds to ACT 56. In ACT 56, the processor 51 performs an announcing operation similar to the processing in ACT 36 of Fig. 9. Then, when a confirmation signal is received from the attendant terminal 20, the processor 51 determines YES in ACT 56. The processor 51 ends the information processing of the procedures shown in the flowcharts of Figs. 9 to 11 and returns to the idle state.

[0103] Here, the processor 51 constitutes a receiving means 81 by processing ACT41, an acquiring means 82 by processing ACT49, a detecting means 83 by processing ACT50 to ACT51, a timing means 84 by processing ACT47, ACT52 to ACT54, and an alarm means 85 by processing ACT55 and ACT56.

[0104] Thus, even if the next customer starts to pay before the attendant collects an item left on the first product placement table 102 or the second product placement table 103, the attendant is notified of the lost item when the time Tq the item has been left on the table exceeds the threshold value Tmax. Therefore, since the attendant collects the abandoned item, there is no problem with the next customer operating the self-service POS terminal 10 to start payment.

[0105] [Variations] In the above embodiment, an example has been given in which each self-service POS terminal 10 is equipped with a monitoring device 50. In another embodiment, one monitoring device 50 can be connected to two or more self-service POS terminals 10 and cameras 60 installed near the self-service POS terminals 10, and can monitor the occurrence of lost items at the two or more self-service POS terminals 10.

[0106] In the above embodiment, the monitoring device 50 has been described as a device having the function of monitoring fraudulent behavior of the payment person and the occurrence of lost items. The monitoring device 50 may also be a device specialized in the function of monitoring the occurrence of lost items. However, by having the function of monitoring fraudulent behavior of the payment person and the occurrence of lost items, the monitoring device 50 uses images captured by one camera 60 to monitor fraudulent behavior during payment and to monitor lost items after payment. This allows for efficient use of resources.

[0107] In the above embodiment, a fraudulent behavior monitoring program is exemplified as an application program for the processor 51 to realize the fraudulent behavior monitoring function 511, and a lost property monitoring program is exemplified as an application program for realizing the lost property monitoring function 512. In this regard, the processor 51 may be configured to realize the fraudulent behavior monitoring function 511 and the lost property monitoring function 512 by a common application program.

[0108] In the above embodiment, the first product placement table 102 and the second product placement table 103 of the self-service POS terminal 10 are considered to be placement units. The placement units are not limited to these. For example, if there is a space in the main body 101 to place an item, that space may also be included in the placement units.

[0109] In the above embodiment, the position coordinates of the center of the article are detected in the processing of ACT23 in Fig. 10 and ACT52 in Fig. 11. However, it is also possible to detect position coordinates other than the center of the article. The point is that it is sufficient to be able to identify whether the article is moving or not.

[0110] In the above embodiment, the threshold value Tmax compared with the abandoned time Tp or the abandoned time Tq is the same in the processing of ACT 33 of FIG. 10 and ACT 55 of FIG. 11. In this regard, for example, the first threshold value Tmax compared with the abandoned time Tq in ACT 55 of FIG. 11 may be set to a value smaller than the second threshold value Tmax compared with the abandoned time Tp in ACT 33 of FIG. 10. When the next customer begins their transaction, an item placed on the first product placement table 102 or the second product placement table 103 is likely to be an abandoned item. Therefore, by setting the first threshold value Tmax to a value smaller than the second threshold value Tmax, the timing at which the item is left as lost can be set earlier. Needless to say, the first threshold value Tmax may also be set to a value greater than the second threshold value Tmax.

[0111] In the above embodiment, an example is given in which the attendant terminal 20 that receives the notification signal displays an image 91 to notify the attendant that a lost item is present in the list of purchased items. In another embodiment, information specifying whether the lost item is placed on the first item placement tray 102 or the second item placement tray 103 may also be displayed. Furthermore, it is possible for the detection means 83 to identify whether the abandoned item is a product or a shopping basket using existing technology. Therefore, the type of abandoned item may also be displayed. Furthermore, since an image of the abandoned item is stored in the image memory 521, that image may also be displayed. By displaying the image, the attendant can more reliably find the lost item.

[0112] Although several embodiments of the checkout system 100 including a monitoring device 50 capable of monitoring lost items at a self-service POS terminal 10 have been described above, such embodiments are not limited to these. These embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are within the scope of the invention and the scope of the inventions and their equivalents as set forth in the claims. The following is a summary of the scope of claims as originally filed in this application. [C1] an acquisition means for acquiring an image of a photographing area including at least a placement unit on which a purchase item is placed, the photographing area including the placement unit; a detection unit that detects an article placed on the placement unit based on the image acquired by the acquisition unit; a timing means for timing the time that the article detected by the detecting means is placed in the same place; a notification means for notifying the user that an item has been lost when the time measured by the timing means exceeds a threshold value; A monitoring device comprising: [C2] a receiving means for receiving a payment completion signal from the payment device, the payment completion signal notifying the completion of payment for the purchased items; Further comprising: The monitoring device according to C1, wherein the detection means starts an operation of detecting the item using the payment completion signal as a trigger. [C3] The receiving means further receives a payment start signal from the payment device notifying the start of payment for the purchased items, The monitoring device according to C2, wherein the detection means stops the detection operation if the item has not been detected at the time the payment start signal is received. [C4] If the detecting means has detected the item at the time when the payment start signal is received, the detecting means continues the detecting operation, The monitoring device described in C3, wherein the timing means times the time that an item that was detected immediately before receiving the payment start signal is placed in the same location, but does not time any item that is detected after receiving the payment start signal. [C5] a payment device that allows customers to pay for purchased items by their own operation; a camera that photographs an imaging area of ​​the settlement device including a placement unit on which the purchased item is placed; a monitoring device that monitors whether or not there is any lost item left in the placement section; Equipped with The monitoring device an acquisition means for acquiring an image captured by the camera; a detection unit that detects an article placed on the placement unit based on the image acquired by the acquisition unit; a timing means for timing the time that the article detected by the detecting means is placed in the same place; a notification means for notifying the user that an item has been lost when the time measured by the timing means exceeds a threshold time; An accounting system that includes: [C6] The monitoring device fraudulent behavior monitoring means for monitoring fraudulent behavior of customers operating the payment device based on images captured by the camera; The accounting system of C5, further comprising: [C7] The monitoring device's computer, an acquisition means for acquiring an image of a photographing area including at least a placement unit on which a purchased commodity is placed, the placement unit being a payment device; a detection unit that detects an article placed on the placement unit based on the image acquired by the acquisition unit; a timing means for timing the time that the article detected by the detecting means is placed in the same place; and a notification means for notifying the user that an item has been lost when the time measured by the timing means exceeds a threshold time; A program to function as a [Explanation of symbols]

[0113] 10...self-service POS terminal, 20...attendant terminal, 30...store server, 40...communication network, 50...monitoring device, 51...processor, 52...main memory, 53...auxiliary storage device, 54...timer, 55...camera interface, 56...terminal interface, 57...network interface, 58...system transmission path, 60...camera, 71...acquisition means, 72...behavior detection means, 73...determination means, 74...warning means, 81...receiving means, 82...acquisition means, 83...detection means, 84...timing means, 85...alarm means, 100...accounting system, 101...main body, 102...first product placement stand, 103...second product placement stand, 511...fraudulent behavior monitoring function, 512...lost property monitoring function, 521...image memory, 522...first table, 523...second table.

Claims

1. an acquisition means for acquiring an image of a photographing area including at least a placement unit on which a purchase item is placed, the photographing area including the placement unit; a detection unit that detects an article placed on the placement unit based on the image acquired by the acquisition unit; a timing means for timing the time that the article detected by the detecting means is placed in the same place; a notification means for notifying the user that an item has been lost when the time measured by the timing means exceeds a threshold value; Equipped with a receiving means for receiving from the payment device a payment completion signal notifying completion of payment for the purchased goods, and further receiving from the payment device a payment start signal notifying start of payment for the purchased goods; Further comprising: The detection means starts detecting the item when triggered by the payment completion signal, and stops the detection operation if the item has not been detected at the time of receiving the payment start signal.

2. If the detecting means has detected the item at the time when the payment start signal is received, the detecting means continues the detecting operation, 2. The monitoring device according to claim 1, wherein the timing means measures the time that an item detected immediately before receiving the payment start signal is placed in the same location, but does not measure the time that an item detected after receiving the payment start signal is placed in the same location.

3. a payment device that allows customers to pay for purchased items by their own operation; a camera that photographs an imaging area of ​​the settlement device including a placement unit on which the purchased item is placed; a monitoring device that monitors whether or not there is any lost item left in the placement section; Equipped with The monitoring device an acquisition means for acquiring an image captured by the camera; a detection unit that detects an article placed on the placement unit based on the image acquired by the acquisition unit; a timing means for timing the time that the article detected by the detecting means is placed in the same place; a notification means for notifying the user that an item has been lost when the time measured by the timing means exceeds a threshold time; Equipped with a receiving means for receiving from the payment device a payment completion signal notifying completion of payment for the purchased goods, and further receiving from the payment device a payment start signal notifying start of payment for the purchased goods; Further comprising: The detection means initiates detection of the item in response to the payment completion signal, and stops detection if the item has not been detected at the time the payment start signal is received.

4. The monitoring device fraudulent behavior monitoring means for monitoring fraudulent behavior of customers operating the payment device based on images captured by the camera; 4. The accounting system of claim 3, further comprising:

5. The monitoring device's computer, an acquisition means for acquiring an image of a photographing area including at least a placement unit on which a purchased commodity is placed, the placement unit being a payment device; a detection unit that detects an article placed on the placement unit based on the image acquired by the acquisition unit; a timing means for timing the time that the article detected by the detecting means is placed in the same place; and a notification means for notifying the user that an item has been lost when the time measured by the timing means exceeds a threshold time; It functions as a receiving means for receiving from the payment device a payment completion signal notifying completion of payment for the purchased goods, and further receiving from the payment device a payment start signal notifying start of payment for the purchased goods; It further functions as A program for causing the detection means to function such that it starts the detection operation of the item when triggered by the payment completion signal, and stops the detection operation if the item has not been detected at the time the payment start signal is received.

Images