Fraud monitoring device, product sales data processing device and program

Abstract

We provide a fraud monitoring device, a product sales data processing device, and a program that can reliably detect fraudulent activity. [Solution] The fraud monitoring device includes: a video information acquisition unit (fraud judgment information acquisition unit) that acquires video footage of a customer grasping a product and moving it from a first area where the product is placed, across a second area facing a scanner on a self-service POS terminal for registering products, to a third area where scanned products are placed; a product code acquisition unit (fraud judgment information acquisition unit) that acquires a product code (identification information) that identifies the product, read by the scanner; a fraud judgment unit that, based on the video, determines that the next product has entered the second area, initiates a fraud judgment process to determine whether fraud has occurred with respect to the product moved immediately before the next product; and a judgment result output unit that, if the fraud judgment unit determines that fraud has occurred, outputs information indicating that fraud has occurred to the self-service POS terminal.

Description

Technical Field

[0004] , , , , , , , , ,

[0005] ,

[0001] Embodiments of the present invention relate to an illegal monitoring device, a product sales data processing device, and a program.

Background Art

[0002] Self-checkouts where customers themselves register and pay for the products they wish to purchase are increasing. To prevent fraud such as unregistered products in such self-checkouts, for example, there is a known monitoring device that monitors the customer's actions by comparing the movement of the products held by the customer with the identification information (e.g., product code) read by the product registration device.

[0003] Since such a monitoring device has a high computational load, it is generally configured on, for example, an edge server separate from the self-checkout. In such a case, since the self-checkout and the monitoring device exchange information through communication, there is a possibility that the timing of detecting customer fraud is delayed compared to the timing when the customer committed the fraud. Therefore, for example, when the information indicating that the product has been scanned reaches the monitoring device late even though the customer has finished scanning the product, the monitoring device may misjudge that there is an unscanned product.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide an illegal monitoring device, a product sales data processing device, and a program that can surely perform illegal determination.

Means for Solving the Problems

[0005] The fraud monitoring device of this embodiment comprises a fraud judgment information acquisition unit, a fraud determination unit, and a determination result notification unit. The fraud judgment information acquisition unit acquires video footage of a customer grasping a product they wish to purchase and moving it from a first area where the product is placed, across a second area facing a scanner provided by a product sales data processing device that registers products, to a third area where scanned products are placed, and also acquires identification information that uniquely identifies the product, as read by the scanner. The fraud determination unit, based on the video footage, determines that the next product has entered the second area and initiates a fraud determination process to determine whether fraud has occurred with respect to the product moved immediately before the next product. The determination result notification unit outputs information indicating that fraud has occurred to the product sales data processing device if the fraud determination unit determines that fraud has occurred.

[0006] Furthermore, the product sales data processing device of the embodiment includes an identification information acquisition unit, a product registration unit, an identification information output unit, a fraud detection result acquisition unit, and a detection result display unit. The identification information acquisition unit acquires identification information that uniquely identifies the product that the customer wishes to purchase. The product registration unit registers the product having the identification information acquired by the identification information acquisition unit. The identification information output unit outputs the product identification information to the fraud monitoring device. The fraud detection result acquisition unit acquires the result of a fraud detection process from the fraud monitoring device when the next product to be registered by the customer moves from the first area on which it is placed to the second area facing the scanner that acquires the product identification information. The detection result display unit displays information indicating that fraud occurred when the previous product was registered, provided that the fraud detection result acquisition unit has acquired information indicating that fraud occurred when the previous product was registered. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a block diagram showing the schematic configuration of the fraud monitoring system according to the embodiment. [Figure 2] Figure 2 is a front view showing an example of a self-service POS terminal included in the fraud monitoring system of the embodiment. [Figure 3]Figure 3 is a hardware block diagram showing an example of the hardware configuration of a self-service POS terminal included in the fraud monitoring system of the embodiment. [Figure 4] Figure 4 is a hardware block diagram showing an example of the hardware configuration of the fraud monitoring device included in the fraud monitoring system of the embodiment. [Figure 5] Figure 5 shows an example of an image captured by the camera. [Figure 6] Figure 6 illustrates an example of how a fraud monitoring device recognizes a customer's scanning activity from images captured by a camera. [Figure 7] Figure 7 is the first diagram showing an example of a time chart for the fraud detection process performed by the fraud monitoring system of the embodiment. [Figure 8] Figure 8 is a second figure showing an example of a time chart for the fraud detection process performed by the fraud monitoring system of the embodiment. [Figure 9] Figure 9 shows an example of a notification screen displayed on a self-service POS terminal when a fraud monitoring device determines that a customer has committed fraudulent activity. [Figure 10] Figure 10 is a functional block diagram showing an example of the functional configuration of the fraud monitoring system of the embodiment. [Figure 11] Figure 11 is a flowchart showing an example of the processing flow performed by a self-service POS terminal equipped with the fraud monitoring system of the embodiment. [Figure 12] Figure 12 is a flowchart showing an example of the processing flow performed by the fraud monitoring device included in the fraud monitoring system of the embodiment. [Modes for carrying out the invention]

[0008] The embodiments will be described in detail below with reference to the drawings. However, the invention is not limited to the embodiments described below.

[0009] The following describes an embodiment of the fraud monitoring system 10 of this disclosure with reference to the drawings.

[0010] (Outline configuration of the fraud monitoring system) The schematic configuration of the fraud monitoring system 10 of the embodiment will be explained using Figure 1. Figure 1 is a block diagram showing the schematic configuration of the fraud monitoring system of the embodiment.

[0011] The fraud monitoring system 10 monitors for fraudulent activity in retail stores and other shops, such as customers failing to scan product identification information (e.g., product code) when registering and paying for purchased items, and instead placing the items in their reusable bags. The fraud monitoring system 10 also issues a notification when fraud is detected.

[0012] The fraud monitoring system 10 comprises a self-service POS terminal 20, a fraud monitoring device 40, a store server 52, and an attendant terminal 54, all of which are connected to each other by an in-store network 56 such as a LAN.

[0013] Self-service POS terminals 20 are generally installed in multiple locations within a store and are operated by customers themselves to perform registration processing, where they register the products they wish to purchase, and payment processing, where they accept payment for the registered products. Self-service POS terminals 20 are an example of a product sales data processing device in this disclosure.

[0014] The fraud monitoring device 40 is implemented, for example, on an edge server and communicates with the self-service POS terminal 20. The fraud monitoring device 40 obtains from the self-service POS terminal 20 the following: identification information (terminal ID) that uniquely identifies the self-service POS terminal 20, identification information (e.g., product code) that uniquely identifies the product purchased by the customer, obtained by the scanner 28 (see Figure 2) on the self-service POS terminal 20, and video footage (see Figure 2) of the customer grasping and moving the product. The fraud monitoring system 10 may also be configured with the fraud monitoring device 40 located on the cloud.

[0015] Further, the unauthorized monitoring device 40 performs unauthorized determination processing to determine, based on the product identification information obtained from the self-POS terminal 20 and the customer video obtained from the camera 50, whether a product for which the identification information has not been read is stored in a my bag or the like. Further, when the unauthorized monitoring device 40 detects customer fraud, it notifies the self-POS terminal 20 and the attendant terminal 54 to that effect.

[0016] Note that the unauthorized monitoring device 40 may be connected to a plurality of self-POS terminals 20 and perform unauthorized determination processing for the plurality of self-POS terminals 20 with a single unauthorized monitoring device 40.

[0017] The camera 50 is connected to the unauthorized monitoring device 40. The camera 50 captures the state in which a customer registering a product at the self-POS terminal 20 allows the product to be read by the scanner 28 and move it. Details of the video captured by the camera 50 will be described later (see FIGS. 5 and 6). When monitoring a plurality of self-POS terminals 20 with a single unauthorized monitoring device 40, a plurality of cameras 50 are provided, and each self-POS terminal 20 is monitored by a different camera 50.

[0018] The store server 52 monitors the operating states of each part by communicating with the self-POS terminal 20, the unauthorized monitoring device 40, and the attendant terminal 54. Further, the store server 52 acquires information related to settlement (sales information) from the self-POS terminal 20. Further, the store server 52 appropriately distributes the latest version of the product master, which is a master file storing product information, to the self-POS terminal 20.

[0019] The attendant terminal 54 acquires information related to whether a customer is committing fraud from the unauthorized monitoring device 40. When the operator of the attendant terminal 54 determines, based on the information obtained from the unauthorized monitoring device 40, that there is a possibility that a customer is committing fraud, for example, the operator instructs a non-illustrated mobile terminal possessed by a store clerk to go to the corresponding self-POS terminal 20 to check the situation.

[0020] The system configuration of the fraud monitoring system 10 is not limited to the example shown in Figure 1, and the self-service POS terminal 20 may also have the functions of the fraud monitoring device 40.

[0021] (Overview of a self-service POS terminal) The schematic configuration of the self-service POS terminal 20 included in the fraud monitoring system 10 will be explained using Figure 2. Figure 2 is a front view showing an example of a self-service POS terminal included in the fraud monitoring system of this embodiment.

[0022] The self-service POS terminal 20 includes a display 26, a touch panel 27, a scanner 28, a payment terminal 29, a receipt printer 30, a warning light 31, a basket stand 17, and a product storage stand 18.

[0023] The display 26 displays various information related to product registration and payment, as well as warning information indicating that a customer has committed fraud. The display 26 is, for example, an LCD panel or an OLED panel.

[0024] The touch panel 27 is installed stacked on the display 26 and acquires customer operation information for various operation buttons displayed on the display 26.

[0025] The scanner 28 has a reading window 281 on its front. The scanner 28 has a built-in camera that captures and reads barcodes and other code symbols attached to products held up to the reading window 281. As a result, the scanner 28 reads the product identification information (e.g., product code) registered in the code symbol.

[0026] The payment terminal 29 receives payment instructions from the customer for the registered goods. The payment terminal 29 handles cashless payments such as credit card payments, code payments, and electronic money payments. The self-service POS terminal 20 may also be equipped with a change dispenser (not shown) to allow cash payments.

[0027] The receipt printer 30 prints the transaction results onto a receipt and issues it.

[0028] The warning light 31 illuminates or flashes when a malfunction occurs in the self-service POS terminal 20 and the customer requests assistance from a store employee. The warning light 31 may also illuminate or flash if the customer engages in fraudulent activity to alert nearby store employees.

[0029] The basket stand 17 is a stand on which customers place their shopping baskets 11 containing their goods.

[0030] The product storage platform 18 is a platform on which customers place their reusable shopping bags 12, etc., to hold their registered products.

[0031] The customer takes out items one by one from the shopping basket 11 placed on the basket stand 17 and holds them up to the scanning window 281 of the scanner 28. Then, the customer places the items that were held up to the scanning window 281 of the scanner 28 into a reusable bag 12 or other container placed on the product storage stand 18. In other words, the items move along the trajectory of arrow A in Figure 2.

[0032] A camera 50 is installed facing downwards near the warning light 31. The camera 50 photographs the area P that encompasses the basket stand 17 (shopping basket 11), the scanner 28, and the reusable bag 12 (product storage stand 18). In other words, the camera 50 photographs the customer scanning products and moving them along arrow A.

[0033] (Hardware configuration of self-service POS terminals) The hardware configuration of the self-service POS terminal 20 will be explained using Figure 3. Figure 3 is a hardware block diagram showing an example of the hardware configuration of a self-service POS terminal included in the fraud monitoring system of the embodiment.

[0034] The self-service POS terminal 20 has a configuration in which a control unit 21, a storage unit 22, a peripheral device controller 24, and a communication controller 25 are connected to each other by an internal bus 23.

[0035] The control unit 21 controls the overall operation of the self-service POS terminal 20. The control unit 21 comprises a CPU (Central Processing Unit) 211, a ROM (Read Only Memory) 212, and a RAM (Random Access Memory) 213. The CPU 211 connects to the ROM 212 and the RAM 213 via internal buses such as an address bus and a data bus. The CPU 211 loads various programs stored in the ROM 212 and the memory unit 22 into the RAM 213. The CPU 211 controls the operation of the self-service POS terminal 20 by operating according to the various programs loaded into the RAM 213. In other words, the control unit 21 has the configuration of a typical computer.

[0036] The storage unit 22 is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Alternatively, the storage unit 22 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 22 stores the control program 221, the product master 222, the product registration file 223, and the settlement file 224.

[0037] The control program 221 is a program that controls the overall operation of the self-service POS terminal 20. The control program 221 may be provided stored in the memory unit 22, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD. The control program 221 may also be provided by storing it on a computer connected to a network and downloading it via the network. Furthermore, the control program 221 may be provided or distributed via a network such as the Internet.

[0038] The product master 222 is a master file that stores product information such as the product name and price of the products to be sold. The product master 222 stores the product name, unit price, additional information, etc., associated with a product code that uniquely identifies each product. Since the contents of the product master are updated as needed, the self-service POS terminal 20 maintains the contents of the product master 222 at all times by obtaining the latest product master from the store server 52 via the communication controller 25.

[0039] The product registration file 223 is a file that stores product information (e.g., product name, quantity, unit price, etc.) including the product code of the registered product, and the total purchase amount, associated with a transaction code that identifies a single transaction.

[0040] The settlement file 224 is a file that stores the results of settlements. The settlement file 224 stores the transaction date, transaction time, payment amount, settlement method, etc., associated with a transaction code that identifies a single transaction.

[0041] The control unit 21 is connected via the peripheral device controller 24 to various input / output devices, namely the display 26, touch panel 27, scanner 28, payment terminal 29, receipt printer 30, and warning light 31. Since the general outlines of each input / output device have been described above, further explanation will be omitted.

[0042] The control unit 21 receives and exchanges various information with the fraud monitoring device 40, the store server 52, and the attendant terminal 54 via the communication controller 25.

[0043] The control unit 21 outputs the product code read by the scanner 28, associated with the time it was read, to the fraud monitoring device 40 via the communication controller 25. The control unit 21 also obtains the fraud detection result from the fraud monitoring device 40 via the communication controller 25.

[0044] Furthermore, the control unit 21 outputs sales information, including the product registration file 223 and the payment file 224, to the store server 52 via the communication controller 25. The control unit 21 also obtains the latest product master from the store server 52 via the communication controller 25.

[0045] (Hardware configuration of the fraud monitoring device) The hardware configuration of the fraud monitoring device 40 will be explained using Figure 4. Figure 4 is a hardware block diagram showing an example of the hardware configuration of the fraud monitoring device included in the fraud monitoring system of the embodiment.

[0046] The fraud monitoring device 40 has a configuration in which a control unit 41, a storage unit 42, a peripheral device controller 44, and a communication controller 45 are connected to each other by an internal bus 43.

[0047] The control unit 41 controls the overall operation of the fraud monitoring device 40. The control unit 41 comprises a CPU 411, a ROM 412, and a RAM 413. The CPU 411 connects to the ROM 412 and the RAM 413 via internal buses such as an address bus and a data bus. The CPU 411 loads various programs stored in the ROM 412 and the memory unit 42 into the RAM 413. The CPU 411 controls the operation of the fraud monitoring device 40 by operating according to the various programs loaded into the RAM 413. In other words, the control unit 41 has the configuration of a typical computer.

[0048] The storage unit 42 is a storage device such as an HDD or SSD. Alternatively, the storage unit 42 may be a non-volatile memory such as flash memory that retains stored information even when the power is turned off. The storage unit 42 stores the control program 421, the video file 422, and the product code file 423.

[0049] The control program 421 is a program that controls the overall operation of the unauthorized monitoring device 40. The control program 421 may be provided stored in the storage unit 42, or it may be provided as an installable or executable file recorded on a computer-readable non-temporary recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD. The control program 421 may also be provided by storing it on a computer connected to a network and downloading it via the network. Furthermore, the control program 421 may be provided or distributed via a network such as the Internet.

[0050] The video file 422 is a file that stores the video footage captured by the camera 50. The video file 422 stores the video footage captured by the camera 50 and a time code indicating the time of capture, associated with identification information that identifies the self-service POS terminal 20 on which the camera 50 is installed.

[0051] The product code file 423 is a file that stores product codes obtained from the self-service POS terminal 20 and scanned by the customer using the scanner 28. The product code file 423 stores the product code and the time the product code was read, in association with identification information that identifies the self-service POS terminal 20.

[0052] The control unit 41 is connected to the camera 50 via the peripheral device controller 44. The functions of the camera 50 have been described above, so a further explanation will be omitted.

[0053] The control unit 41 receives and receives various information from the self-service POS terminal 20 and the attendant terminal 54 via the communication controller 45.

[0054] The control unit 41 obtains the product code read by the scanner 28 from the self-service POS terminal 20 via the communication controller 45. The control unit 41 also outputs the fraud detection result to the self-service POS terminal 20 via the communication controller 45.

[0055] Furthermore, the control unit 41 outputs the fraud detection result to the attendant terminal 54 via the communication controller 45.

[0056] (Images captured by a camera and methods for analyzing them) Figures 5 and 6 illustrate the image I captured by camera 50 and how the fraud monitoring device 40 analyzes image I. Figure 5 is an example of an image captured by the camera. Figure 6 is an example of how the fraud monitoring device recognizes a customer's scanning action from the image captured by the camera.

[0057] As shown in Figure 5, Image I includes a customer 2, a shopping basket 11 placed on a basket stand 17, a reusable shopping bag 12 placed on a product storage stand 18, and a scanner 28.

[0058] When registering an item, customer 2 grasps item 13 with their right hand and removes it from the shopping basket 11. Then, customer 2 holds the barcode or other code symbol 14 attached to item 13 up to the reading window 281 of the scanner 28. Once the item code is read, customer 2 places the item 13 they are holding into their reusable bag 12. In other words, customer 2 repeatedly performs the process of scanning the item code with the scanner 28 and placing it into the reusable bag 12 for all items 13 in the shopping basket 11. During this process, all items 13 move from the shopping basket 11, passing near the reading window 281 of the scanner 28, towards the reusable bag 12. The fraud monitoring device 40 of the fraud monitoring system 10 acquires multiple images I, i.e., video, taken at different times and analyzes how customer 2 scans the items 13 taken from the shopping basket 11 and places them into the reusable bag 12.

[0059] The fraud monitoring device 40 uses a series of temporally consecutive images I acquired from the self-service POS terminal 20 to determine whether customer 2 is committing fraud when holding product 13 over the scanner 28. These temporally consecutive images I are, for example, video captured at 30 frames per second, i.e., video rate. The capture interval for images I is not limited to the video rate, but may be a predetermined time interval.

[0060] The fraud monitoring device 40 sets up windows Wa, Wb, and Wc inside image I.

[0061] Window Wa is set in an area encompassing the shopping basket 11. More specifically, the fraud monitoring device 40 sets Window Wa at the location of the basket stand 17 on which the shopping basket 11 is placed. Window Wa is an example of a first area in this disclosure.

[0062] Window Wb is set in the area facing the reading window 281 of the scanner 28. Window Wb is an example of a second area in this disclosure. Since window Wb is an area of ​​interest for the fraud monitoring device 40 to determine whether product 13 has been scanned by scanner 28, window Wb will hereafter also be referred to as ROI (Region of Interest).

[0063] Window Wc is set in an area encompassing the reusable bag 12. More specifically, the fraud monitoring device 40 sets Window Wc at the location of the product storage tray 18 on which the reusable bag 12 is placed. Window Wc is an example of a third area in this disclosure.

[0064] Image I is a view of customer 2 from above, and therefore, customer 2's right hand 3 and left hand 4 are visible in Image I. The fraud monitoring device 40 performs a well-known person detection to detect the posture of customer 2 as seen in Image I. This detects a region Wr encompassing customer 2's right hand 3 and a region Wl encompassing customer 2's left hand 4. The fraud monitoring device 40 tracks customer 2's right hand 3 and left hand 4 by performing the same process on multiple temporally consecutive images I.

[0065] The fraud monitoring device 40 then detects that customer 2's right hand 3 or left hand 4 has entered window Wa where the shopping basket 11 is located and that customer 2 has grasped product 13. Customer 2's right hand 3 or left hand 4 entering window Wa is determined by the overlap of the window Wa with the location of the area Wr encompassing the right hand 3 or the area Wl encompassing the left hand 4. Furthermore, customer 2 grasping product 13 is determined by the detection of area Wm moving together with area Wr or area Wl. Figure 6 shows customer 2 grasping product 13 with their right hand 3.

[0066] The fraud monitoring device 40 determines whether the area Wm representing product 13 has crossed window Wb in the direction of arrow B in Figure 6. Whether the area Wm representing product 13 has crossed window Wb is determined, for example, by monitoring the time change in the overlap state between area Wm and window Wb.

[0067] Furthermore, the fraud monitoring device 40 determines whether the area Wm representing product 13 has left the window Wb, i.e., the ROI, in the direction of arrow B. Whether the area Wm representing product 13 has left the window Wb in the direction of arrow B can be determined, for example, by monitoring the time change in the overlap state between area Wm and window Wb.

[0068] Furthermore, the fraud monitoring device 40 obtains the product code of product 13, which has been read by the scanner 28 from the self-service POS terminal 20.

[0069] The fraud monitoring device 40 determines that there was no fraud in the registration operation of product 13 if the scanner 28 of the self-POS terminal 20 acquires the product code during the time period when the area Wm representing product 13 crosses area Wb (ROI) in the direction of arrow B, and area Wm representing product 13 is located in area Wb (ROI).

[0070] On the other hand, the fraud monitoring device 40 determines that there was fraud in the registration operation of product 13 if the scanner 28 of the self-service POS terminal 20 did not acquire the product code even though the area Wm representing product 13 crossed the area Wb (ROI) in the direction of arrow B.

[0071] (Timeline of fraud detection process performed by the fraud monitoring system (1)) Figure 7 illustrates the time chart of the fraud detection process performed by the fraud monitoring system 10 of the embodiment. Figure 7 is the first figure showing an example of the time chart of the fraud detection process performed by the fraud monitoring system of the embodiment. In particular, Figure 7 is an example of a time chart when two types of products are scanned consecutively.

[0072] At time t1, customer 2 presses the registration start button (not shown) displayed on the display 26 to begin registering the product.

[0073] The self-service POS terminal 20 acquires information indicating that the registration start button was pressed at time t2. The self-service POS terminal 20 then instructs the fraud monitoring device 40 to start taking pictures with the camera 50.

[0074] The fraud monitoring device 40 starts taking pictures with the camera 50 at time t3.

[0075] At time t4, customer 2 picks up the first item (hereinafter referred to as the first item) and moves it to area Wb (ROI).

[0076] The fraud monitoring device 40 detects that the first item entered the region Wb (ROI) at time t5 by processing the image captured by the camera 50.

[0077] Then, at time t6, customer 2 holds the first product over the reading window 281 of scanner 28.

[0078] The self-service POS terminal 20 reads the product's barcode at time t7 and obtains the product code.

[0079] At time t8, the self-service POS terminal 20 outputs the scanned product code to the fraud monitoring device 40.

[0080] The fraud monitoring device 40 acquires the product code output by the self-service POS terminal 20 at time t12. Thus, a delay time Δt occurs between the time the self-service POS terminal 20 outputs the product code and the time the fraud monitoring device 40 acquires the product code, as shown in Figure 7. This is due to the time delay required for data communication between the self-service POS terminal 20 and the fraud monitoring device 40. Because of this delay time Δt, the fraud monitoring device 40 sets the timing for starting the fraud detection process to the time when it is recognized that customer 2 has moved the next product to area Wb (ROI) (time t14 in Figure 7). More details will be described later.

[0081] During the delay time Δt, customer 2 removes the first product from area Wb (ROI) at time t9.

[0082] The fraud monitoring device 40 detects that the product has left the region Wb (ROI) at time t10 by processing the image captured by the camera 50.

[0083] At time t11, the self-service POS terminal 20 displays product information based on the scanned product code on the display 26.

[0084] At time t13, customer 2 picks up the next item (hereinafter referred to as the second item) and moves it to area Wb (ROI).

[0085] The fraud monitoring device 40 processes the image captured by the camera 50 to detect that the second item entered the region Wb (ROI) at time t14.

[0086] When the fraud monitoring device 40 detects that the second product has entered the region Wb (ROI), i.e., at time t14, it starts a fraud detection process to determine whether customer 2 has committed fraud in registering the product.

[0087] At time t15, the fraud monitoring device 40 finalizes the judgment result of the fraud detection process for the first product. The fraud monitoring device 40 then outputs the judgment result of the fraud detection process to the self-service POS terminal 20. If fraud is detected at this time, the fraud monitoring device 40 outputs to the self-service POS terminal 20 information indicating that fraud has occurred, as well as an image taken by the camera 50 of the first product entering the region Wb (ROI). Although not shown in the time chart, at time t15, the fraud monitoring device 40 also outputs the judgment result of the fraud detection process to the attendant terminal 54.

[0088] At time t16, the self-service POS terminal 20 obtains the result of the fraud detection process for the first product from the fraud monitoring device 40.

[0089] Then, at time t17, the self-service POS terminal 20 displays the judgment result on the display 26. Note that the judgment result only needs to be displayed when fraud is detected. An example of how the judgment result is displayed will be described in detail later (see Figure 9).

[0090] (Timeline of fraud detection process performed by the fraud monitoring system (2)) Using Figure 8, another time chart of the fraud detection process performed by the fraud monitoring system 10 of the embodiment will be explained. Figure 8 is a second figure showing an example of a time chart of the fraud detection process performed by the fraud monitoring system of the embodiment. In particular, Figure 8 is an example of a time chart when the product scan is completed and the checkout button is pressed.

[0091] Customer 2 picks up the last item at time t21 and moves it to area Wb (ROI).

[0092] The fraud monitoring device 40 processes the images captured by the camera 50 to detect that the last item entered the region Wb (ROI) at time t22.

[0093] At time t23, customer 2 holds the last item up to the reading window 281 of scanner 28.

[0094] The self-service POS terminal 20 reads the product's barcode at time t24 and obtains the product code.

[0095] At time t25, the self-service POS terminal 20 outputs the scanned product code to the fraud monitoring device 40.

[0096] The fraud monitoring device 40 acquires the product code output by the self-service POS terminal 20 at time t29. As described above, a delay time Δt occurs between the time the self-service POS terminal 20 outputs the product code and the time the fraud monitoring device 40 acquires the product code.

[0097] At time t28, the self-service POS terminal 20 displays product information based on the scanned product code on the display 26.

[0098] At time t30, customer 2 presses the unillustrated payment button displayed on display 26 to begin the payment process.

[0099] The self-service POS terminal 20 acquires information indicating that the payment button was pressed at time t31. The self-service POS terminal 20 then outputs an instruction to the fraud monitoring device 40 to stop the camera 50 from taking pictures.

[0100] At time t32, the fraud monitoring device 40 receives an instruction from the self-service POS terminal 20 to stop the camera 50 from taking pictures. The fraud monitoring device 40 then instructs the camera 50 to stop taking pictures.

[0101] Furthermore, at time t32, the fraud detection device 40 starts a fraud detection process to determine whether customer 2 has committed any fraudulent activity in registering products.

[0102] At time t33, the fraud monitoring device 40 finalizes the result of the fraud detection process for the last item. The fraud monitoring device 40 then outputs the result of the fraud detection process to the self-service POS terminal 20. If fraud is detected, the fraud monitoring device 40 outputs to the self-service POS terminal 20 information indicating that fraud has occurred, along with an image taken by camera 50 of the last item entering area Wb (ROI). Although not shown in the time chart, at time t33, the fraud monitoring device 40 also outputs the result of the fraud detection process to the attendant terminal 54.

[0103] At time t34, the self-service POS terminal 20 obtains the result of the fraud detection process for the last item from the fraud monitoring device 40.

[0104] Then, at time t35, the self-service POS terminal 20 displays the judgment result on the display 26.

[0105] (Notification screen displayed on self-service POS terminal) Using Figure 9, we will explain the notification screen that the fraud monitoring device 40 displays on the display 26 of the self-service POS terminal 20 when it determines that a customer has committed fraud. Figure 9 is a diagram showing an example of a notification screen that is displayed on the display of the self-service POS terminal when the fraud monitoring device determines that a customer has committed fraud.

[0106] When the self-service POS terminal 20 receives information indicating that the fraud monitoring device 40 has determined that customer 2 has committed fraud, it displays a notification screen 60 on the display 26.

[0107] The notification screen 60 includes a message 61, an image 62, and a close button 63.

[0108] Message 61 is information prompting you to scan item 13 again.

[0109] Image 62 is an image that includes product 13, which the fraud monitoring device 40 has determined to be suspected of fraud. The fraud monitoring system 10 of this disclosure notifies of the suspected fraud related to the previous product 13 when the next product after product 13 moved by customer 2 enters the second area immediately in front of scanner 28. Therefore, the notification is made for a product 13 that is different from the product that customer 2 is currently holding. To avoid confusion for customer 2, image 62 is displayed showing customer 2 holding product 13 that is the subject of the notification.

[0110] The close button 63 is used to close the notification screen 60. When the close button 63 is pressed, the display 26 shows the screen that was displayed before the notification screen 60 was shown.

[0111] (Functional configuration of the fraud monitoring system) The functional configuration of the fraud monitoring system 10 of the embodiment will be explained using Figure 10. Figure 10 is a functional block diagram showing an example of the functional configuration of the fraud monitoring system of the embodiment.

[0112] First, let's explain the functional configuration of the self-service POS terminal 20.

[0113] The control unit 21 of the self-service POS terminal 20 deploys the control program 221 to the RAM 213 and operates it, thereby realizing the following functional units as shown in Figure 10: the operation information acquisition unit 71, the operation information output unit 72, the product code acquisition unit 73, the product registration unit 74, the product code output unit 75, the fraud detection result acquisition unit 76, the detection result display unit 77, and the payment processing unit 78. Some or all of these functional parts may be realized by dedicated hardware.

[0114] The operation information acquisition unit 71 receives an instruction from customer 2 to start product registration. Specifically, the operation information acquisition unit 71 determines that an instruction to start product registration has been given when it detects that a product registration start button (not shown) displayed on the display 26 has been pressed. The operation information acquisition unit 71 also receives an instruction from customer 2 that the registration of all products has been completed. Specifically, the operation information acquisition unit 71 determines that product registration has been completed when it detects that a registration completion button (not shown) displayed on the display 26 has been pressed.

[0115] The operation information output unit 72 outputs to the fraud monitoring device 40 an instruction to start product registration obtained from customer 2, and an instruction indicating that the registration of all products has been completed.

[0116] The product code acquisition unit 73 acquires identification information (e.g., a product code) that uniquely identifies the product that customer 2 wishes to purchase by reading the product code symbol held over the reading window 281 of the scanner 28. The product code acquisition unit 73 is an example of an identification information acquisition unit in this disclosure.

[0117] The product registration unit 74 registers products that have product codes acquired by the product code acquisition unit 73. Specifically, the product registration unit 74 generates a product registration file 223 (see Figure 3) in which product information including product codes is registered, associated with a transaction code that identifies a single transaction.

[0118] The product code output unit 75 outputs the product code acquired by the product code acquisition unit 73 to the fraud monitoring device 40, associating it with the time it was read (acquisition time). The product code output unit 75 is an example of an identification information output unit in this disclosure.

[0119] The fraud detection result acquisition unit 76 acquires the result of the fraud detection process from the fraud monitoring device 40 when the next product to be registered by customer 2 moves from the first area where it is placed to the second area facing the scanner 28 that acquires product identification information. This process determines whether fraud occurred during the registration of the previous product.

[0120] The judgment result display unit 77, on the condition that the fraud judgment result acquisition unit 76 has acquired information indicating that fraud occurred during the registration of the previous product, displays information indicating that fraud occurred during the registration of the previous product on its own display 26 as a notification screen 60 (see Figure 9).

[0121] The payment processing unit 78 performs payment processing to accept payment for the total amount of the products registered in the product registration file 223. The payment processing unit 78 accepts payment for the total amount of the products according to the payment method specified by customer 2 (credit card payment, code payment, electronic money payment, etc.). If the self-service POS terminal 20 supports cash payment, the payment processing unit 78 may also accept cash payment for the products from the customer.

[0122] Next, the functional configuration of the fraud monitoring device 40 will be explained.

[0123] The control unit 41 of the fraud monitoring device 40 deploys the control program 421 to the RAM 413 and operates it, thereby realizing the operation information acquisition unit 81, the video information acquisition unit 82, the product code acquisition unit 83, the product registration behavior recognition unit 84, the fraud determination unit 85, and the determination result output unit 86 shown in Figure 10 as functional units. Some or all of these functional units may be realized by dedicated hardware.

[0124] The operation information acquisition unit 81 acquires from the self-service POS terminal 20 an instruction from customer 2 to start product registration and an instruction that all product registrations have been completed. Upon receiving the instruction to start product registration, the operation information acquisition unit 81 instructs the video information acquisition unit 82 to start recording video with the camera 50. Furthermore, upon receiving the instruction that all product registrations have been completed, the operation information acquisition unit 81 instructs the video information acquisition unit 82 to stop recording video with the camera 50.

[0125] The video information acquisition unit 82 instructs the camera 50 to start and stop recording video. The video information acquisition unit 82 also acquires video from the camera 50 showing the customer 2 grasping the product 13 they wish to purchase and moving it from the first area where the product 13 is placed, across the second area facing the scanner 28 of the self-service POS terminal 20 for registering the product 13, to the third area where the scanned product 13 is placed. In other words, the camera 50 acquires multiple images I taken at different times that are consecutive in time. The video information acquisition unit 82 is an example of a fraud judgment information acquisition unit in this disclosure.

[0126] The product code acquisition unit 83 acquires the product code read by the scanner 28 of the self-service POS terminal 20. The product code acquisition unit 83 is an example of a fraud judgment information acquisition unit in this disclosure.

[0127] The product registration behavior recognition unit 84 processes the video information acquisition unit 82 to recognize whether the product 13 held by customer 2 has entered the area directly in front of the scanner 28 of the self-service POS terminal 20, i.e., the second area. The product registration behavior recognition unit 84 also recognizes whether the product 13, having entered the second area, has crossed the second area and moved to the third area. Furthermore, the product registration behavior recognition unit 84 recognizes whether the product code acquisition unit 83 has acquired the product code.

[0128] The fraud detection unit 85, based on the video information acquisition unit 82's acquisition of video information, determines that the next product has entered the second area, and then starts a fraud detection process to determine whether fraud occurred with respect to the product that was moved immediately before the next product. The fraud detection unit 85 determines that fraud has occurred if product 13 crosses window Wb in Figure 6, but the self-POS terminal 20 does not read the product code of product 13. The fraud detection unit 85 also determines that no fraud has occurred if product 13 crosses window Wb in Figure 6, and the self-POS terminal 20 reads the product code of product 13.

[0129] The judgment result output unit 86 outputs information indicating that fraud has occurred to the self-service POS terminal 20 when the fraud detection unit 85 determines that fraud has occurred. Although not shown in Figure 10, the judgment result output unit 86 also outputs the judgment result of the fraud detection process to the attendant terminal 54.

[0130] (Process flow performed by a self-service POS terminal) Figure 11 illustrates the processing flow performed by the self-service POS terminal 20 included in the fraud monitoring system 10 of this embodiment. Figure 11 is a flowchart showing an example of the processing flow performed by the self-service POS terminal included in the fraud monitoring system of this embodiment.

[0131] The operation information acquisition unit 71 determines whether it has received an instruction from customer 2 to start product registration (step S11). If it is determined that an instruction to start product registration has been received (step S11: Yes), the process proceeds to step S12. On the other hand, if it is not determined that an instruction to start product registration has been received (step S11: No), the determination in step S11 is repeated.

[0132] In step S11, if it is determined that an instruction to start product registration has been received, the operation information output unit 72 outputs an instruction to start product registration to the fraud monitoring device 40 (step S12).

[0133] The product code acquisition unit 73 determines whether the scanner 28 has acquired a product code (step S13). If it is determined that a product code has been acquired (step S13: Yes), the process proceeds to step S14. On the other hand, if it is not determined that a product code has been acquired (step S13: No), the determination in step S13 is repeated.

[0134] In step S13, if it is determined that a product code has been obtained, the product code output unit 75 outputs the obtained product code to the fraud monitoring device 40 (step S14).

[0135] The product registration unit 74 registers the products with product codes acquired by the product code acquisition unit 73 in the product registration file 223 (step S15).

[0136] The operation information acquisition unit 71 determines whether it has received an instruction from customer 2 to complete product registration (step S16). If it is determined that an instruction to complete product registration has been received (step S16: Yes), the process proceeds to step S17. On the other hand, if it is not determined that an instruction to complete product registration has been received (step S16: No), the process proceeds to step S22.

[0137] In step S16, if it is determined that an instruction for product registration completion has been received, the operation information output unit 72 outputs an instruction for product registration completion to the fraud monitoring device 40 (step S17).

[0138] The fraud detection result acquisition unit 76 acquires the result of the fraud detection process, which determines whether fraud occurred during product registration, from the fraud monitoring device 40 (step S18).

[0139] The fraud detection result acquisition unit 76 determines whether fraud has been detected (step S19). If fraud is detected (step S19: Yes), the process proceeds to step S20. On the other hand, if fraud is not detected (step S19: No), the process proceeds to step S21.

[0140] In step S19, if fraud is detected, the detection result display unit 77 displays information on the display 26 as a notification screen 60 indicating that fraud occurred during product registration (step S20). After that, customer 2 rescans the product that was detected as fraudulent, and then the self-service POS terminal 20 performs the payment process. However, for the sake of simplicity, here we will assume that the self-service POS terminal 20 terminates processing after step S20.

[0141] On the other hand, if no fraud is detected in step S19, the settlement processing unit 78 performs a settlement process to accept payment for the total amount of the products registered in the product registration file 223 (step S22). After that, the self-service POS terminal 20 terminates the process.

[0142] Let's return to step S16. If it is not determined in step S16 that an instruction for product registration completion has been received, the product code acquisition unit 73 determines whether the scanner 28 has acquired a product code (step S22). If it is determined that a product code has been acquired (step S22: Yes), the process proceeds to step S23. On the other hand, if it is not determined that a product code has been acquired (step S22: No), the determination in step S22 is repeated.

[0143] In step S22, if it is determined that a product code has been obtained, the product code output unit 75 outputs the obtained product code to the fraud monitoring device 40 (step S23).

[0144] The product registration unit 74 registers the products with product codes acquired by the product code acquisition unit 73 in the product registration file 223 (step S24).

[0145] The fraud detection result acquisition unit 76 acquires the result of the fraud detection process, which determines whether fraud occurred during product registration, from the fraud monitoring device 40 (step S25).

[0146] The fraud detection result acquisition unit 76 determines whether fraud has been detected (step S26). If fraud is detected (step S26: Yes), the process proceeds to step S20. On the other hand, if fraud is not detected (step S26: No), the process returns to step S16 and continues with product registration.

[0147] In step S26, if fraud is detected, the detection result display unit 77 displays information on the display 26 as a notification screen 60 indicating that fraud occurred during product registration (step S20). After that, customer 2 rescans the product that was detected as fraudulent and continues product registration. However, for the sake of simplicity in this explanation, the self-service POS terminal 20 will be assumed to terminate processing after step S20.

[0148] (Process flow performed by the fraud monitoring device) The processing flow of the fraud monitoring device 40 included in the fraud monitoring system 10 of the embodiment will be explained using Figure 12. Figure 12 is a flowchart showing an example of the processing flow of the fraud monitoring device included in the fraud monitoring system of the embodiment.

[0149] The operation information acquisition unit 81 determines whether it has received an instruction to start product registration from the self-service POS terminal 20 (step S31). If it is determined that an instruction to start product registration has been received (step S31: Yes), the process proceeds to step S32. On the other hand, if it is not determined that an instruction to start product registration has been received (step S31: No), the determination in step S31 is repeated.

[0150] In step S31, if it is determined that an instruction to start product registration has been received, the video information acquisition unit 82 instructs the camera 50 to start recording video (step S32).

[0151] The product registration behavior recognition unit 84 processes the video information acquisition unit 82 to determine whether the product 13 held by customer 2 has entered the area immediately in front of the scanner 28 of the self-service POS terminal 20, i.e., the second area (step S33). If it is determined that the product 13 held by customer 2 has entered the second area (step S33: Yes), the process proceeds to step S34. On the other hand, if it is not determined that the product 13 held by customer 2 has entered the second area (step S33: No), the determination in step S33 is repeated.

[0152] In step S33, when it is determined that the product 13 held by customer 2 has entered the second area, the product code acquisition unit 83 determines whether it has acquired the product code read by the scanner 28 from the self-service POS terminal 20 (step S34). If it is determined that the product code has been acquired (step S34: Yes), the process proceeds to step S35. On the other hand, if it is not determined that the product code has been acquired (step S34: No), the process proceeds to step S36.

[0153] In step S34, if it is determined that the product code has been obtained, the fraud detection unit 85 determines that no fraud has occurred (step S35). The process then proceeds to step S37.

[0154] On the other hand, if it is determined in step S34 that the product code has not been obtained, the fraud detection unit 85 determines that fraud has occurred (step S36). The process then proceeds to step S37.

[0155] Following step S35 or step S36, the product registration behavior recognition unit 84 processes the video captured by the camera 50 to determine whether the product held by customer 2 has entered the area immediately in front of the scanner 28's reading window 281, i.e., the second area (step S37). If it is determined that the product held by customer 2 has entered the second area (step S37: Yes), the process proceeds to step S38. On the other hand, if it is not determined that the product held by customer 2 has entered the second area (step S37: No), the determination in step S37 is repeated.

[0156] In step S37, if it is determined that the product held by customer 2 has entered the second area, the determination result output unit 86 outputs the determination result of the fraud determination unit 85 to the self-service POS terminal 20 (step S38).

[0157] The operation information acquisition unit 81 determines whether it has received an instruction from the self-service POS terminal 20 to complete product registration (step S39). If it is determined that an instruction to complete product registration has been received (step S39: Yes), the process proceeds to step S40. On the other hand, if it is not determined that an instruction to complete product registration has been received (step S39: No), the process returns to step S33.

[0158] In step S39, if it is determined that an instruction to complete product registration has been received, the video information acquisition unit 82 instructs the camera 50 to stop recording video (step S40).

[0159] The product code acquisition unit 83 determines whether it has acquired the product code read by the scanner 28 from the self-service POS terminal 20 (step S41). If it is determined that the product code has been acquired (step S41: Yes), the process proceeds to step S42. On the other hand, if it is not determined that the product code has been acquired (step S41: No), the process proceeds to step S43.

[0160] In step S41, if it is determined that the product code has been obtained, the fraud detection unit 85 determines that no fraud has occurred (step S42). The process then proceeds to step S44.

[0161] On the other hand, if it is determined that the product code has not been obtained in step S41, the fraud detection unit 85 determines that fraud has occurred (step S43). The process then proceeds to step S44.

[0162] Following step S42 or step S43, the judgment result output unit 86 outputs the judgment result of the fraud detection unit 85 to the self-service POS terminal 20 (step S44). After that, the fraud monitoring device 40 terminates its processing.

[0163] (Effects of the embodiment) As described above, the fraud monitoring device 40 of the embodiment includes: a video information acquisition unit 82 (fraud judgment information acquisition unit) that acquires video footage of a customer 2 grasping the product 13 they wish to purchase and moving it from a first area where the product 13 is placed, across a second area facing the scanner 28 of the self-service POS terminal 20 (product sales data processing device) that registers the product 13, to a third area where scanned products 13 are placed; a product code acquisition unit 83 (fraud judgment information acquisition unit) that acquires a product code (identification information) that uniquely identifies the product 13 read by the scanner 28; a fraud judgment unit 85 that, on the condition that it is determined based on the video that the next product has entered the second area, starts a fraud judgment process to determine whether fraud has occurred with respect to the product 13 that was moved immediately before the next product; and a judgment result output unit 86 that, if the fraud judgment unit 85 determines that fraud has occurred, outputs information indicating that fraud has occurred to the self-service POS terminal 20. Therefore, even if there is a time delay in sending and receiving information between the self-service POS terminal 20 and the fraud detection device 40, fraud detection can be reliably performed. This ensures that the fraud detection result is reliably communicated to the customer.

[0164] Furthermore, in the fraud monitoring device 40 of the embodiment, the fraud determination unit 85 starts fraud determination processing for the last product entered into the second area, provided that it has obtained information from the self-service POS terminal 20 (product sales data processing device) indicating that the customer 2 has completed registering all the products they wish to purchase. Therefore, the fraud determination result for the last registered product can be reliably communicated to the customer.

[0165] Furthermore, the self-service POS terminal (product sales data processing device) of this embodiment includes a product code acquisition unit 73 (identification information acquisition unit) that acquires a product code (identification information) that uniquely identifies the product that customer 2 wishes to purchase, a product registration unit 74 that registers the product having the product code acquired by the product code acquisition unit 73, a product code output unit 75 (identification information output unit) that outputs the product code of the product to the fraud monitoring device 40 that monitors fraud by customer 2, a fraud determination result acquisition unit 76 that acquires the result of a fraud determination process from the fraud monitoring device 40 that determines whether fraud occurred when the previous product was registered when the next product to be registered by the customer moves from the first area where it is placed to the second area facing the scanner 28 that acquires the product identification information, and a determination result display unit 77 that displays information indicating that fraud occurred when the previous product was registered, provided that the fraud determination result acquisition unit 76 has acquired information indicating that fraud occurred when the previous product was registered.Therefore, even if there is a time delay in sending and receiving information between the self-service POS terminal 20 and the fraud monitoring device 40, fraud determination can be reliably performed. This ensures that the results of any fraudulent activity are reliably communicated to the customer.

[0166] Furthermore, in the self-service POS terminal (product sales data processing device) of this embodiment, the judgment result display unit 77 displays an image 62 of the product that has been determined to have been fraudulently acted upon, which was obtained from the fraud monitoring device 40, when the fraud monitoring device 40 determines that fraud has occurred. Therefore, the customer can be reliably informed of the product suspected of being fraudulent. As a result, the customer can reliably identify the product that needs to be rescanned.

[0167] Although embodiments of the present invention have been described above, these embodiments are illustrative and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0168] 2 customers 10 Fraud Monitoring System 11 Shopping cart 12 reusable bags 13 products 14 Code Symbols 17 Basket stand 18 product storage units 20 Self-service POS terminal (product sales data processing device) 26 displays 27 Touch panel 28 Scanners 281 Reading window 31 Warning light 40. Fraud detection device 422 video files 423 Product Code File 50 Cameras 52 Store Servers 54 Attendant terminal 56 In-store network 60 News screen 61 Message 62 images 63 Close button 71 Operation information acquisition section 72 Operation Information Output Unit 73. Product Code Acquisition Unit (Identification Information Acquisition Unit) 74 Product Registration Department 75. Product code output unit (identification information output unit) 76 Fraud judgment result acquisition unit 77 Judgment result display section 78 Payment Processing Unit 81 Operation information acquisition section 82 Video Information Acquisition Unit (Fraud Judgment Information Acquisition Unit) 83. Product Code Acquisition Unit (Fraud Detection Information Acquisition Unit) 84 Product Registration Behavior Recognition Unit 85 Fraud Judgment Department 86. Output unit for judgment result Image I Wa window (first region) Wb window (second area) Wc window (third area) Δt Delay time [Prior art documents] [Patent Documents]

[0169] [Patent Document 1] International Publication No. 2021 / 176840

Claims

1. A fraud detection information acquisition unit acquires video footage of a customer grasping the product they wish to purchase and moving it from a first area where the product is placed, across a second area facing a scanner provided by a product sales data processing device that registers the product, to a third area where scanned products are placed, and identification information that uniquely identifies the product, as read by the scanner. A fraud detection unit initiates a fraud detection process to determine whether fraud occurred with respect to the product moved immediately before the next product, based on the condition that the next product has entered the second area based on the aforementioned video. The system includes a determination result output unit that, when the fraud detection unit determines that fraud has occurred, outputs information indicating that fraud has occurred to the product sales data processing unit. Fraud monitoring device.

2. The fraud detection unit, Conditional on obtaining information from the product sales data processing device indicating that the customer has completed registering all products they wish to purchase, the fraud detection process for the last product to enter the second area is initiated. The fraud monitoring device according to claim 1.

3. An identification information acquisition unit that acquires identification information to uniquely identify the product that the customer wishes to purchase, A product registration unit that registers products having identification information acquired by the aforementioned identification information acquisition unit, The fraud monitoring device that monitors fraudulent activity by the aforementioned customer is equipped with an identification information output unit that outputs identification information for the aforementioned product, When the next product to be registered by the customer moves from the first area where it is placed to the second area facing the scanner that acquires the identification information of the product, the fraud detection result acquisition unit acquires the result of the fraud detection process from the fraud monitoring device, which determines whether fraud occurred when the previous product was registered. The fraud detection result acquisition unit has, on the condition that it has acquired information indicating that fraud occurred when the previous product was registered, a determination result display unit that displays information indicating that fraud occurred when the previous product was registered, Product sales data processing device.

4. The aforementioned determination result display unit is, If the fraud monitoring device determines that fraud has occurred, it will display an image of the product that has been determined to have been fraudulently obtained from the fraud monitoring device. Product sales data processing device according to claim 3.

5. The computer that controls the fraud monitoring device, A fraud detection information acquisition unit acquires video footage of a customer grasping the product they wish to purchase and moving it from a first area where the product is placed, across a second area facing a scanner provided by a product sales data processing device that registers the product, to a third area where scanned products are placed, and identification information that uniquely identifies the product, as read by the scanner. A fraud detection unit initiates a fraud detection process to determine whether fraud occurred with respect to the product moved immediately before the next product, based on the condition that the next product has entered the second area based on the aforementioned video. The fraud detection unit operates as a determination result output unit that outputs information indicating that fraud has occurred to the product sales data processing device when it determines that fraud has occurred. program.

Images