Fraud detection device, fraud detection system, and fraud detection program

Abstract

To enable inexpensive and accurate detection of fraudulent acts in which articles are taken out of a store without being settled after registration.SOLUTION: A fraud detection device includes a person detection unit, an article detection unit, and a fraud reporting unit. The person detection unit detects persons in a first area around a target device based on an image of the target device, which is the subject of the fraud detection. The article detection unit detects at least one of the articles in a second area around the target device and a container containing the articles based on the image. The fraud reporting unit reports fraud when the person detection unit does not detect any person and the article detection unit does not detect either the articles or the container while the settlement process is being executed for the articles.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] Embodiments of the present invention relate to a fraud detection device, a fraud detection system, and a fraud detection program.

Background Art

[0002] Patent Document 1 discloses a technique for suppressing fraud of taking out registered goods outside the store without settlement in a POS system including a registration device and a settlement device separately. This is to photograph a registered customer at the time of registration with the registration device and notify if the registered customer is not present in the photographed image on the settlement device side.

[0003] In the POS system disclosed in Patent Document 1, it is assumed that the registered customer registered by the registration device and the settlement customer who settles by the settlement device are the same person. Therefore, a high-performance processing device is required to perform processing that requires high person recognition ability, such as detecting feature amounts of a person and determining the same person by comparing the feature amounts. In addition, the photographing of the settlement customer on the settlement device side is not necessarily taken at the same photographing angle as the photographing of the registered customer on the registration device side, and misrecognition of the same person may occur. Therefore, the POS system disclosed in Patent Document 1 has a problem that fraud cannot be detected accurately and inexpensively.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by the embodiments of the present invention is to provide a fraud detection device, a fraud detection system, and a fraud detection program that can inexpensively and accurately detect fraud of taking out registered goods outside the store without settlement.

Means for Solving the Problem

[0006] In one embodiment, the fraud detection device includes a person detection unit, a product detection unit, and a fraud notification unit. The person detection unit detects a person within a first area around the target device based on an image capturing the target device that is the target for fraud detection. The product detection unit detects at least one of a product and a container that houses the product within a second area around the target device based on the image. The fraud notification unit notifies of fraud when, in a state where settlement processing is being executed for the product, the person detection unit does not detect a person and the product detection unit does not detect either the product or the container.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the settlement device will be described with reference to the drawings.

[0009] [First Embodiment] The fraud detection system according to the first embodiment corresponds to a semi-self-checkout system including one or more registration devices for registering products (hereinafter referred to as purchased products) purchased by customers by store clerks, and one or more settlement devices for customers to perform settlement based on the registration information corresponding to the purchased products registered by the registration devices. Specifically, in this embodiment, such a semi-self-checkout system is applied to the fraud detection system. Note that the settlement device can be rephrased as a cash register, a payment terminal, an accounting device, an accounting terminal, etc.

[0010] FIG. 1 is a diagram showing an example of the layout of a semi-self-checkout system in a store. As shown in FIG. 1, two registration devices 10 and two settlement devices 20 are arranged in a checkout area CA provided in the store. Note that FIG. 1 shows an example in which two registration devices 10 and two settlement devices 20 are installed in the checkout area CA, but the number of registration devices 10 and settlement devices 20 installed in the checkout area CA is not particularly limited. Also, the number of registration devices 10 and the number of settlement devices 20 do not have to be the same and may be different from each other.

[0011] The registration device 10 is a device for a store clerk SP to register purchased goods. This registration device 10 is installed on the counter CT. The settlement device 20 is a device for a customer CS to execute settlement processing for the goods they have purchased. As shown in FIG. 1, the settlement device 20 is installed at a position in the middle of the flow line from the counter CT where the registration device 10 is installed towards the soccer table ST.

[0012] Here, an example of the usage mode of the checkout system in the checkout area CA will be described. When the customer CS enters the store from the entrance, in FIG. 1 (not shown in the figure), they go to the sales floor of the goods, and put the goods they want to purchase into the product cage BK1. Then, after putting all the goods they want to purchase into the product cage BK1, the customer CS goes to the checkout area CA of the store and queues up behind other customers CS waiting in the waiting lane for product registration. Then, when their turn comes, the customer CS places the product cage BK1 on the counter CT.

[0013] The store clerk SP operating the registration device 10 performs an operation of registering the goods in the product cage BK1 brought by the customer CS as the goods in one transaction corresponding to the customer CS. The store clerk SP puts the registered goods into the product cage BK2 for storing the registered goods. The product cage BK1 and the product cage BK2 are distinguishable from each other by changing the appearance form such as changing the color. When the store clerk SP completes the registration of the goods corresponding to the customer CS, the store clerk SP performs an operation of instructing the registration device 10 to perform a total calculation.

[0014] In response to an operation instructing this total calculation, the registration device 10 calculates the total amount of the products registered in one transaction corresponding to the customer CS, and generates registration information including those amounts and information such as the device number of the registration device 10 and the name of the in-charge store clerk. Also, the registration device 10 identifies one of the settlement devices 20 that is not in use. Then, the registration device 10 displays the device number etc. of the identified settlement device 20 on the display screen on the store clerk SP side, and transmits the generated registration information to the settlement device 20. The store clerk SP verbally conveys the device number etc. of the settlement device 20 to the customer CS. The customer CS moves to the location where the settlement device 20 is installed for settlement, carrying the product cage BK2 containing the purchased products that have been registered. A mounting table LT for mounting the product cage BK2 is installed beside the settlement device 20. Therefore, the customer CS who has moved to the location where the identified settlement device 20 is installed can mount the product cage BK2 on the mounting table LT installed beside it, and execute the settlement process using the settlement device 20.

[0015] Note that when the number of purchased products is small, the product cages BK1 and BK2 are not used, and the customer CS may move while holding the products exposed. In such a case, the products will be directly placed on the mounting table LT.

[0016] In response to a start operation from customer CS or in response to detection of customer CS by a sensor or the like, the settlement device 20 displays a settlement guide or the like according to the registration information transmitted from the registration device 10. Customer CS pays the price according to the displayed guide by inserting cash corresponding to the price or by using an electronic commerce (EC) method such as credit card payment, electronic money payment, point payment, or code payment, that is, executes a settlement process. Code payment is also referred to as mobile payment or smartphone payment. For example, it can be used by associating a code with a credit card or electronic money in advance in a product purchase app installed on an information terminal such as a smartphone owned by customer CS. In response to the completion of the settlement, the settlement device 20 issues a receipt. Customer CS receives the issued receipt. After completing the settlement, customer CS with the product cage BK2 containing the purchased products moves, for example, to the soccer table ST and can transfer the products from the product cage BK2 to a product bag BG or the like using the soccer table ST. Then, after finishing transferring the products to the product bag BG, customer CS exits the checkout area CA from the exit of the checkout area CA not shown and further exits the store.

[0017] As described above, the semi-self checkout system has a configuration in which a registration device 10 for the operation of registering the purchased products bought by customer CS by a store clerk SP and a settlement device 20 for customer CS to execute a settlement process are provided separately. In a store equipped with such a semi-self checkout system, there is a possibility of an illegal act called "basket escape" in which the customer leaves the store with the unbilled products without settling the purchased products. This embodiment improves the detection performance of such illegal acts in the semi-self checkout system to suppress such illegal acts. That is, in this embodiment, the settlement device 20 becomes the target device that is the target for detecting fraud.

[0018] FIG. 2 is a block diagram of a semi-self-checkout system to which the fraud detection system according to the first embodiment is applied. In the checkout system, a plurality of registration devices 10, a plurality of settlement devices 20, and a plurality of cameras 30 are connected via a LAN (Local Area Network). One camera 30 is provided corresponding to each settlement device 20. That is, the settlement device 20 and the camera 30 have a one-to-one correspondence. Note that the LAN may be a wired LAN, a wireless LAN, or a combination of wired and wireless.

[0019] FIG. 3 is a block diagram showing a main part configuration of the settlement device 20 to which the fraud detection device according to the first embodiment is applied, that is, in which the fraud detection device is incorporated. FIG. 4 is a perspective view showing the appearance of the settlement device 20. As shown in FIG. 3, the settlement device 20 includes a processor 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an HDD (Hard Disk Drive) 204, a LAN interface 505, an I / O (Input / Output) device control unit 206, a deposit / withdrawal device 207, a scanner 208, a printer 209, a card reader 210, a touch panel 211, and a display 212. In FIG. 3, "interface" is abbreviated as "I / F". A computer of the settlement device 20 is configured by connecting the processor 201, the ROM 202, and a memory including the RAM 203.

[0020] The processor 201 corresponds to the central part of the above computer. The processor 201 controls each part in order to realize various functions as the settlement device 20 and the fraud detection device according to the present embodiment in accordance with an operating system or a control program. The processor 201 is, for example, a CPU (Central Processing Unit). The processor 201 may be, for example, an MPU (Micro Processing Unit), an SoC (System on a Chip), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field-Programmable Gate Array). Alternatively, the processor 201 may be a combination of a plurality of these.

[0021] The ROM 202 and the RAM 203 correspond to the main memory part of the above computer. The ROM 202 stores an operating system or a control program. The RAM 203 stores data necessary for the processor 201 to execute processes for controlling each part. As one of such data, the RAM 203 stores a settlement mode flag 2031. The settlement mode flag 2031 is, for example, a 1-bit flag set to indicate that the settlement process is being performed. Further, the RAM 203 can store registration information transmitted from the registration device 10 as one of such data. Furthermore, the RAM 203 is used as a work area in which data is appropriately rewritten by the processor 201.

[0022] The HDD 204 stores various control programs. The control programs can include a fraud detection program for operating the settlement device 20 as a fraud detection device according to the embodiment. Further, the HDD 204 can also store the logs of the settlement process as log data. Note that the settlement device 20 may be provided with a rewritable storage device such as an EEPROM (Electric Erasable Programmable Read-Only Memory) or an SSD (Solid State Drive) instead of or in addition to the HDD 204.

[0023] The LAN interface 505 is a communication device for communicating with the registration device 10 and the camera 30 via the LAN.

[0024] The I / O device control unit 206 controls the transmission of control information from the processor 201 to the deposit / withdrawal device 207, the scanner 208, the printer 209, the card reader 210, the touch panel 211, and the display 212, and the transmission of data in the reverse direction from these deposit / withdrawal device 207, scanner 208, printer 209, card reader 210, touch panel 211, and display 212 to the processor 201.

[0025] The deposit / withdrawal device 207, the scanner 208, the printer 209, the card reader 210, the touch panel 211, and the display 212 will be described later.

[0026] As shown in FIG. 4, the settlement device 20 includes a first housing 213 and a second housing 214. The first housing 213 includes the deposit / withdrawal device 207 and the scanner 208. The second housing 214 is placed on the upper surface 215 of the first housing 213 and includes the display 216, the printer 209, and the card reader 210.

[0027] The deposit and withdrawal device 207 includes a coin insertion slot 217, a bill insertion slot 218, a bill dispensing outlet 219, and a coin dispensing section 220. The deposit and withdrawal device 207 stores the bills inserted into the bill insertion slot 218 in a bill storage section (not shown). Further, the deposit and withdrawal device 207 stores the coins inserted into the coin insertion slot 217 in a coin storage section (not shown). Then, the deposit and withdrawal device 207 dispenses change bills from the bill dispensing outlet 219 in response to a change dispensing request from the processor 201. Also, the deposit and withdrawal device 207 dispenses change coins to the coin dispensing section 220 in response to a change dispensing request from the processor 201.

[0028] The scanner 208 is, for example, an image sensor such as a CCD (Charge Coupled Device) or an optical scanner as an imaging section for reading a code symbol for code settlement.

[0029] The printer 209 has a printing section (not shown) built into the second housing 214 and a receipt issuing port 222 provided on the front surface 221 of the second housing 214. The printer 209 prints a receipt and issues it from the receipt issuing port 222 according to the control of the processor 201.

[0030] The card reader 210 is arranged on the left side of the second housing 214 and on the upper surface 215 of the first housing 213. The card reader 210 reads and writes information on a credit card, a membership card, etc. inserted through the card insertion slot 223. Further, the card reader 210 may be provided with a wireless communication function for enabling non-contact settlement called touch settlement, contactless, or electronic money settlement by performing wireless communication with a credit card or a smartphone equipped with a non-contact IC chip or an IC tag.

[0031] The touch panel 211 is provided on the surface of a display 216 attached above the second housing 214. The display 216 is composed of, for example, a liquid crystal panel and displays information regarding the operating state of the settlement device 20 in images and characters. The touch panel 211 is provided on the surface of the display 216 and outputs information based on the position touched by the customer CS to the processor 201.

[0032] In addition, the settlement device 20 includes a columnar display pole 224 erected on the rear side of the upper surface 215 of the first housing 213. The display pole 224 has a light emitting portion 225 at its upper tip portion.

[0033] The light emitting portion 225 includes a plurality of LEDs (Light Emitting Diodes) that emit blue light and a plurality of LEDs that emit red light. Note that the reason for using a plurality of LEDs is to obtain emission intensity, so if the intensity can be maintained with a single LED, each color of LED may be a single one. The light emitting portion 225 may use LEDs of a color different from blue and red, or may use other light emitting elements. Since the light emitting portion 225 is provided at the upper end of the tall display pole 224, it is easily visible not only to the store clerk SP who is an operator of the registration device 10 but also to other store clerks SP. Since light emission can be easily confirmed from the position of the registration device 10, it is also easily visible for the customer CS standing in front of the registration device 10 to visually confirm.

[0034] When the settlement device 20 receives registration information from the registration device 10, the light emitting portion 225 emits light in a first light emitting state indicating a state of waiting for the customer CS, for example, flashing blue. Also, when there is a possibility that fraud such as basket escape has occurred with respect to the settlement process for the registration information received by the settlement device 20 from the registration device 10, the light emitting portion 225 enters a second light emitting state, for example, flashing red, to notify the store clerk SP of the abnormality.

[0035] By switching the light emission pattern of the light emitting portion 225 in this way, when there is a possibility that basket escape has occurred, the store clerk SP can be alerted. By adopting a configuration in which the light emission pattern changes when there is a possibility that fraud such as basket escape has occurred, the customer is made aware that monitoring against fraudulent acts is being carried out, thereby aiming to suppress fraudulent acts. The method for determining the occurrence of fraud and switching the light emission pattern will be described later together with the functional configuration of the processor 201 of the settlement device 20.

[0036] In addition, the display pole 224 has a display 212 for displaying images and characters below the light-emitting unit 225. The display 212 is composed of, for example, a liquid crystal panel. The display 212 displays, for example, the content of an error generated by the settlement device 20 and the like.

[0037] The customer CS operates this settlement device 20 to perform a process of paying the price of a product, that is, a settlement process, in cash, by credit card, electronic money, or the like. In the settlement process, the settlement device 20 displays the total billing amount of a single transaction on the display 216 based on the registration information, that is, the transaction information received from the registration device 10, and controls the process related to the accounting for the amount. As described above, the registration information includes information such as the breakdown of purchased products, the total amount, the device number of the registration device 10 as the transmission source, and the name of the responsible store clerk in a single transaction to be processed. The registration information may further include other information necessary for receipt printing. For example, the registration information can include promotional information for receipt printing as other information.

[0038] As shown in FIG. 4, on the right side of the settlement device 20, for example, facing the front, there is a mounting table LT for mounting a product or a product cage BK2. The mounting table LT is an example of a mounting portion for mounting at least one of the product and the product cage BK2 when the settlement process is executed in the settlement device 20. A camera 30 is installed above the settlement device 20 and the mounting table LT so as to photograph the periphery of the settlement device 20 and the mounting table LT. The camera 30 is, for example, attached to an arm extending upward from the settlement device 20 and installed at the tip portion thereof facing downward. Alternatively, the camera 30 may be installed by embedding it in the ceiling or hanging it. In this embodiment, the installation method of the camera 30 is not particularly limited as long as the necessary area can be photographed.

[0039] Hereinafter, the fraud detection operation by the fraud detection system according to this embodiment having such a configuration and the fraud detection device applied to the settlement device 20 will be described.

[0040] FIGS. 5 to 7 are schematic diagrams for explaining the situation in which the processor 201 of the settlement device 20 to which the fraud detection device is applied determines the presence or absence of fraud. When the settlement device 20 is in the settlement standby state after receiving the registration information from the registration device 10, the camera 30 photographs an area including the periphery of the settlement device 20 and the mounting table LT. In FIGS. 5 to 7, this area is shown as the photographing area PA. For example, as shown in FIG. 5, when a person is detected in front of the settlement device 20, the processor 201 determines that it is OK, that is, there is no fraud. Further, for example, as shown in FIG. 6, when a product or the product cage BK2 is detected on the mounting table LT, the processor 201 also determines that it is OK, that is, there is no fraud. On the other hand, for example, as shown in FIG. 7, when neither the person in front of the settlement device 20 nor the product or the product cage BK2 on the mounting table LT is detected, the processor 201 determines that it is NG, that is, there is fraud.

[0041] Next, a specific example of the operation of the settlement device 20 and the fraud detection device applied to the settlement device 20 will be described.

[0042] FIG. 8 is a flowchart showing the main procedure of the information processing related to settlement executed by the processor 201 of the settlement device 20. For example, the processor 201 executes this process based on the control program stored in the ROM 202 or the HDD 204 in response to the power-on of the settlement device 20 or a predetermined start operation by the touch panel 211 in the clerk mode by the clerk SP. Unless otherwise specified, the processing operation of the processor 201 shall transition to ACT(x + 1) after ACTx (x is a natural number). Also, the procedure shown in FIG. 8 is an example. The procedure is not particularly limited as long as the same result can be obtained.

[0043] As ACT201, the processor 201 determines whether there is a settlement instruction. Specifically, the processor 201 makes this determination by determining whether it has received registration information transmitted from any of the registered devices 10 via the LAN by the LAN interface 505. If there is no settlement instruction, the processor 201 determines NO in ACT201 and repeats this ACT201 again. Thus, the processor 201 waits for a settlement instruction from any of the registered devices 10. And if there is a settlement instruction, the processor 201 determines YES in ACT201 and proceeds to ACT202.

[0044] As ACT202, the processor 201 starts the light emission of the light emitting unit 225. Although omitted in FIG. 3, the light emitting unit 225 is also connected to the I / O device control unit 206, and the processor 201 can control the lighting state of the light emitting unit 225 via this I / O device control unit 206. Therefore, the processor 201 causes the light emitting unit 225 to emit light in a first light emission state indicating that it is in a state of waiting for the customer CS, for example, blinking blue.

[0045] As ACT203, the processor 201 sets a settlement mode flag 2031 indicating that the settlement process is in progress in the RAM 203. For example, the processor 201 sets the value of the settlement mode flag 2031 stored in the RAM 203 to "1".

[0046] As ACT204, the settlement process is executed. In this settlement process, the processor 201 causes the display 216 to display a settlement guide or the like according to the received registration information, accepts settlement by the customer CS by the deposit / withdrawal device 207, the scanner 208 or the card reader 210, and if the settlement is completed, issues a receipt by the printer 209.

[0047] After the completion of the settlement process, as ACT205, the processor 201 clears the settlement mode flag 2031 set in the RAM 203. For example, the processor 201 sets the value of the settlement mode flag 2031 stored in the RAM 203 to "0".

[0048] As ACT206, the processor 201 ends the light emission of the light emitting unit 225. After that, the processor 201 proceeds to ACT201.

[0049] FIG. 9 is a flowchart showing the main procedure of the fraud detection process executed by the processor 201 of the settlement device 20. This fraud detection process is a process as a fraud detection device by the processor 201. The processor 201 starts this fraud detection process, for example, together with the start of the information processing related to the settlement shown in FIG. 8. That is, the processor 201 executes the information processing related to the settlement and the fraud detection process in parallel.

[0050] As ACT211, the processor 201 determines whether or not the settlement mode flag 2031 is set in the RAM 203. That is, the processor 201 determines whether or not the value of the settlement mode flag 2031 stored in the RAM 203 is “1”. If the settlement mode flag 2031 is not set, the processor 201 determines NO in ACT211 and repeats this ACT211 again. In this way, the processor 201 waits for the settlement mode flag 2031 to be set, that is, waits for the start of the settlement process. And if the settlement mode flag 2031 is set, the processor 201 determines YES in ACT211 and proceeds to ACT212.

[0051] As ACT212, the processor 201 acquires a captured image from the corresponding camera 30. For example, the HDD 204 stores correspondence relation information indicating which of the plurality of cameras 30 corresponds to the settlement device 20. The processor 201 acquires the captured image of the corresponding camera 30 connected to the LAN via the LAN interface 505 based on the correspondence relation information.

[0052] As ACT213, the processor 201 determines whether a customer CS has been detected based on the acquired captured image. That is, the processor 201 determines whether there is a person in front of the front surface of the payment device 20. If the customer CS is detected, the processor 201 determines YES in ACT213 and proceeds to ACT215. If the customer CS is not detected, the processor 201 determines NO in ACT213 and proceeds to ACT214.

[0053] As ACT214, the processor 201 determines whether a product has been detected based on the acquired captured image. For example, the processor 201 determines whether a product or a product cage BK2 is placed on the placement table LT. If the product or the product cage BK2 is placed, the processor 201 determines YES in ACT214 assuming that the product has been detected and proceeds to ACT211. If the product is not detected, the processor 201 determines NO in ACT214 and proceeds to ACT216.

[0054] Here, the detection of the customer and the product in ACT213 and ACT214 will be described in more detail. FIG. 10 is a schematic diagram for explaining the detection area in the captured image. Since the camera 30 is fixedly installed, the positional relationship between the camera 30, the payment device 20, and the placement table LT is invariant. Therefore, as shown in FIG. 10, the captured image PI always includes a payment device image 20I that is an image of the payment device 20 and a placement table image LTI that is an image of the placement table LT. And, in some cases, the captured image PI also includes a customer image CSI that is an image of the customer CS. The processor 201 sets a customer detection area ROI1 for detecting a customer and a product detection area ROI2 for detecting a product for the captured image PI, cuts out the images of the respective detection areas, and detects a customer or a product from the cut-out images. Note that since the positional relationship between the camera 30, the payment device 20, and the placement table LT is invariant, the positional relationship between the customer detection area ROI1 and the product detection area ROI2 set by the processor 201 is also invariant.

[0055] Specifically, the processor 201 sets a certain area on the front surface of the cash register 20 as the customer detection area ROI1. Then, the processor 201 uses known image recognition technology to detect whether there is a person in the image of this customer detection area ROI1. For example, the processor 201 calculates the Histograms of Oriented Gradients (HOG) feature amount from the image of the customer detection area ROI1, and linearly classifies the feature amount by a learned Support Vector Machine (SVM) to detect a person in an upright posture in the image of the customer detection area ROI1. The area corresponding to the customer detection area ROI1 in the checkout area CA is an example of the first area around the cash register 20 that is the target for detecting fraud. The processor 201 is an example of a person detection unit that detects a person in the first area around the cash register 20 based on the captured image PI captured by the camera 30. The camera 30 is an example of an image sensor.

[0056] Also, the processor 201 sets a certain area including the mounting table LT as the product detection area ROI2. Then, the processor 201 detects whether there is a product or a product basket BK2 in the image of this product detection area ROI2. For example, the processor 201 previously acquires an image of the product detection area ROI2 when no product or product basket BK2 is placed on the mounting table LT, and compares this image with the current image of the product detection area ROI2 to recognize the product or the product basket BK2. Known image processing technology shall be used for the recognition. For example, the HOG feature amount is calculated from each other's images, and it is recognized whether there is a product or a product basket BK2 depending on whether the Euclidean distance of the calculated HOG feature amount exceeds a threshold value. The area corresponding to the product detection area ROI2 in the checkout area CA is an example of the second area around the cash register 20. The product basket BK2 is an example of a container that houses products. The processor 201 is an example of a product detection unit that detects at least one of a product and a container that houses products in the second area around the cash register 20 based on an image capturing the cash register 20 which is the target device.

[0057] FIG. 11 is a schematic diagram for explaining another example of a detection area in a captured image. Customer CS does not always put the merchandise in merchandise basket BK2 and move to the checkout device 20, and may use shopping cart SC. That is, customer CS may put the merchandise basket BK2 containing the merchandise on shopping cart SC, i.e., contain it, and move. Thus, shopping cart SC is an example of a container that contains merchandise. When customer CS performs checkout processing at checkout device 20, customer CS places shopping cart SC in a holding area around mounting table LT. To handle such a case where shopping cart SC is used, as shown in FIG. 11, merchandise detection area ROI2 set by processor 201 is an area such that even merchandise cart image SCI, which is an image of shopping cart SC in captured image PI, also enters that merchandise detection area ROI2. Thus, the second area, the area around checkout device 20, can be an area including mounting table LT, which is a mounting portion on which merchandise basket BK2 is placed when checkout processing is performed at checkout device 20, and a holding area where shopping cart SC around mounting table LT is held. Processor 201 detects whether any of merchandise, merchandise basket BK2, and shopping cart SC exists in the image of this merchandise detection area ROI2. Note that customer detection area ROI1 and merchandise detection area ROI2 may partially overlap.

[0058] As described above, processor 201 cuts out an image within customer detection area ROI1 corresponding to the first area from captured image PI including the first area and the second area captured by camera 30 to detect a person, and also cuts out an image within merchandise detection area ROI2 corresponding to the second area from captured image PI to detect merchandise or merchandise basket BK2 or shopping cart SC, which is a container.

[0059] Return to the description of FIG. 9. As ACT215, the processor 201 determines whether the customer CS is facing the payment device 20. The processor 201 is an example of a person orientation detection unit that detects the orientation of the person detected by the person detection unit. If the customer CS is facing the payment device 20, the processor 201 determines YES in ACT215 and proceeds to ACT211. If the customer CS is not facing the payment device 20, the processor 201 determines NO in ACT215 and proceeds to ACT214. That is, even though it is determined in ACT213 that the customer CS has been detected, it proceeds to ACT214 in the same manner as when the customer CS has not been detected. That is, the detection result of the customer CS is ignored. In this way, the processor 201 is an example of a detection invalidation unit that invalidates the detection of the person detection unit when the orientation of the person is not in the direction of the payment device 20.

[0060] Here, the orientation of the customer determined in ACT215 will be described in more detail. FIG. 12 is a schematic diagram for explaining 0° of the orientation of the customer in the captured image. As shown in FIG. 12, the processor 201 sets the direction from the center of the customer detection area ROI1 set around the payment device 20 to the center of the payment device image 20I, which is the image of the payment device 20, as 0°.

[0061] FIG. 13 is a schematic diagram for explaining the range in which it is determined that the customer CS is facing the payment device 20. As shown in FIG. 13, if the orientation of the person detected from the customer detection area ROI1 is within ±45°, the processor 201 determines that it is OK, that is, the customer CS is facing the payment device 20. Conversely, if the orientation of the person detected from the customer detection area ROI1 is not within ±45°, the processor 201 determines that it is NG, that is, the customer CS is not facing the payment device 20. Here, the "orientation of the person" means the "orientation of the face and body" in the customer image CSI. Since the camera 30 is shooting from above the customer CS, the processor 201 can easily determine the orientation of the face and the body. When the orientation of the face and the body are different, it may be possible to prioritize one predetermined orientation such as the face, or calculate the orientation by taking the average or weighted average of each orientation. Also, the position of the customer image CSI within the customer detection area ROI1 is not constant. In ACT215, the processor 201 determines only the orientation without considering the position.

[0062] Return to the description of FIG. 9. As ACT216, the processor 201 determines whether or not a specified period has elapsed since the payment mode flag 2031 was set. This specified period may be time or the number of acquired captured images PI, that is, the number of frames. This specified period can be set based on the distance from the registration device 10 to the payment device 20. The processor 201 sets the specified period based on the device number of the registration device 10 included in the registration information. For example, it is also possible to store the distance information for each registration device 10 in the HDD 204 and have the processor 201 calculate the specified period from the distance information, or to store the specified period for each registration device 10 in the HDD 204 in advance. If the specified period has not elapsed, the processor 201 determines NO in ACT216 and proceeds to ACT211. If the specified period has elapsed, the processor 201 determines YES in ACT216 and proceeds to ACT217.

[0063] Note that if the settlement process ends and the settlement mode flag 2031 is cleared during the loop of ACT211 to ACT216, it will be determined as NO in ACT211, so the processes of ACT212 to ACT216 will end.

[0064] As ACT217, the processor 201 issues a fraud notification on the assumption that there is a high possibility of an illegal act of taking out a registered product outside the store without settlement. The processor 201 is an example of a fraud notification unit that issues a fraud notification when the person detection unit does not detect a person and the product detection unit does not detect either the product or the container while the settlement process for the product is being executed.

[0065] The fraud notification includes, for example, the emission of light from the light emitting unit 225. That is, the processor 201 causes the light emitting unit 225 to emit light in a second light emission state, for example, blinking red, to notify the store clerk SP of the possibility of fraud. Further, the fraud notification may be such that the LAN interface 505 transmits fraud information indicating a high possibility of fraud to the registration device 10, which is the transmission source of the registration information, via the LAN. When the registration device 10 receives this fraud information, it notifies the store clerk SP of the possibility of fraud by displaying it on the display screen on the store clerk SP side of the registration device 10. Furthermore, if a camera is also arranged on the registration device 10 side to capture an image of a registered customer such as a face image, the registration device 10 protects the image of the registered customer allocated to the settlement device 20 that transmitted the fraud information from being deleted, or saves it as a separate file with a file name.

[0066] As ACT218, the processor 201 determines whether there is a predetermined operation for canceling the fraud notification. This cancellation operation may be an operation on the touch panel 211 in the clerk mode by the clerk SP, or may be the reception of cancellation information transmitted in response to the cancellation operation on the registration device 10. If the cancellation operation is not performed, the processor 201 determines NO in ACT218 and repeats this ACT217 again. Thus, the processor 201 waits for the cancellation operation. If the cancellation operation is performed, the processor 201 determines YES in ACT218 and proceeds to ACT211.

[0067] As described above, the processor 201 of the settlement device 20 to which the fraud detection device according to the first embodiment is applied detects a person within the first area around the settlement device 20 that is the target of fraud detection, and also detects at least one of the goods within the second area around the settlement device 20 and at least one of the goods cages BK2 and the goods carts SC that store the goods. When a person is not detected and none of the goods, the goods cage BK2, and the goods cart SC are detected while the settlement process for the goods by the customer CS is being executed, fraud is notified. Thus, according to the fraud detection device according to the first embodiment, since it is not necessary to detect who the person is, a processor 201 having high-performance processing capabilities is not required. Therefore, it is possible to inexpensively and accurately detect the illegal act of taking out the goods outside the store without settling them after registration. Also, on the premise that the registered customer and the settling customer are the same person, for example, in the case where a husband transports the goods to the registration device 10 and a wife settles them at the settlement device 20 when multiple people go shopping, it will be recognized that an illegal act has occurred. In contrast, in the fraud detection device according to the first embodiment, since it does not assume that the registered customer and the settling customer are the same person, such false fraud notifications do not occur.

[0068] Also, the processor 201 of the settlement device 20 to which the fraud detection device according to the first embodiment is applied detects the orientation of the detected person, and if this orientation is not in the direction of the settlement device 20, the detection by the person detection unit is invalidated. Therefore, according to the fraud detection device according to the first embodiment, when the detected person is not facing the settlement device 20, there is a possibility that the person is not the one who performs the settlement. By ignoring the detection of such a person, fraud notification can be surely performed.

[0069] Note that the second area is an area including a mounting table LT on which at least one of the product and the product basket BK2 is placed when the customer CS performs a settlement process in the settlement device 20. Alternatively, the second area is an area including, in addition to the mounting table LT, a storage portion around the mounting table LT where the shopping cart SC is stored when the customer CS settlement process is performed in the settlement device 20. Therefore, according to the fraud detection device according to the first embodiment, it is possible to detect a product for an area where the probability of the presence of a product is high when the customer CS performs a settlement process.

[0070] In addition, the processor 201 of the settlement device 20 to which the fraud detection device according to the first embodiment is applied detects a person based on an image within a customer detection area ROI1 corresponding to the captured image of the first area captured by the camera 30. More specifically, the processor 201 cuts out an image within the customer detection area ROI1 corresponding to the first area from the captured image PI including the first area and the second area captured by the camera 30, and detects a person. As described above, according to the fraud detection device according to the first embodiment, a person can be detected by narrowing down to an image of a specific area from the captured image PI captured by the camera 30. Therefore, since the entire captured image PI is not targeted for processing, high-speed processing can be achieved. This person detection does not include the recognition of the person, but only whether the person exists or not. Therefore, the processor 201 is not required to have high processing power.

[0071] In addition, the processor 201 of the settlement device 20 to which the fraud detection device according to the first embodiment is applied detects at least one of a product, a product cage BK2, and a product cart SC based on an image within a product detection area ROI2 corresponding to an image of the second area captured by the camera 30. More specifically, the processor 201 cuts out an image within the product detection area ROI2 corresponding to the second area from the captured image PI including the first area and the second area captured by the camera 30, and detects at least one of a product, a product cage BK2, and a product cart SC. As described above, according to the fraud detection device according to the first embodiment, the product, the product cage BK2, and the product cart SC are detected by narrowing down to an image of a specific area from the captured image PI captured by the camera 30. Therefore, since the entire captured image PI is not targeted for processing, high-speed processing can be achieved. This detection of products and the like does not include recognition of what the products and the like are, but only whether they exist or not, so the processor 201 is not required to have high processing capabilities.

[0072] [Second Embodiment] Next, the second embodiment will be described. For configurations and operations similar to those of the first embodiment, the same reference numerals as those in the first embodiment are used, and the description thereof is omitted.

[0073] In the second embodiment, the fraud detection process in the fraud detection device applied to the settlement device 20 is different from that in the first embodiment. FIG. 14 is a flowchart showing the main procedure of the fraud detection process executed by the processor 201 of the settlement device 20 to which the fraud detection device according to the second embodiment is applied. As shown in FIG. 14, in the second embodiment, a touch panel input detection subroutine is inserted as ACT220 before the process of determining whether the settlement mode flag 2031 of ACT211 is set.

[0074] Figure 15 is a flowchart showing the main procedure of the touch panel input detection subroutine executed by this ACT220. In this touch panel input detection subroutine, the processor 201 determines, as ACT2201, whether there has been an input operation by the customer CS on the touch panel 211. The processor 201 is an example of an input detection unit that detects an input operation by a person on the settlement device 20. If there has been an input operation on the touch panel 211, the processor 201 determines YES in ACT2201 and proceeds to ACT2202. If there has been no input operation on the touch panel 211, the processor 201 determines NO in ACT2201 and proceeds to ACT2203.

[0075] As ACT2202, the processor 201 resets a timer built in the processor 201 or a separately provided timer (not shown).

[0076] As ACT2203, the processor 201 counts up the timer.

[0077] As ACT2204, the processor 201 determines whether the time measured by the timer has elapsed a predetermined time. This predetermined time is not particularly limited, but for example, it is about 5 seconds. If the predetermined time has not elapsed, the processor 201 determines NO in ACT2204 and proceeds to ACT2201. If the predetermined time has elapsed, the processor 201 determines YES in ACT2204, ends this touch panel input detection subroutine, and proceeds to ACT221.

[0078] By executing such a touch panel input detection subroutine before ACT211, if an input operation on the touch panel 211 is detected, the unauthorized detection process will not be performed for a predetermined time. In other words, for a predetermined time after the touch panel 211 is operated, the unauthorized notification of ACT217 will not be issued, that is, the unauthorized notification is suppressed. Originally, the fact that the touch panel 211 is operated means that the customer CS is in front of the settlement device 20, so there is no need to detect the customer, that is, a situation where an unauthorized notification is issued does not occur. However, since the operation of the touch panel 211 is not equal to settlement, there remains a possibility of unauthorized leaving without settlement. Therefore, here, after a predetermined time has elapsed, the suppression of the unauthorized notification is released. In this way, the processor 201 is an example of a notification suppression unit that suppresses the notification process of the unauthorized notification unit for a predetermined time when an input operation by a person on the settlement device 20 is detected.

[0079] As described above, when the processor 201 of the settlement device 20 to which the unauthorized detection device according to the second embodiment is applied detects an input operation on the touch panel 211 by the customer CS on the settlement device 20, the unauthorized notification process is suppressed for a predetermined time. In this way, according to the unauthorized detection device according to the second embodiment, unnecessary processing can be prevented from being performed when the touch panel 211 is being operated, and by providing a predetermined time, unauthorized actions can be prevented from being overlooked.

[0080] [Third Embodiment] Next, the third embodiment will be described. Regarding the same configuration and the same operation as those in the first embodiment, the same reference numerals as those in the first embodiment are used, and the description thereof is omitted.

[0081] The fraud detection system according to the first embodiment corresponded to a semi-self-checkout system including a registration device 10 and a settlement device 20. However, this embodiment corresponds to a full-self-checkout system that uses a self-POS terminal as a registration and settlement device that integrates both functions of the registration device and the settlement device. That is, in this embodiment, the self-POS terminal serves as a target device to be detected for fraud.

[0082] FIG. 16 is a perspective view showing the appearance of a self-POS terminal 40 to which the fraud detection device according to the third embodiment is applied, that is, in which the fraud detection device is incorporated. FIG. 17 is a block diagram showing the main configuration of the self-POS terminal 40.

[0083] As shown in FIG. 16, the self-POS terminal 40 includes a main body 401 installed on the floor surface and a weighing unit 402 installed beside the main body 401. A display pole 403 and a touch panel 404 are attached to the upper part of the main body 401. The main body 401 is provided with a basket stand 405 at the center of the side surface on the side opposite to the side where the weighing unit 402 is installed. The basket stand 405 is for placing a merchandise basket BK1 in which a customer CS who has come from the sales floor has put the purchased goods. The customer CS stands in front of the main body 401 in FIG. 16 so as to be able to see the screen of the touch panel 404 and performs operations. For this reason, when viewed from the customer CS, there is a basket stand 405 on the right side with the main body 401 in between, and a weighing unit 402 on the left side. In the following description, the side where the customer CS stands is defined as the front of the main body 401, the side where the weighing unit 402 is installed is defined as the left side of the main body 401, and the side where the basket stand 405 is provided is defined as the right side of the main body 401.

[0084] The weighing unit 402 has a structure in which a weighing pan 407 is provided on the upper part of the housing 406, and a bag holder 408 is attached above the weighing pan 407. The upper surface of the weighing pan 407 serves as a placement surface 409. The bag holder 408 includes a pair of holding arms 410, and the holding arms 410 hold merchandise bags BG such as a cash register bag and a so-called my bag that a customer CS brings in. The weighing unit 402 measures the weight of the merchandise placed in the cash register bag or my bag held by the holding arm 410 and placed on the placement surface 409. The weighing unit 402 including the bag holder 408 and the placement surface 409 corresponds to the mounting table LT of the first embodiment. Also, the cash register bag or my bag, which is the merchandise bag BG, corresponds to the merchandise cage BK2 in the first embodiment.

[0085] The display pole 403 includes a light emitting part 411 that selectively emits, for example, blue and red light at its tip. The display pole 403 displays the state of the self-POS terminal 40, such as standby, operation, call, error, occurrence of fraud, etc., according to the emission color and emission state of the light emitting part 411.

[0086] The touch panel 404 is composed of a display 412 for displaying various screens to the user who operates the self-POS terminal 40, and a touch sensor for detecting the user's touch input to the screen. In the self-POS terminal 40, the user is usually the customer CS.

[0087] On the front of the main body 401, a scanner 413, a card insertion slot 414, and a receipt issuing port 415 are arranged. Further, on the front of the main body 401, a coin insertion slot 416, a coin payout slot 417, a bill insertion slot 418, and a bill payout slot 419 are also formed. Also, a communication cable 420 extends from the right side surface of the main body 401 to the outside, and a reader / writer 421 for an electronic money medium is connected to the tip of the communication cable 420.

[0088] Above the self-POS terminal 40, a camera 30 is arranged. The shooting area PA of the camera 30 includes the periphery of the self-POS terminal 40. Specifically, the camera 30 is installed so that its captured image PI includes the image of the self-POS terminal 40 including the weighing unit 402 and the image of the customer CS operating the self-POS terminal 40.

[0089] As shown in FIG. 17, the self-POS terminal 40 includes a processor 422, a ROM 423, a RAM 424, an HDD 425, a LAN interface 426, an I / O device control unit 427, a deposit-withdrawal device 428, a printer 429, a card reader 430, and the aforementioned touch panel 404, display 412, and scanner 413. By connecting the processor 422 to the memory including the ROM 423 and the RAM 424, the computer of the self-POS terminal 40 is configured.

[0090] The processor 422 corresponds to the central part of the above computer. The processor 422 controls each part in order to realize various functions of the self-POS terminal 40 and the fraud detection device according to the present embodiment according to an operating system or a control program. The processor 422 may be, for example, a CPU, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 422 may be a combination of a plurality of these.

[0091] The ROM 423 and the RAM 424 correspond to the main memory part of the above computer. The ROM 423 stores an operating system or a control program. The RAM 424 stores data necessary for the processor 422 to execute processes for controlling each part. As one of such data, the RAM 424 stores registration information 4241 and a settlement mode flag 4242. The registration information 4241 includes details of purchased goods in a transaction, the total amount, etc. The settlement mode flag 4242 is the same as the settlement mode flag 2031 in the first embodiment. Also, the RAM 424 is used as a work area in which data is appropriately rewritten by the processor 422.

[0092] The HDD 425 stores various control programs. The control programs can include a fraud detection program for operating the self-POS terminal 40 as a fraud detection device according to the embodiment. Further, the HDD 425 stores a product master 4251 that stores product names, prices, etc. associated with product codes for each product sold in the store, and a transaction file 4252 that includes registration information related to transactions. The transaction file 4252 is a data file for storing data related to one business transaction processed by the self-POS terminal 40. The transaction file 4252 stores data such as, for example, a transaction number, purchased product data, total points, total amount, discount amount, settlement amount, etc. The transaction number is a series of numbers issued each time a business transaction is processed by the self-POS terminal 40. The purchased product data is record data created for each product bought and sold in the business transaction identified by the transaction number. Here, the purchased product data is composed of items such as a product code, product name, price, points, amount, etc. The points are the number of purchased points of the product identified by the product code. The amount is the amount for the number of purchased points. The transaction file 4252 can store a plurality of purchased product data. The total points is the total of the points of each purchased product data. The total amount is the total of the amounts of each purchased product data. The discount amount is the amount discounted from the total amount. The settlement amount is the amount obtained by subtracting the discount amount from the total amount. Note that the self-POS terminal 40 may be provided with a rewritable storage device such as an EEPROM or an SSD instead of or in addition to the HDD 425.

[0093] The LAN interface 426 is a communication device for communicating with the camera 30 via the LAN.

[0094] The I / O device control unit 427 controls the transmission of control information from the processor 422 to the weighing unit 402, touch panel 404, display 412, scanner 413, reader / writer 421, deposit / withdrawal device 428, printer 429, and card reader 430, and the transmission of data in the reverse direction from these weighing unit 402, touch panel 404, display 412, scanner 413, reader / writer 421, deposit / withdrawal device 428, printer 429, and card reader 430 to the processor 422.

[0095] The deposit / withdrawal device 428 sorts the coins inserted into the coin insertion slot 416 one by one to identify the denominations and stores them in the safe by denomination. Also, the deposit / withdrawal device 428 takes out the coins of the corresponding denomination from the safe based on, for example, the change data and pays them out through the coin payout slot 417. Further, the deposit / withdrawal device 428 sorts the banknotes inserted into the banknote insertion slot 418 one by one to identify the denominations and stores them in the safe by denomination. Also, the deposit / withdrawal device 428 takes out the banknotes of the corresponding denomination from the safe based on, for example, the change data and pays them out through the banknote payout slot 419.

[0096] The printer 429 prints receipt data representing the content of the commercial transaction on the receipt paper. The receipt paper with the receipt data printed thereon is discharged from the receipt issuing port 415 and cut by a cutter (not shown) to be issued as a receipt or receipt voucher.

[0097] The card reader 430 reads the card data recorded on a card medium such as a credit card or point card. The card reader 430 draws the card medium inserted into the card insertion slot 414 into the main body 401, reads the card data, and then discharges it from the card insertion slot 414.

[0098] Scanner 413 reads the code symbol from the product. Each product sold in the store is attached with a code symbol in which product ID etc. for identifying the product is encoded. The code symbol is, for example, a bar code. The code symbol may be, for example, a two-dimensional data code. Scanner 413 may be of a type that reads the code symbol by scanning a laser beam, or may be of a type that reads the code symbol from an image captured by an imaging device. Also, scanner 413 may be used to read the code symbol for code settlement.

[0099] In the self-POS terminal 40 having such a configuration and the fraud detection device applied to the self-POS terminal 40, it operates as follows.

[0100] FIG. 18 is a flowchart showing the main procedure of the information processing executed by the processor 422 of the self-POS terminal 40. The processor 422 executes this process based on a control program stored in the ROM 423 or the HDD 425, for example, in response to the power-on of the self-POS terminal 40 or a predetermined start operation on the touch panel 404 in the clerk mode by the clerk SP. Note that the procedure shown in FIG. 18 is an example. The procedure is not particularly limited as long as the same result can be obtained.

[0101] As ACT401, the processor 422 determines whether to start registration. Specifically, the processor 422 determines whether there is a start operation on the touch panel 404 by the customer CS. Alternatively, the LAN interface 426 may acquire a captured image from the corresponding camera 30 via the LAN and determine based on whether the customer CS is detected in the customer detection area ROI1. Also, if the self-POS terminal 40 is equipped with a human detection sensor such as an infrared sensor or an ultrasonic sensor, this determination can also be made based on the presence or absence of human detection by an image such as an infrared image or an ultrasonic image captured by the human detection sensor. The infrared sensor and the ultrasonic sensor are examples of image sensors. If registration is not started, the processor 422 determines NO in ACT401 and repeats this ACT401 again. Thus, the processor 422 waits for the start of registration. And when starting registration, the processor 422 determines YES in ACT401 and proceeds to ACT402.

[0102] As ACT402, the processor 422 executes a registration process. The customer CS takes out the purchased product from the product cage BK1 placed on the cage 405, has the code symbol of the product read by the scanner 413, and then puts it into a product bag BG such as a shopping bag or a my bag set in the weighing unit 402. Therefore, the processor 422 reads the code symbol from the product by the scanner 413, reads out the product information corresponding to the product code indicated by the read code symbol from the product master 4251, and registers it in the registration information 4241. When the registration of all purchased products is completed, the customer CS performs a predetermined input operation such as an operation of the "Proceed to Checkout" button for instructing checkout displayed on the display 412 of the touch panel 404. Thereby, the self-POS terminal 40 ends this registration process.

[0103] As ACT403, the processor 422 sets an accounting mode flag 4242 indicating that the accounting process is in progress in the RAM 424. For example, the processor 422 sets the value of the accounting mode flag 4242 stored in the RAM 424 to "1".

[0104] As ACT404, perform the settlement process. In this settlement process, the processor 422 causes a display such as a settlement guide to be displayed on the display 412 according to the registration information 4241, accepts settlement by the customer CS using the deposit / withdrawal device 428 or the card reader 430, and if the settlement is completed, issues a receipt using the printer 429.

[0105] After the settlement process ends, as ACT405, the processor 422 updates the transaction file 4252 according to the registration information 4241 for which the settlement has ended. That is, the processor 422 adds and records the log of this transaction to the transaction file 4252.

[0106] As ACT406, the processor 422 clears the settlement mode flag 4242 set in the RAM 424. For example, the processor 422 sets the value of the settlement mode flag 4242 stored in the RAM 424 to "0". After that, the processor 422 proceeds to ACT201.

[0107] The main procedure of the fraud detection process executed by the processor 422 of the self-POS terminal 40 is the same as that described with reference to FIG. 9 in the first embodiment. In the description with reference to FIG. 9, it is only necessary to read the settlement device 20 and the processor 201 as the self-POS terminal 40 and the processor 422, and the registration information as the registration information 4241 stored in the RAM 424, so the description is omitted here. In this embodiment, since the customer CS does not move, it goes without saying that the specified period can be short.

[0108] As described above, the processor 422 of the self-POS terminal 40 to which the fraud detection device according to the third embodiment is applied can operate in the same manner as the processor 201 of the settlement device 20 in the first embodiment. Therefore, according to the fraud detection device according to the third embodiment, even if the object to detect fraud is the self-POS terminal 40, the same effect as in the first embodiment can be achieved.

[0109] [Fourth Embodiment] Next, a fourth embodiment will be described. Regarding the configuration and operation similar to those of the first embodiment, the same reference numerals as those in the first embodiment are used, and the description thereof will be omitted.

[0110] In this embodiment, an example is provided in which a fraud detection device is provided separately from the settlement device 20, which is a target device for detecting fraud. FIG. 19 is a block configuration diagram of a semi-self checkout system to which a fraud detection system including the fraud detection device according to the fourth embodiment is applied. The checkout system includes a plurality of registration devices 10, a plurality of settlement devices 20, a plurality of cameras 30, and a server device 50 to which the fraud detection device according to the fourth embodiment is applied, which are connected via a LAN.

[0111] FIG. 20 is a block diagram showing a main part configuration of the server device 50 to which the fraud detection device according to the fourth embodiment is applied, that is, in which the fraud detection device is incorporated. As shown in FIG. 20, the self-POS terminal 40 includes a processor 501, a ROM 502, a RAM 503, an HDD 504, a LAN interface 505, an I / O device control unit 506, an input device 507, and an output device 508. A computer of the server device 50 is configured by connecting the processor 501, the ROM 502, and a memory including the RAM 503.

[0112] The processor 501 corresponds to the central part of the above computer. The processor 501 controls each part in order to realize various functions as the server device 50 and the fraud detection device according to this embodiment according to an operating system or a control program. The processor 501 may be, for example, a CPU, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 501 may be a combination of a plurality of these.

[0113] ROM 502 and RAM 503 correspond to the main memory part of the above computer. ROM 502 stores an operating system or a control program. RAM 503 stores data necessary for the processor 501 to execute processes for controlling each part. As one of such data, RAM 503 stores a flag table 5031. Also, RAM 503 is used as a work area where data is appropriately rewritten by the processor 501.

[0114] Figure 21 is a diagram showing the data configuration of the flag table 5031 included in the RAM 503 of the server device 50. As shown in Figure 21, the flag table 5031 stores an accounting mode flag and a registration device ID in association with an accounting device ID, which is a device number or the like for uniquely identifying each of the plurality of settlement devices 20. The accounting mode flag corresponds to the accounting mode flag 2031 in the first embodiment. That is, since the server device 50 individually executes fraud detection for each of the plurality of settlement devices 20, it stores an accounting mode flag for each settlement device 20. The registration device ID is a device number or the like for uniquely identifying the registration device 10 that is the source of the registration information for performing accounting by the settlement device 20 indicated by the accounting device ID.

[0115] The HDD 504 stores various control programs. The control program can include a fraud detection program for operating the server device 50 as a fraud detection device according to the embodiment. Also, the HDD 504 stores a device setting table 5041. Note that the server device 50 may be provided with a rewritable storage device such as an EEPROM or an SSD instead of or in addition to the HDD 504.

[0116] FIG. 22 is a diagram showing the data configuration of the device setting table 5041 provided in the HDD 504 of the server device 50. As shown in FIG. 22, the device setting table 5041 stores a camera ID and a specified period in association with the settlement device ID of each of a plurality of settlement devices 20. The camera ID is the device number or the like of the camera 30 corresponding to the settlement device 20. The specified period is a period set based on the distance from the registration device 10 to the settlement device 20, and includes a plurality of periods corresponding to each of the plurality of registration devices 10. That is, the device setting table 5041 stores a specified period associated with a registration device ID, which is a device number or the like for uniquely identifying each of the plurality of registration devices 10. The settlement device ID stored in this device setting table 5041 is read by the processor 501 when the server device 50 is activated and set in the flag table 5031 of the RAM 503.

[0117] The LAN interface 505 is a communication device for communicating with the registration device 10, the settlement device 20, and the camera 30 via the LAN.

[0118] The I / O device control unit 506 controls the transmission of control information from the processor 501 to the input device 507 and the output device 508, and the transmission of data in the reverse direction, from the input device 507 and the output device 508 to the processor 501.

[0119] The input device 507 is a user interface device such as a pointing device such as a keyboard or a mouse. The output device 508 is a user interface device such as a liquid crystal display. Note that these input device 507 and / or output device 508 may not be possessed by the server device 50. That is, the input device 507 and / or the output device 508 can be provided outside the server device 50 as a management terminal or the like connected to the LAN. In this case, the processor 501 can use the input device 507 and / or the output device 508 via the LAN by the LAN interface 505.

[0120] Figures 23 and 24 are sequence diagrams showing the outline of the operation of the fraud detection system. Figure 23 shows the case where there is no fraud, and Figure 24 shows the case where there is fraud.

[0121] As shown in Figure 23, when the registration process is performed in the registration device 10 (step S1), an accounting instruction is sent from the registration device 10 to the settlement device 20 (step S2). This accounting instruction includes, for example, registration information. When receiving this accounting instruction, the settlement device 20 executes an accounting process for accepting settlement by the customer CS (step S3).

[0122] In parallel with this, the registration device 10 sends a monitoring instruction to the server device 50 (step S4). This monitoring instruction includes, for example, a monitoring device ID for identifying the source registration device 10 and an accounting device ID for identifying the accounting device 20 to be monitored. The server device 50 that has received the monitoring instruction acquires a captured image from the camera 30 having a camera ID corresponding to the accounting device 20 indicated by the accounting device ID (step S5). The server device 50 executes fraud detection based on this captured image (step S6) and determines whether there is fraud (step S7). If there is no fraud, the server device 50 determines whether the settlement in the monitored accounting device 20 has ended (step S8). If the settlement has not ended yet, it repeats from step S5.

[0123] When the accounting process is completed, the settlement device 20 sends an end notification to the server device 50 (step S9). This end notification includes an accounting device ID for identifying the settlement device 20. When receiving the end notification, the server device 50 can determine which settlement device 20 has completed the accounting process based on the accounting device ID. So, when it determines the end in step S8, it ends the fraud detection for the settlement device 20.

[0124] On the other hand, in the case where there is a fraudulent act of leaving the store with unaccounted goods, as shown in Figure 24, the server device 50 detects that there is fraud (step S7) and performs a fraud notification by the output device 508 (step S10).

[0125] Next, a specific example of the operation of the fraud detection device applied to the server device 50 will be described. FIG. 25 is a flowchart showing the main procedure of the information processing related to settlement executed by the processor 501 of the server device 50. The processor 501 executes this process based on a control program stored in the ROM 502 or the HDD 504, for example, upon receiving power supply to the server device 50. Note that the procedure shown in FIG. 25 is an example. The procedure is not particularly limited as long as the same result can be obtained.

[0126] As ACT501, the processor 501 generates a flag table 5031. Specifically, the processor 501 reads the settlement device ID from the device setting table 5041 stored in the HDD 504, and creates a flag table 5031 having the read settlement device IDs in the RAM 503.

[0127] As ACT502, the processor 501 determines whether there is a monitoring instruction. Specifically, the processor 501 determines whether it has received a monitoring instruction from any of the registration devices 10 of the settlement device IDs set in the flag table 5031 via the LAN by the LAN interface 505. If there is a monitoring instruction, the processor 501 determines YES in ACT5202 and proceeds to ACT504. If there is no monitoring instruction, the processor 501 determines NO in ACT502 and proceeds to ACT503.

[0128] As ACT503, the processor 501 determines whether it has received an end notification. Specifically, the processor 501 determines whether it has received an end notification from any of the registration devices 10 of the settlement device IDs set in the flag table 5031 via the LAN by the LAN interface 505. If it has received an end notification, the processor 501 determines YES in ACT5203 and proceeds to ACT505. If it has not received an end notification, the processor 501 determines NO in ACT503 and proceeds to ACT502.

[0129] In this way, the processor 501 waits for a monitoring instruction or an end notification.

[0130] If there is a monitoring instruction, as ACT504, the processor 501 sets the corresponding settlement mode flag. Specifically, based on the settlement device ID included in the monitoring instruction, the processor 501 sets the settlement mode flag associated with the corresponding settlement device ID in the flag table 5031. For example, the processor 501 sets the value of its settlement mode flag to "1".

[0131] As ACT505, the processor 501 stores the specified period. Specifically, the processor 501 reads the specified period from the device setting table 5041 of the HDD504 based on the registration device ID and the settlement device ID included in the monitoring instruction, and stores the read specified period associated with the corresponding settlement device ID in the flag table 5031. Then, the processor 501 proceeds to ACT502.

[0132] Also, if an end notification is received, as ACT506, the processor 501 clears the corresponding settlement mode flag. Specifically, based on the settlement device ID included in the end notification, the processor 501 clears the settlement mode flag associated with the corresponding settlement device ID in the flag table 5031. For example, the processor 501 sets the value of its settlement mode flag to "0".

[0133] As ACT507, the processor 501 deletes the corresponding specified period. Specifically, the processor 501 deletes the specified period associated with the corresponding settlement device ID in the flag table 5031. Then, the processor 501 proceeds to ACT502.

[0134] FIG. 26 is a flowchart showing the main procedure of the fraud detection process executed by the processor 501 of the server device 50. This fraud detection process is a process as a fraud detection device by the processor 501. The processor 501 starts this fraud detection process, for example, together with the start of the information processing related to the settlement shown in FIG. 9. That is, the processor 501 executes the information processing related to the settlement and the fraud detection process in parallel.

[0135] As ACT511, the processor 501 determines whether the settlement mode flag is set in the flag table 5031 of the RAM 503. That is, the processor 501 determines whether there is a settlement mode flag with a value of "1" in the flag table 5031. If none of the settlement mode flags are set, the processor 501 determines NO in ACT511 and repeats this ACT511 again. In this way, the processor 501 waits for any of the settlement mode flags in the flag table 5031 to be set, that is, waits for the start of the settlement process. And if any of the settlement mode flags are set, the processor 501 determines YES in ACT511 and proceeds to ACT512. Note that the processing below ACT512 is executed for each of the settlement devices 20 of the settlement device ID for which the settlement mode flag is set. In other words, the processor 501 executes the processing below ACT512 in parallel for each of the settlement devices 20 of the settlement device ID for which the settlement mode flag is set.

[0136] As ACT512, the processor 501 identifies the camera 30 corresponding to the settlement device 20 of the target settlement device ID. Specifically, the processor 501 reads out the camera ID stored in association with the target settlement device ID from the device setting table 5041 of the HDD 504.

[0137] As ACT513, the processor 501 acquires a photographed image from the identified camera 30. Specifically, the processor 501 acquires the photographed image of the camera 30 indicated by the camera ID from among the plurality of cameras 30 connected to the LAN through the LAN interface 505.

[0138] As ACT514, the processor 501 determines whether a customer CS has been detected based on the acquired captured image. That is, the processor 501 determines whether there is a person in front of the front surface of the target payment device 20. The processor 501 is an example of a person detection unit that detects a person within a first area around the payment device 20 based on the captured image PI captured by the camera 30. If the customer CS is detected, the processor 501 determines YES in ACT514 and proceeds to ACT516. If the customer CS is not detected, the processor 501 determines NO in ACT514 and proceeds to ACT515.

[0139] As ACT515, the processor 501 determines whether a product has been detected based on the acquired captured image. The processor 501 is an example of a product detection unit that detects at least one of a product and a container that houses the product within a second area around the payment device 20. If the product is detected, the processor 501 determines YES in ACT515 and proceeds to ACT511. If the product is not detected, the processor 501 determines NO in ACT515 and proceeds to ACT517.

[0140] As ACT516, the processor 501 determines whether the customer CS is facing the target payment device 20. If the customer CS is facing the payment device 20, the processor 501 determines YES in ACT516 and proceeds to ACT511. If the customer CS is not facing the payment device 20, the processor 501 determines NO in ACT516 and proceeds to ACT515.

[0141] As ACT517, the processor 501 determines whether a specified period has elapsed since the payment mode flag was set. If the specified period has not elapsed, the processor 501 determines NO in ACT517 and proceeds to ACT511. If the specified period has elapsed, the processor 501 determines YES in ACT517 and proceeds to ACT518.

[0142] During the loops of ACT511 to ACT517, when the settlement process ends and the settlement mode flag is cleared, it will be determined as NO in ACT511, so the processes of ACT512 to ACT517 will end. In other words, the processor 501 executes the operations of the person detection unit, the product detection unit, and the fraud notification unit until the settlement process in the settlement device 20 ends.

[0143] As ACT518, the processor 501 issues a fraud notification assuming that there is a high possibility of an illegal act of taking out a registered product outside the store without settlement. The processor 501 is an example of a fraud notification unit that issues a fraud notification when, while the customer CS is performing a settlement process on a product, the person detection unit does not detect a person and the product detection unit does not detect either the product or the container. This fraud notification may be performed, for example, by the output device 508, or the fraud information indicating a high possibility of fraud may be transmitted to the registration device 10, which is the transmission source of the monitoring instruction, via the LAN through the LAN interface 505. Also, as described in the first embodiment, instead of protecting the registered customer's image from being deleted or saving it as a separate file in the registration device 10, such operations may be performed in the server device 50. After that, the processor 501 proceeds to ACT511.

[0144] As described above, the processor 501 of the server device 50 to which the fraud detection device according to the fourth embodiment is applied executes the operations of person detection, product detection, and fraud notification until the settlement process in the settlement device 20 ends. Therefore, according to the fraud detection device according to the fourth embodiment, fraud in each of the plurality of settlement devices 20 connected via the LAN can be detected by one server device 50.

[0145] In addition, also in the fourth embodiment, as in the second embodiment, when an input operation on the touch panel 211 of the settlement device 20 is detected, the processor 501 may not perform the fraud detection process for a predetermined time. This can be easily realized by notifying the server device 50 that there is input from the settlement device 20 in response to the input operation on the touch panel 211, and the processor 501 determines whether it has received the notification.

[0146] Furthermore, it goes without saying that the fourth embodiment is also applicable to a full self-checkout system using the self-POS terminal 40 as described in the third embodiment. That is, the processor 501 of the server device 50 to which the fraud detection device according to the fourth embodiment is applied executes operations of person detection, commodity detection, and fraud notification from the end of the registration process at the self-POS terminal 40 until the end of the settlement process. Therefore, according to the fraud detection device according to the fourth embodiment, fraud in each of the plurality of self-POS terminals 40 connected via the LAN can be detected by one server device 50.

[0147] [Fifth Embodiment] Next, the fifth embodiment will be described. Regarding the same configuration and the same operations as those in the first embodiment, the same reference numerals as those in the first embodiment are used, and the description thereof is omitted.

[0148] FIG. 27 is a block configuration diagram of a semi-self-checkout system to which the fraud detection system according to the fifth embodiment is applied. In the first embodiment, the settlement device 20 and the camera 30 had a one-to-one relationship, but in this embodiment, as shown in FIG. 27, it is an example in the case where a plurality of cameras 30, for example, two cameras 30, photograph one settlement device 20. That is, the settlement device 20 and the camera 30 have a two-to-one relationship.

[0149] FIG. 28 is a schematic diagram for explaining a detection area in a captured image PI of the camera 30. Since the camera 30 is fixedly installed, the positional relationship between the camera 30, the two settlement devices 20, and the two mounting tables LT is unchanged. Therefore, as shown in FIG. 28, the captured image PI always includes a settlement device image 20I that is an image of each settlement device 20 and a mounting table image LTI that is an image of each mounting table LT. In addition, a part of each of a counter table image CTI that is an image of the counter table CT, a registration device image 10I that is an image of the registration device 10, and a soccer table image STI that is an image of the soccer table ST is also included. And, optionally, the captured image PI also includes a customer image CSI that is an image of the customer CS and a store clerk image SPI that is an image of the store clerk SP. The processor 201 sets a customer detection area ROI1 and a product detection area ROI2 for each settlement device 20 and mounting table LT with respect to the captured image PI, cuts out an image of each detection area, and detects a customer or a product from the cut-out image. The two product detection areas ROI2 may also be an area such that a product cart image SCI that is an image of the product cart SC in the captured image PI also enters the product detection area ROI2. In this way, the processor 201 cuts out an image of a first area and an image of a second area of one settlement device 20 that the fraud detection device should detect fraud from the captured image PI captured by one camera 30 that captures the periphery of the plurality of settlement devices 20.

[0150] In the present embodiment, the information processing and fraud detection processing related to settlement executed by the processor 201 of each settlement device 20 are the same as those in the first embodiment described with reference to FIGS. 8 and 9.

[0151] As described above, the processor 20 of the settlement device 20 to which the fraud detection device according to the fifth embodiment is applied cuts out an image of a first area of one settlement device 20 that the processor 20 should detect fraud from the captured image PI captured by one camera 30 that captures the periphery of the plurality of settlement devices 20, and detects a person. Therefore, according to the fraud detection device according to the fifth embodiment, similar to the fraud detection device according to the first embodiment, it is possible to detect a person by narrowing down to an image of a specific area from the captured image PI captured by the camera 30, so that high-speed processing can be achieved. In addition, since one camera 30 can be shared by a plurality of fraud detection devices, the cost can be reduced as a fraud detection system.

[0152] In addition, the processor 201 of the settlement device 20 to which the fraud detection device according to the fifth embodiment is applied cuts out an image of a second area of one settlement device 20 that the processor 201 should detect fraud from the captured image PI captured by one camera 30 that captures the peripheries of the plurality of settlement devices 20, and detects at least one of a product, or the product cage BK2 and the product cart SC that are containers. Therefore, according to the fraud detection device according to the fifth embodiment, similar to the fraud detection device according to the first embodiment, it is possible to detect a product, the product cage BK2, and the product cart SC by narrowing down to an image of a specific area from the captured image PI captured by the camera 30, so that high-speed processing can be achieved. In addition, since one camera 30 can be shared by a plurality of fraud detection devices, the cost can be reduced as a fraud detection system.

[0153] It should be noted that this fifth embodiment is of course applicable to the second to fourth embodiments as well.

[0154] As described above, the embodiments of the fraud detection system and the fraud detection device have been described, but such embodiments are not limited thereto. For example, in any of the embodiments, as the transmission destination of the fraud notification, specific monitoring terminals such as a security guard terminal used by a security guard or an attendant terminal used by a store clerk called an attendant may be added.

[0155] Also, although the correspondence between the settlement device 20 or the self-POS terminal 40 and the camera 30 has been described by taking the examples of one-to-one or two-to-one, it may further be in a many-to-one relationship, or conversely, it may be in a one-to-many relationship. For example, the customer CS, the merchandise, the merchandise basket BK2, and the shopping cart SC may be photographed by separate cameras 30. Further, the camera 30 is not limited to being installed above the device. For example, the cameras 30 for photographing the merchandise, the merchandise basket BK2, and the shopping cart SC are arranged above the device, and the camera 30 for photographing the customer CS is arranged to photograph from the touch panel 211, 404 side or the lateral direction of the device. When the camera 30 is arranged on the touch panel 211, 404 side or the lateral direction of the device in this way, instead of determining the "orientation of the face and body" in the customer image CSI as the "orientation of the person", the "orientation of the face" or the "orientation of the body" may be used. Also, for the camera 30 arranged above, instead of installing it as a dedicated camera, if a security camera is already installed on the ceiling in advance, it may be used.

[0156] In addition, in the detection of the customer CS and the detection of the product, a detection method that does not use the camera 30 can also be adopted. For example, in the detection of the customer CS, an infrared image or an ultrasonic image captured using an infrared sensor or an ultrasonic sensor can be used. In this case, the sensor is arranged facing the front of the device, and the detection area is set as the customer detection area ROI1. From the state where there is no person in front of the device, it is possible to detect the customer CS in the customer detection area ROI1 around the device based on whether the distance changes by more than a preset threshold. Similarly, in the detection of the product, the infrared sensor or the ultrasonic sensor can be arranged upward from the placement surface of the placement table LT or facing the front of the placement table LT, and the detection area can be set as the product detection area ROI2. Alternatively, in the self-POS terminal 40, since it is equipped with the weighing unit 402, using the bag holder 408 and the placement surface 409 of this weighing unit 402 as the product detection area ROI2, it is possible to detect the product based on whether there is a change of more than the threshold from the weight value when the product is not placed on the weighing unit 402. Similarly, in the settlement device 20, by enabling the measurement of the weight of the object placed on the placement table LT, the product can be detected.

[0157] Also, in the flowchart of FIG. 9, the order of ACT213, ACT2150, and ACT214 may be reversed. That is, the detection of the product may be performed first and then the detection of the customer CS. Similarly, in the flowchart of FIG. 8, the order of ACT202 and ACT203 may also be reversed. Alternatively, ACT202 and ACT203 may be performed in parallel. Thus, as long as there is no conflict with the preceding or subsequent processes, the order of the processes may be changed or a plurality of processes may be performed in parallel.

[0158] In addition, in the above-described embodiment, the control program executed by the processor 201 of the settlement device 20, the processor 422 of the self-POS terminal 40, or the processor 501 of the server device 50 may be configured to be recorded and provided on a computer-readable recording medium such as a CD-ROM. Further, the control program may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0159] In addition, although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and are included in the invention described in the claims and the equivalent scope thereof.

[0160] [Appendix] From the above specific embodiments, an invention having the following configuration can be extracted. [1] A person detection unit that detects a person in a first area around the target device based on an image capturing the target device that is a target for detecting fraud, A product detection unit that detects at least one of a product and a container that houses the product in a second area around the target device based on the image, A fraud notification unit that notifies fraud when the person detection unit does not detect the person and the product detection unit does not detect any of the product and the container while an accounting process for the product is being executed. A fraud detection device comprising: [2] A person orientation detection unit that detects the orientation of the person detected by the person detection unit, A detection invalidation unit that invalidates the detection of the person detection unit when the orientation of the person detected by the person orientation detection unit is not in the direction of the target device. The fraud detection device according to [1], further comprising: [3] An input detection unit that detects an input operation by the person on the target device, a notification suppression unit that suppresses the notification process of the false notification unit for a predetermined time when the input detection unit detects the input operation, The fraud detection device according to [1] or [2], further comprising: [4] The second area is an area including a placement unit on which at least one of the product and the container is placed when the settlement process is executed in the target device. The fraud detection device according to [1]. [5] The container includes at least one of a product cage that houses the product and a product cart that houses the product cage. [5] The second area is an area including a placement unit on which the product cage is placed and a placement unit around the placement unit where the product cart is placed when the settlement process is executed in the target device. The fraud detection device according to [1]. [6] The person detection unit detects the person based on a captured image of the first area captured by the camera. The fraud detection device according to [1]. [7] The person detection unit cuts out an image of the first area from the captured image including the first area and the second area captured by the camera, and detects the person. The fraud detection device according to [6]. [8] The person detection unit cuts out an image of the first area of one target device that the fraud detection device should detect fraud from the captured image captured by the camera that captures the periphery of a plurality of the target devices, and detects the person. The fraud detection device according to [6]. [9] The product detection unit detects at least one of the product and the container based on a captured image of the second area captured by the camera. The fraud detection device according to [1].

[10] The product detection unit cuts out an image of the second area from the captured image including the first area and the second area captured by the camera, and detects at least one of the product and the container. The fraud detection device according to [9].

[11] The commodity detection unit cuts out an image of the second area of one target device, which the fraud detection device should detect fraud for, from the captured image taken by the camera that captures the periphery of the plurality of target devices, and detects the commodity or the container. The fraud detection device according to [9].

[12] The target device is a settlement device that performs settlement processing for a purchased commodity registered by a registration device. The fraud detection device is the fraud detection device according to [1], which is incorporated into the settlement device.

[13] The target device is a settlement device that performs settlement processing for a purchased commodity registered by a registration device. The fraud detection device includes a server device connected to the settlement device, which executes the operations of the person detection unit, the commodity detection unit, and the fraud notification unit until the settlement processing in the settlement device is completed. The fraud detection device according to [1].

[14] The target device is a registration settlement device that performs a registration process for registering a purchased commodity and a settlement process for the purchased commodity. The fraud detection device is the fraud detection device according to [1], which is incorporated into the registration settlement device.

[15] The target device is a registration settlement device that performs a registration process for registering a purchased commodity and a settlement process for the purchased commodity as the specified process. The fraud detection device includes a server device connected to the registration settlement device, which executes the operations of the person detection unit, the commodity detection unit, and the fraud notification unit from the end of the registration process in the registration settlement device until the end of the settlement process. The fraud detection device according to [1].

[16] A plurality of target devices that are targets for detecting fraud, One image sensor that captures images of the plurality of target devices, A fraud detection device connected to the image sensor, and is provided with, The fraud detection device is a person detection unit that detects a person in the first area around one of the plurality of target devices based on the image of the image sensor that captures the plurality of target devices. Based on the image of the image sensor that captures the plurality of target devices, a product detection unit that detects at least one of a product and a container that houses the product within a second area around the one target device among the plurality of target devices; When the person detection unit does not detect the person and the product detection unit does not detect either the product or the container in a state where the one target device is performing a settlement process for the product, a fraud notification unit that notifies of fraud; A fraud detection system comprising:

[17] A computer including a processor and a memory, a person detection unit that detects a person within a first area around the target device based on an image that captures the target device that is a target for detecting fraud; a product detection unit that detects at least one of a product and a container that houses the product within a second area around the target device based on the image; a fraud notification unit that notifies of fraud when the person detection unit does not detect the person and the product detection unit does not detect either the product or the container in a state where a settlement process for the product is being performed; A fraud detection program for causing the computer to function as such.

[18] Comprising a processor and a memory, wherein the processor detects a person within a first area around the target device based on an image that captures the target device that is a target for detecting fraud, detects at least one of a product and a container that houses the product within a second area around the target device based on the image, and notifies of fraud when the person is not detected and neither the product nor the container is detected in a state where a settlement process for the product is being performed. A fraud detection device configured as described above.

Explanation of Reference Numerals

[0161] 10... Registration device, 10I... Registration device image, 20... Settlement device, 20I... Settlement device image, 30... Camera, 40... Self-POS terminal, 50... Server device, 201,422,501... Processor, 202,423,502... ROM, 203,424,503... RAM, 2031,4242... Settlement mode flag, 204,425,504... HDD, 205,426,505... LAN interface, 206,427,506... I / O device control unit, 207,428... Deposit / withdrawal device, 208,413... Scanner, 209,429... Printer, 210,430... Card reader, 211,404... Touch panel, 216,412... Display, 225,411... Light-emitting unit, 402... Scale unit, 405... Basket, 407... Scale pan, 408... Bag holder, 409... Placing surface, 410... Holding arm, 421... Reader / writer, 4241... Registration information, 4251... Product master, 4252... Transaction file, 5031... Flag table, 5041... Device setting table, 507... Input device, 508... Output device, BG... Product bag, BK1,BK2... Product basket, CA... Checkout area, CS... Customer, CSI... Customer image, CT... Counter, CTI... Counter image, LT... Placing table, LTI... Placing table image, PA... Shooting area, PI... Shooting image, ROI1... Customer detection area, ROI2... Product detection area, SC... Shopping cart, SCI... Shopping cart image, SP... Store clerk, SPI... Store clerk image, ST... Soccer table, STI... Soccer table image.

Claims

1. A person detection unit that detects a person within a first area around the target device based on an image capturing the target device that is a target for detecting fraud; A product detection unit that detects at least one of a product and a container that houses the product within a second area around the target device based on the image; A fraud notification unit that notifies of fraud when the person detection unit does not detect the person and the product detection unit does not detect either the product or the container while the settlement process is being executed for the product; A fraud detection device comprising the above.

2. A person orientation detection unit that detects the orientation of the person detected by the person detection unit; A detection disabling unit that disables the detection by the person detection unit when the orientation of the person detected by the person orientation detection unit is not in the direction of the target device; The fraud detection device according to claim 1, further comprising the above.

3. An input detection unit that detects an input operation on the target device by the person; A notification suppression unit that suppresses the notification process of the fraud notification unit for a predetermined time when the input detection unit detects the input operation; The fraud detection device according to claim 1 or 2, further comprising the above.

4. The fraud detection device according to claim 1, wherein the second area is an area including a placement portion on which at least one of the product and the container is placed when the settlement process is executed on the target device.

5. The container includes at least one of a product cage that houses the product and a product cart that houses the product cage, The fraud detection device according to claim 1, wherein the second area is an area including a placement portion on which the product cage is placed and a placement portion around the placement portion where the product cart is placed when the settlement process is executed on the target device.

6. A plurality of target devices that are targets for detecting fraud; One image sensor that captures images of the plurality of target devices; A fraud detection device connected to the image sensor; Comprising, The fraud detection device is, A person detection unit that detects a person within a first area around one of the plurality of target devices based on the image of the image sensor capturing the plurality of target devices; A product detection unit that detects at least one of a product and a container that houses the product within a second area around the one of the plurality of target devices based on the image of the image sensor capturing the plurality of target devices; When the single target device is performing the settlement process for the product, if the person detection unit does not detect the person and the product detection unit does not detect either the product or the container, a fraud notification unit that notifies of fraud; A fraud detection system comprising the same.

7. A computer comprising a processor and a memory, A person detection unit that detects a person in a first area around the target device based on an image capturing the target device that is the target of fraud detection, A product detection unit that detects at least one of a product and a container that houses the product in a second area around the target device based on the image, A fraud notification unit that notifies of fraud when the person detection unit does not detect the person and the product detection unit does not detect either the product or the container while the settlement process for the product is being executed, A fraud detection program for causing the computer to function as such.

Images