Information processing system, program, and monitoring method

Abstract

To provide a system or the like capable of flexibly responding to the needs of taking measures against fraudulent behavior based on the scale of damage caused by the fraudulent behavior and maliciousness of the fraudulent behavior.SOLUTION: An information processing system comprises: a fraudulent behavior detection unit for detecting fraudulent behavior of a customer associated with commodity purchase at a store; a determination unit for determining whether or not an amount of fraudulent behavior by the customer as detected by the fraudulent behavior detection unit meets a predetermined standard; and a first output unit configured to output, to a store side terminal, a first alert indicating occurrence of fraudulent behavior when the amount of fraudulent behavior by the customer is determined to meet the standard by the determination unit.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information processing system, a program, and a monitoring method. [Background technology]

[0002] Patent Document 1 discloses a fraud detection system that sends an alert to a store clerk terminal when it detects fraudulent behavior or an operational error by a person. This fraud detection system evaluates a person's product purchasing behavior based on the number of times the person staying in a store acquires products sold in the store and the number of products they have registered for purchase. The fraud detection system then detects fraudulent behavior or an operational error by the person based on the evaluation result of the person's product purchasing behavior. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-7363 Summary of the Invention [Problem to be solved by the invention]

[0004] When a store detects fraudulent behavior by a customer, it will take action such as contacting the police, etc. However, as these actions take a very long time, there was a need from stores to take action based on the scale of damage caused by the fraudulent behavior and the maliciousness of the fraudulent behavior. The present invention aims to provide a system etc. that can flexibly respond to the need to take measures in accordance with the scale of damage caused by fraudulent behavior, the maliciousness of the fraudulent behavior, etc. [Means for solving the problem]

[0005] The present invention, which was completed with the above objective in mind, is an information processing system comprising a detection unit that detects fraudulent behavior by customers in relation to the purchase of products at a store, a determination unit that determines whether the amount of fraudulent behavior by the customer detected by the detection unit satisfies a predetermined standard, and an output unit that outputs a first alert to a store terminal indicating that fraudulent behavior has occurred when the determination unit determines that the amount of fraudulent behavior by the customer satisfies the standard. Here, the judgment unit may be characterized in that if the fraudulent behavior of the customer detected by the detection unit is a predetermined number of times or more, it judges that the criterion is met, and if the fraudulent behavior of the customer detected by the detection unit has not reached the predetermined number of times, it judges that the criterion is not met. The determination unit may also be characterized in that it determines whether the amount of fraudulent behavior, including fraudulent behavior during past settlement processes, of a customer for whom fraudulent behavior has been detected by the detection unit satisfies the criterion. Furthermore, when the detection unit detects fraudulent behavior of a customer, the detection unit may obtain information regarding the type of fraudulent behavior, and the determination unit may determine, for each type of fraudulent behavior, whether the amount of fraudulent behavior of the customer satisfies the criterion. The judgment unit may also be characterized in that it determines that the criterion is met if the fraudulent behavior of the customer detected by the detection unit is the second or subsequent fraudulent behavior, while it determines that the criterion is met if the fraudulent behavior of the customer detected by the detection unit is of a predetermined type, even if the fraudulent behavior of the customer is not the second or subsequent fraudulent behavior. The store may further include a second detection unit for detecting customers entering the store, and a second output unit for, when a customer with a history of a predetermined type of fraudulent behavior is detected by the second detection unit, outputting a second alert to the store terminal at the time the customer is detected, indicating that a customer with that history has been detected. The device may further include a memory unit that stores count information, which is information regarding the number of times fraudulent behavior has been committed, for each customer and for each type of fraudulent behavior, and the determination unit may determine, based on the count information, whether the amount of fraudulent behavior of the customer detected by the detection unit satisfies the criterion for each type of fraudulent behavior. The store may further include a memory unit that stores count information for each customer, which is information regarding the number of times fraudulent behavior has been committed, and the determination unit determines whether the amount of fraudulent behavior of the customer detected by the detection unit satisfies the standard based on the count information, and the memory unit may initialize the count information when the customer's behavior at the store satisfies predetermined conditions. The output unit may also output the first alert to a customer terminal used by a customer, in addition to the store terminal. The output unit may be configured to output, in addition to the first alert, information indicating the severity of the fraudulent behavior of the customer to the store terminal. From another perspective, the present invention is a program that enables a computer to perform the following functions: detect fraudulent behavior by customers in relation to the purchase of products at a store; determine whether the amount of detected fraudulent behavior by the customer satisfies a predetermined standard; and, if the amount of fraudulent behavior by the customer satisfies the standard, output a first alert to a store terminal indicating that fraudulent behavior has occurred. From another perspective, the present invention is a monitoring method in a monitoring system, which detects fraudulent behavior by a customer in relation to the purchase of goods at a store, determines whether the amount of the detected fraudulent behavior by the customer satisfies a predetermined standard, and if the amount of fraudulent behavior by the customer satisfies the standard, outputs a first alert to a store terminal indicating that fraudulent behavior has occurred. [Effects of the Invention]

[0006] According to the present invention, it is possible to provide a system etc. that can flexibly respond to the need to take measures in accordance with the scale of damage caused by fraudulent behavior, the maliciousness of the fraudulent behavior, etc. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of the configuration of a store in which a surveillance system to which the present embodiment is applied is used; [Figure 2] 1 is a diagram illustrating an example of the overall configuration of a monitoring system to which the present embodiment is applied. [Figure 3] 1 is a diagram illustrating an example of a schematic configuration of a transaction device to which the present embodiment is applied. [Figure 4] 1 is a block diagram showing an example of the functional configuration of a transaction device to which the present embodiment is applied. [Figure 5] FIG. 2 illustrates an example of a schematic configuration of a management device. [Figure 6] 10 is a diagram showing an example of the relationship between whether or not a first alert is output by a first output unit and the type and amount of fraudulent behavior detected by a fraudulent behavior detection unit. FIG. [Figure 7] 10 is a flowchart illustrating an example of a monitoring process performed by a control unit. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. <Embodiment 1> (Overall configuration of monitoring system 1) FIG. 1 is a diagram showing an example of the configuration of a store S in which a surveillance system 1 to which this embodiment is applied is used. FIG. 2 is a diagram showing an example of the overall configuration of a monitoring system 1 to which this embodiment is applied. The monitoring system 1 is an example of an information processing system, and is used to monitor the behavior of customers in a store and to output an alert if a customer engages in fraudulent behavior. The store S in which the surveillance system 1 is used is not particularly limited, but examples include convenience stores, supermarkets, drug stores, etc. Typically, the store S has a sales floor where products are displayed and customers shop, and a back office used by staff working at the store S. FIG. 1 shows only the sales floor of the store S. In the description of this embodiment, the sales floor of the store S may be simply referred to as the store S.

[0009] A store S is provided with a plurality of product shelves 2. Products are displayed on each of the product shelves 2. The store S is also provided with a checkout device 30. In this example, the store S is provided with a plurality of checkout devices 30. A customer who enters the store S through the entrance 5 passes through the aisles formed between the shelves 2, selects products displayed on the shelves 2, and places them in a shopping cart 300 (see FIG. 3, which will be described later).The customer then uses the accounting device 30 provided in the accounting area 3 to carry out payment for the products placed in the shopping cart 300.

[0010] The monitoring system 1 includes a store terminal 10, an image capturing device 20, an accounting device 30, and a management device 100. The store terminal 10, the image capturing device 20, the accounting device 30, and the management device 100 are connected via a network 90. The network 90 is not particularly limited as long as it is a communication network used for data communication between devices, and can be, for example, a local area network (LAN). The communication line used for data communication can be either wired or wireless, or a combination of both. Examples of wireless LANs include Wi-Fi (registered trademark) and Bluetooth (registered trademark). In this embodiment, the monitoring system 1 or the management device 100 is an example of an information processing system.

[0011] (Store terminal 10) The store terminal 10 is a device used by a person who monitors fraudulent behavior by customers in the store S. The store terminal 10 is a device that displays a first alert output from the management device 100, indicating that fraudulent behavior by a customer has occurred in relation to the purchase of a product in the store S. In the following description, fraudulent behavior by a customer in relation to the purchase of a product in the store S may be simply referred to as fraudulent behavior. Fraudulent behavior will be described in detail later. Examples of users of the store terminal 10 include staff working at the store S, such as employees and the store manager, and security guards who crack down on fraudulent behavior at the store S. In the following description, users of the store terminal 10 may be referred to as store clerks.

[0012] The store terminal 10 has a control unit 11 including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The ROM stores a control program executed by the CPU. The CPU reads the control program stored in the ROM and executes the control program using the RAM as a working area. When the control program is executed by the CPU, each unit of the store terminal 10 is controlled. The store terminal 10 also includes a display unit 12 that displays various information to the store clerk. Examples of the display unit 12 include a liquid crystal display and an organic EL (Electro Luminescence) display.

[0013] The store terminal 10 can be exemplified as a computer operated by a store clerk of the store S. The store terminal 10 may be a notebook PC, a tablet PC, a tablet terminal, a personal digital assistant (PDA), or a multi-function mobile phone (a so-called "smartphone"). In this example, the store terminal 10 is located in a sales floor of the store S. The store terminal 10 may also be located in a back office of the store S. Furthermore, if the store terminal 10 is a portable device such as a mobile information terminal or a multi-function mobile phone, the store clerk may carry the store terminal 10.

[0014] (photography device 20) The photographing device 20 photographs customers who enter the store S and acquires photographed images. The photographed images acquired by the photographing device 20 may be moving images or still images acquired at predetermined time intervals. The photographing device 20 then outputs the acquired photographed images to the management device 100. The photographing device 20 may be, for example, a small camera such as a CCD camera.

[0015] In this embodiment, multiple image capturing devices 20 are installed in store S. Specifically, the image capturing devices 20 include an entrance camera 20a that captures images of customers entering store S through the entrance / exit 5. The image capturing devices 20 also include an exit camera 20b that captures images of customers leaving store S through the entrance / exit 5. The image capturing devices 20 also include an in-store camera 20c that captures images of customers moving around store S and selecting products from the shelves 2. The image capturing devices 20 also include an accounting camera 20d that captures images of customers near checkout devices 30. In this example, multiple in-store cameras 20c are installed in store S to avoid creating blind spots so that customers in store S can always be captured. In this example, multiple accounting cameras 20d are installed corresponding to each checkout device 30. The accounting cameras 20d may capture images of customers waiting in line around the checkout device 30 in addition to customers paying for products at the checkout device 30. Note that FIG. 2 omits the multiple in-store cameras 20c and checkout camera 20d and shows only one.

[0016] (Accounting device 30) The checkout device 30 is a self-checkout that is operated by the customer themselves. In addition, the customer uses the checkout device 30 to register products and pay the purchase price for the registered products. In the description of this embodiment, the processes performed by the checkout device 30, such as registering products and paying the purchase price, may be collectively referred to as the product settlement process. The checkout device 30 is an example of a settlement terminal that performs the product settlement process. In this example, multiple checkout devices 30 are installed within the store S. In FIG. 2, multiple accounting devices 30 are omitted and only one is shown.

[0017] FIG. 3 is a diagram showing an example of the schematic configuration of a checkout device 30 to which this embodiment is applied. FIG. 4 is a block diagram showing an example of the functional configuration of a checkout device 30 to which this embodiment is applied. The accounting device 30 includes a POS register 40 that registers products and a change dispenser 50 that deposits and dispenses currency. The accounting device 30 also includes an unregistered product counter 31 on which unregistered products are placed, and a registered product counter 32 on which registered products are placed. The POS register 40 and the change dispenser 50 will be described in detail later.

[0018] (POS register 40) The POS register 40 includes a display operation unit 41, a scanner 42, a card reader 43, a printing unit 44, a communication unit 45, and a hand scanner 46. The display operation unit 41 can be exemplified by a liquid crystal touch panel display. The scanner 42 optically reads the image of the barcode attached to the product to obtain information such as the product name and price.

[0019] The card reader 43 reads information from credit cards, debit cards, prepaid cards, and the like. The printing unit 44 prints a receipt containing transaction details such as the registered product name, price, amount of inserted coins, amount of change, date and time, etc., from the receipt outlet 44a. The communication unit 45 is a communication interface that communicates with the change dispenser 50 and the management device 100.

[0020] Hand scanner 46 is located on top of device housing 33 of checkout device 30. A holder 34 is provided on top of device housing 33, which holds hand scanner 46 with the tip of hand scanner 46 hooked onto it. Hand scanner 46 is held and operated by a customer, and obtains information such as the product name and price by optically reading the image of the barcode attached to the product. Scanner 42 and hand scanner 46 may be any device that reads information for identifying a product, for example, information other than a barcode. Scanner 42 and hand scanner 46 may also read information other than information for identifying a product. For example, scanner 42 and hand scanner 46 may obtain information about a customer who is a member of store S by reading an image of a barcode attached to a membership card or the like of store S.

[0021] The POS register 40 also includes a POS control unit 47 that includes a CPU, a ROM, and a RAM and controls the entire POS register 40. The POS register 40 also includes a storage unit 48 that stores various programs such as an OS (Operating System) and applications, input data for the various programs, output data from the various programs, etc. The storage unit 48 can be exemplified by a storage device such as an HDD (Hard Disk Drive) or semiconductor memory.

[0022] The POS control unit 47 includes a purchase product registration unit 471 , a purchase amount calculation unit 472 , a payment processing unit 473 , and a display control unit 474 . When the scanner 42 or hand scanner 46 reads the barcode attached to the product and acquires information such as the product name and price, the purchased product registration unit 471 registers the product using the acquired information. Furthermore, the purchase product registration unit 471 transmits information about the registered products to the management device 100. Examples of the information about the registered products include the names and quantities of the registered products.

[0023] The purchase amount calculation unit 472 calculates the purchase amount by adding up the prices of all the products registered by the purchase product registration unit 471. The payment processing unit 473 completes the transaction by settling the purchase amount calculated by the purchase amount calculation unit 472 and issues a receipt. That is, the payment processing unit 473 causes the printing unit 44 to eject a receipt on which the transaction details are printed from the receipt outlet 44a. Examples of transaction details include the registered product name and price, the amount of inserted currency, the amount of change, the date and time, etc. The purchase amount may be settled with a payment card such as a credit card, or in cash. If cash payment is selected, the payment processing unit 473 transmits the purchase amount to the change dispenser 50 and completes the payment when it receives a settlement completion notice from the change dispenser 50.

[0024] Furthermore, if the customer does not pay the purchase amount, the payment processing unit 473 outputs a payment incompletion notice indicating that the payment is incomplete to the management device 100. In addition, if the customer does not pay the purchase amount for a certain period after the purchase amount calculation unit 472 calculates the purchase amount, the payment processing unit 473 outputs a payment incompletion notice to the management device 100. Furthermore, when the purchase amount is paid by credit card, the payment processing unit 473 outputs information obtained by reading the credit card with the card reader 43 to the management device 100. Examples of information obtained by reading the credit card include information that can identify the customer using the credit card, such as the credit card number and the name of the credit card holder.

[0025] The display control unit 474 causes the display operation unit 41 to display information for prompting the user to register the product to be purchased. For example, the display control unit 474 causes the display operation unit 41 to display text information such as "Please scan the product barcode." Furthermore, the display control unit 474 causes the display operation unit 41 to display a list of the products registered by the purchase product registration unit 471. Furthermore, the display control unit 474 causes the display operation unit 41 to display the purchase amount calculated by the purchase amount calculation unit 472. Furthermore, when payment by cash is selected, the display control unit 474 causes the display operation unit 41 to display the deposit amount, the amount of change, and the like.

[0026] (Change machine 50) As shown in Fig. 4, the change dispenser 50 includes a communication unit 51, a display operation unit 52, and a memory unit 53. The communication unit 51 is a communication interface that communicates with the POS register 40 and the management device 100. The display operation unit 52 is a liquid crystal touch panel display or the like. The memory unit 53 can be exemplified as a storage device such as a HDD or semiconductor memory.

[0027] The change machine 50 also includes a change control unit 54 that includes a CPU, ROM, and RAM and controls the entire change machine 50, a banknote handling unit 60, and a coin handling unit 70. The change machine 50 also includes a banknote insertion slot 61 (see FIG. 3) used for inserting banknotes, and a banknote dispensing opening 62 (see FIG. 3) used for dispensing banknotes. The change machine 50 also includes a coin insertion slot 71 (see FIG. 3) used for inserting coins, and a coin dispensing opening 72 (see FIG. 3) used for dispensing coins.

[0028] The change control unit 54 has a deposit processing unit 541 that performs a process of depositing money, and a withdrawal processing unit 542 that performs a process of withdrawing money. When the deposit processing unit 541 receives the purchase amount from the POS register 40, it permits the insertion of coins into the banknote handling unit 60 and the coin handling unit 70. The deposit processing unit 541 then obtains the number of coins of each denomination inserted from the banknote handling unit 60 and the coin handling unit 70 and calculates the total amount of the inserted coins. If the total amount of the inserted coins exceeds the purchase amount, the deposit processing unit 541 calculates the difference between the total amount and the purchase amount as change. The deposit processing unit 541 then instructs the withdrawal processing unit 542 to dispense the calculated change. On the other hand, if the total amount of the inserted coins matches the purchase amount or if the withdrawal processing unit 542 has successfully dispensed the change, the deposit processing unit 541 sends a settlement completion notification to the POS register 40.

[0029] When the withdrawal processing unit 542 receives a change dispensing instruction from the deposit processing unit 541, it determines the denomination and number of coins to be dispensed according to the amount of change. Then, the withdrawal processing unit 542 performs the dispensing process by outputting a dispensing instruction to the banknote processing unit 60 and the coin processing unit 70. Furthermore, when the withdrawal of the change has been completed successfully, the withdrawal processing unit 542 notifies the deposit processing unit 541.

[0030] When a customer wants to register an item in the shopping cart 300, the customer places the item to be purchased in the shopping cart 300 on the unregistered item stand 31. The customer then takes the item out of the shopping cart 300, holds the barcode on the item over the scanner 42 to register it, and then moves it to the registered item stand 32, repeating this process. Furthermore, when a customer places an item on the lower level of the cart 350 on which the shopping basket 300 is placed, the barcode attached to the item is read and registered with the hand scanner 46. When the customer has completed the registration of all the products, he or she pays the purchase amount based on the purchase amount displayed on the display operation unit 41, and the payment process for the products is completed. After completing the payment process for the purchased items, the customer usually leaves the store S through the entrance / exit 5 with the purchased items in hand or with the items still in the cart 350.

[0031] (Management device 100) FIG. 5 is a diagram illustrating an example of a schematic configuration of the management device 100. As shown in FIG. The management device 100 includes a control unit 110 that includes a CPU, a ROM, and a RAM, and controls the entire management device 100. The management device 100 also includes a storage unit 120 that stores various programs such as an OS and applications, input data for the various programs, output data from the various programs, etc. The storage unit 120 can be exemplified by a storage device such as a HDD or semiconductor memory. The management device 100 also includes a communication unit 130 that communicates with other devices that make up the monitoring system 1, such as the store terminal 10, the image capturing device 20, and the accounting device 30. The communication unit 130 can be exemplified as a communication interface.

[0032] The management device 100 can be exemplified as a computer having a function of transmitting and receiving information to and from other devices constituting the monitoring system 1. Like the center server 200, the management device 100 may be a notebook PC, a desktop PC, a tablet PC, a tablet terminal, a personal digital assistant, or a multi-function mobile phone. In this example, the management device 100 is installed in the back office of the store S. Note that the management device 100 may be installed in a headquarters or the like that manages multiple stores including the store S, in addition to the back office of the store S.

[0033] The management device 100 detects fraudulent behavior of a customer regarding the purchase of a product at the store S. If the amount of the detected fraudulent behavior of the customer satisfies a predetermined standard, the management device 100 outputs a first alert to the store terminal 10 indicating that fraudulent behavior has occurred. Here, fraudulent behaviors related to product purchases by customers at the store S are classified into multiple types. First, fraudulent behaviors detected by the management device 100 will be described. Note that the fraudulent behaviors described below are just examples, and the management device 100 may detect other actions of customers at the store S as fraudulent behaviors.

[0034] The fraudulent behavior is classified into fraudulent behavior that mainly takes place in the sales floor of the store S and fraudulent behavior that mainly takes place in the vicinity of the checkout device 30 of the store S. Examples of fraudulent behavior that occurs on the sales floor of store S include shoplifting and shopping cart removal. Shoplifting is the act of trying to take merchandise displayed on a sales floor by putting it in a customer's pocket or personal bag. A basket abandonment is an act of leaving the store S after putting items displayed on the sales floor into the shopping cart 300 without completing the payment process at the accounting device 30.

[0035] Fraudulent behaviors that occur near the checkout device 30 of store S include skipping scans, leaving items at the bottom of the cart, leaving the store without paying, taking unpaid items, falsifying barcodes, and tampering with the checkout device 30. A "skipping scan" is an act of transferring one or more items to the registered item tray 32 without registering them when registering items in the shopping cart 300 using the checkout device 30, and then proceeding with the checkout process. A customer may perform a "skipping scan" by, for example, hiding the barcode on an item and holding it over the scanner 42, or by holding the side of the item without the barcode over the scanner 42. A customer may also perform a "skipping scan" by, for example, holding only one item over the scanner 42, but then transferring multiple identical items to the registered item tray 32.

[0036] Leaving an item at the bottom of the cart refers to the act of carrying out the payment process without registering the item placed at the bottom of the cart 350 carrying the shopping basket 300 in the checkout device 30. Leaving without paying is the act of leaving the store S after registering the product at the accounting device 30 without paying the purchase amount. Taking away unpaid items refers to the act of taking away unpaid items along with the items that have been paid for at the checkout device 30. For example, the customer places a bag or the like containing the unpaid items on the registered item counter 32, then registers the remaining items at the checkout device 30, places them in the bag, and takes away the unpaid items.

[0037] Barcode falsification is the act of registering a product by scanning a barcode other than the one actually attached to the product in the checkout device 30. For example, a customer may falsify a barcode by scanning the barcode of another product that is cheaper than the target product. Also, when purchasing a set product that combines multiple products, a customer may falsify the barcode by scanning the barcode attached to a single product rather than the set product. Unauthorized operation of the accounting device 30 refers to the act of registering a product that is cheaper than the original product or a smaller quantity than the original quantity when operating the display operation unit 41 to register a product that does not have a barcode.

[0038] Here, a customer may commit fraudulent behavior with malicious intent in the store S, or may commit fraudulent behavior by mistake without malicious intent. In this embodiment, when fraudulent behavior is committed in the store S, the management device 100 detects the fraudulent behavior regardless of whether the customer has malicious intent. Note that the above-mentioned fraudulent behaviors include those that are likely to be committed maliciously by customers and those that are likely to be committed accidentally by customers without malicious intent. In other words, the likelihood that each of the above-mentioned fraudulent behaviors will be committed maliciously by customers varies depending on the type. In the description of this embodiment, fraudulent behaviors that are likely to be committed maliciously by customers may be referred to as fraudulent behaviors with a high degree of maliciousness by customers. Furthermore, fraudulent behaviors that are unlikely to be committed maliciously by customers, i.e., that are likely to be committed accidentally by customers, may be referred to as fraudulent behaviors with a low degree of maliciousness.

[0039] For example, among the fraudulent behaviors described above, shoplifting, leaving the cart, leaving the store without paying, falsifying barcodes, and taking away unpaid merchandise are considered to be highly malicious fraudulent behaviors by customers. Furthermore, among the fraudulent behaviors described above, skipping the scan, leaving items at the bottom of the cart, and unauthorized operation of the checkout device 30 are considered to be less malicious fraudulent behaviors by customers. Note that the classification of fraudulent behaviors by their maliciousness is merely an example and is not limiting. Furthermore, multiple types of fraudulent behaviors may be classified into three or more stages according to the degree of maliciousness.

[0040] (Storage unit 120) The storage unit 120 stores the history of fraudulent behavior of customers who use the store S. In addition, the storage unit 120 stores identification information used to identify customers who have committed fraudulent behavior in the store S in the past, in association with the history of fraudulent behavior of the customers. In the description of this embodiment, the identification information used to identify a customer who has committed fraudulent behavior at the store S in the past may be referred to as customer identification information.

[0041] An example of the customer identification information is an image captured by the photographing device 20 when the customer previously committed fraudulent behavior at the store S. More specifically, an example of the customer identification information is a facial image of the customer cut out from the image captured by the photographing device 20. Furthermore, examples of customer identification information include information obtained by reading a credit card with the checkout device 30 when the customer previously committed fraudulent acts at store S. As described above, examples of information obtained by reading a credit card include information that can identify the customer using the credit card, such as the credit card number and name. Furthermore, the customer identification information may include information such as an identification number or ID assigned to each customer who has a history of fraudulent behavior.

[0042] The customer's fraudulent behavior history is information indicating that the customer has engaged in fraudulent behavior in the store S in the past. In this embodiment, the memory unit 120 stores, as the customer's fraudulent behavior history, information indicating the amount of fraudulent behavior that the customer has engaged in in the store S in the past. In this embodiment, the past refers to a period before the entry of the customer into the store was detected by the entry detection unit 112 of the control unit 110, which will be described later. The storage unit 120 updates the history of fraudulent behavior of a customer every time fraudulent behavior of the customer at the store S is detected by the fraudulent behavior detection unit 113 of the control unit 110, which will be described later.

[0043] The amount of fraudulent behavior can be exemplified by the number of times that the customer has committed fraudulent behavior at store S in the past. For example, the management device 100 counts the number of times a customer has committed fraudulent behavior in the store S in the past, and stores the count in the storage unit 120. The management device 100 may count the number of times fraudulent behavior has been committed collectively regardless of the type of fraudulent behavior, or may count the number of times fraudulent behavior has been committed individually for each type of fraudulent behavior.

[0044] Furthermore, the amount of fraudulent behavior can be exemplified by the frequency with which a customer has previously committed fraudulent behavior at store S. In the following description, the frequency with which a customer has previously committed fraudulent behavior at store S may be referred to as the frequency of fraudulent behavior. The management device 100 calculates the frequency of fraudulent behavior based on the time interval during which the customer commits fraudulent behavior, the number of times the customer commits fraudulent behavior in a predetermined period, etc., and stores the calculated frequency in the storage unit 120. The management device 100 may calculate the frequency of fraudulent behavior collectively regardless of the type of fraudulent behavior, or may calculate the frequency of fraudulent behavior individually for each type of fraudulent behavior.

[0045] The amount of fraudulent behavior can be, for example, points calculated by converting the number of times a customer has committed fraudulent behavior into points, taking into account the severity of the fraudulent behavior. As described above, the degree of maliciousness of fraudulent behavior varies depending on the type of fraudulent behavior. The management device 100 adds points that vary depending on the maliciousness of the fraudulent behavior each time a customer commits fraudulent behavior, and stores the points in the storage unit 120. For example, the management device 100 adds more points to fraudulent behaviors that are more malicious, and adds fewer points to fraudulent behaviors that are less malicious. The fraudulent behavior history stored in the storage unit 120 is not limited to the number of times, frequency, points, etc., as described above, as long as it is information that can be used to evaluate the amount of fraudulent behavior of a customer.

[0046] Furthermore, for example, if the memory unit 120 stores the number of times a customer has engaged in fraudulent behavior or the points converted from this number as the amount of fraudulent behavior, the memory unit 120 may initialize this number of times or the points converted from this number of times when the customer's behavior at the store S satisfies a predetermined condition. The number of times a customer has engaged in fraudulent behavior and the points converted from this number of times stored in the memory unit 120 as the amount of fraudulent behavior are examples of number-of-times information. For example, if a customer does not commit any fraudulent behavior for a predetermined period of time after the last time the customer committed fraudulent behavior at store S, the memory unit 120 can initialize the number of times information, assuming that the predetermined condition has been met. Examples of initialization include deleting the history of fraudulent behavior stored in memory unit 120, or reducing the number of times a customer has committed fraudulent behavior or points stored in memory unit 120 to 0, etc.

[0047] (control unit 110) The control unit 110 includes an acquisition unit 111 that acquires images captured by the image capture device 20 and various information output from the checkout device 30. The control unit 110 also includes an entry detection unit 112 that detects a customer's entry into the store S. The control unit 110 also includes a fraudulent behavior detection unit 113 that detects fraudulent behavior by the customer in the store S. The control unit 110 also includes a determination unit 114 that determines whether the amount of fraudulent behavior by the customer satisfies a predetermined criterion when fraudulent behavior is detected by the fraudulent behavior detection unit 113. The control unit 110 also includes a first output unit 115 that outputs a first alert to the store terminal 10 indicating that fraudulent behavior has been committed when the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the criterion. The control unit 110 also includes a second output unit 116 that outputs a second alert to the store terminal 10 indicating that a customer has entered the store when the entry detection unit 112 detects the entry of a customer with a history of fraudulent behavior.

[0048] The acquisition unit 111 acquires images captured by the store entrance camera 20a, the store exit camera 20b, the store camera 20c, and the checkout camera 20d of the image capturing device 20. The acquisition unit 111 also acquires information output from the POS control unit 47 in the POS register 40 of the checkout device 30. Specifically, the acquisition unit 111 acquires information about registered products output from the purchased product registration unit 471 of the POS control unit 47. The acquisition unit 111 also acquires a payment incomplete notification output from the payment processing unit 473 of the POS control unit 47. The acquisition unit 111 also acquires information that can identify the customer using the credit card, obtained by reading the credit card, output from the payment processing unit 473 of the checkout device 30.

[0049] The store entry detection unit 112 detects that a customer has entered the store S based on the captured image, information, etc. acquired by the acquisition unit 111. In the description of this embodiment, the customer's entry into the store S may be referred to as the customer entering the store. The store entry detection unit 112 is an example of a second detection unit for detecting a customer entering the store.

[0050] The store entry detection unit 112 can detect the entrance of a customer near the checkout device 30 installed in the store S. Additionally, the store entry detection unit 112 can detect a customer's entry based on images captured by the accounting camera 20d, which captures images of customers near the accounting device 30. Specifically, the store entry detection unit 112 can determine that a customer has entered the store S if an image of the customer is included in the image captured by the accounting camera 20d. The store entry detection unit 112 can also detect a customer's entry based on information output from the checkout device 30 in response to the customer's operation of the checkout device 30. Specifically, the store entry detection unit 112 can determine that a customer has entered the store S when the acquisition unit 111 acquires information capable of identifying the customer that is output from the checkout device 30 and acquired by reading a credit card.

[0051] Furthermore, the store entry detection unit 112 can detect the entry of a customer at the entrance 5 of the store S. In addition, the store entry detection unit 112 can detect the entry of a customer based on an image captured by the store entry camera 20a, which captures an image of a customer entering the store S through the entrance 5.

[0052] When the store entry detection unit 112 detects that a customer has entered the store, it determines whether the customer whose entry has been detected has a history of fraudulent behavior. In addition, the store entry detection unit 112 determines whether the customer whose entry has been detected has a history of fraudulent behavior based on the captured image and information acquired by the acquisition unit 111 and used to detect the customer's entry, and the customer's identification information stored in the storage unit 120. When the store entry detection unit 112 determines that the customer whose entry has been detected is a customer with a history of fraudulent behavior, it outputs the determination result to the second output unit 116. In other words, the store entry detection unit 112 outputs information to the second output unit 116 that it has detected the entry of a customer with a history of fraudulent behavior.

[0053] For example, the store entry detection unit 112 compares the customer's facial image cut out from the image captured by the photographing device 20 acquired by the acquisition unit 111 with the customer's facial image, which is an example of the customer's identification information stored in the memory unit 120. The store entry detection unit 112 also compares the information capable of identifying the customer using the credit card acquired by the acquisition unit 111 with information acquired by reading the credit card with the checkout device 30 when the customer previously committed fraudulent behavior at store S. The information acquired by reading the credit card with the checkout device 30 when the customer previously committed fraudulent behavior at store S is an example of customer identification information stored in the memory unit 120. Then, based on the comparison result, the store entry detection unit 112 determines whether the customer detected as entering the store has a history of fraudulent behavior. For example, if the captured image and information acquired by the acquisition unit 111 match the customer's identification information stored in the storage unit 120, the store entry detection unit 112 determines that the customer detected as entering the store has a history of fraudulent behavior.

[0054] The store entry detection unit 112 may determine that a customer whose entry has been detected is a customer with a history of fraudulent behavior only if the customer has a history of fraudulent behavior of a predetermined type among multiple types of fraudulent behavior. For example, the store entry detection unit 112 may determine that a customer whose entry has been detected is a customer with a history of fraudulent behavior only if the customer has a history of fraudulent behavior of a high level among multiple types of fraudulent behavior. In other words, the store entry detection unit 112 may determine that a customer whose entry has been detected is a customer without fraudulent behavior if the customer has a history of only fraudulent behavior of a low level among multiple types of fraudulent behavior.

[0055] The fraudulent behavior detection unit 113 detects fraudulent behavior of customers in the store S based on the captured images, information, etc. acquired by the acquisition unit 111. The fraudulent behavior detection unit 113 is an example of a detection unit. The fraudulent behavior detection unit 113 detects fraudulent behavior of customers, for example, by analyzing images captured by the photographing device 20. In addition, the fraudulent behavior detection unit 113 detects fraudulent behavior of customers, for example, by comparing the analysis results of the images captured by the photographing device 20 with information about products registered by the checkout device 30 output from the POS control unit 47.

[0056] The fraudulent behavior detection unit 113 can detect fraudulent behaviors such as shoplifting and basket removal by analyzing images captured by the store exit camera 20b and the store camera 20c, for example. Furthermore, the fraudulent behavior detection unit 113 can detect fraudulent behaviors such as skipping scans, leaving items under the cart, and taking away unpaid items, for example, by analyzing images captured by the checkout camera 20d. Furthermore, the fraudulent behavior detection unit 113 can detect an unsettled store exit based on a settlement incomplete notice output from the POS control unit 47, for example. Additionally, the fraudulent behavior detection unit 113 checks for inconsistencies between the analysis results of images captured by the accounting camera 20d and the information about registered products output from the POS control unit 47. This allows the fraudulent behavior detection unit 113 to detect fraudulent behavior such as barcode falsification and fraudulent operation of the accounting device 30. The methods by which the fraudulent behavior detection unit 113 detects each type of fraudulent behavior are merely examples, and the methods are not limited to these.

[0057] When the fraudulent behavior detection unit 113 detects fraudulent behavior of a customer, the determination unit 114 determines whether the amount of fraudulent behavior of the customer satisfies a predetermined standard based on the fraudulent behavior history of the customer stored in the storage unit 120. The amount of fraudulent behavior is a value that represents the amount of fraudulent behavior committed by the customer during a predetermined period going back from when the fraudulent behavior detection unit 113 detected the fraudulent behavior. The determination unit 114 determines that the amount of fraudulent behavior of the customer satisfies the criterion when the amount of fraudulent behavior of the customer is equal to or greater than a predetermined value. Furthermore, the determination unit 114 determines that the amount of fraudulent behavior of the customer does not satisfy the criterion when the amount of fraudulent behavior of the customer is less than the predetermined value. In the description of this embodiment, the predetermined criterion for the amount of fraudulent behavior may be simply referred to as the criterion. Then, the determination unit 114 outputs the determination result to the first output unit 115.

[0058] First, a case will be described in which the determination unit 114 makes a determination based on the number of fraudulent behaviors of a customer measured collectively regardless of the type of fraudulent behavior, as an example of the amount of fraudulent behavior of a customer. The determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the criterion when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is a predetermined number of times, two or more. For example, the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the criterion when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is the second or subsequent fraudulent behavior. In other words, the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the criterion when the number of times the customer has committed fraudulent behavior in the store S in the past is one or more. Furthermore, the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the criterion when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 has not reached a predetermined number of times. For example, the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the criterion when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is the first fraudulent behavior. In other words, the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the criterion when the customer has not committed fraudulent behavior at store S in the past.

[0059] Next, a case will be described in which the determination unit 114 makes a determination based on the number of fraudulent behaviors of a customer measured individually for each type of fraudulent behavior, as an example of the amount of fraudulent behavior of a customer. The determination unit 114 determines whether the amount of fraudulent behavior satisfies a criterion for each type of fraudulent behavior. The determination unit 114 determines whether the amount of fraudulent behavior satisfies a criterion by comparing the number of times the customer has committed fraudulent behavior for the type of fraudulent behavior detected by the fraudulent behavior detection unit 113 with a predetermined reference number. For example, the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the criterion when the number of times the customer has committed fraudulent behavior for the type of fraudulent behavior detected by the fraudulent behavior detection unit 113 is equal to or greater than the reference number. Furthermore, the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the criterion when the number of times the customer has committed fraudulent behavior for the type of fraudulent behavior detected by the fraudulent behavior detection unit 113 is less than the reference number.

[0060] The reference number of times used by the determination unit 114 for determination may differ depending on the type of fraudulent behavior. For example, the determination unit 114 may set the reference number of times used to determine highly malicious fraudulent behavior among multiple types of fraudulent behavior to a value smaller than the reference number of times used to determine less malicious fraudulent behavior. In this case, when the fraudulent behavior of a customer is highly malicious, the amount of fraudulent behavior by the customer is more likely to be determined to meet the reference number than when the fraudulent behavior is less malicious. This makes it easier for the first output unit 115 to output a first alert indicating that fraudulent behavior has been committed to the store terminal 10, as will be described later.

[0061] Furthermore, the determination unit 114 may set the reference number of times used to determine a predetermined type of fraudulent behavior among multiple types of fraudulent behavior as one. In other words, the determination unit 114 determines that the fraudulent behavior detected by the fraudulent behavior detection unit 113 satisfies the criterion if it is the second or subsequent fraudulent behavior. On the other hand, if the fraudulent behavior detected by the fraudulent behavior detection unit 113 is of a predetermined type, the determination unit 114 may determine that the criterion is satisfied even if the fraudulent behavior of the customer is not the second or subsequent fraudulent behavior. In other words, if the fraudulent behavior detected by the fraudulent behavior detection unit 113 is of a predetermined type, the determination unit 114 may determine that the criterion is satisfied even if it is the first fraudulent behavior. For example, the determination unit 114 may set the reference number of times used to determine whether a fraudulent behavior of a predetermined type is highly malicious as 1. As a result, if the fraudulent behavior committed by a customer is highly malicious, the first output unit 115 will output a first alert indicating that fraudulent behavior has been committed to the store terminal 10, even if it is the first fraudulent behavior.

[0062] In addition, a case will be described in which the determination unit 114 makes a determination based on the frequency of fraudulent behavior of a customer as an example of the amount of fraudulent behavior of a customer. The determination unit 114 compares the frequency of fraudulent behavior of a customer whose fraudulent behavior has been detected by the fraudulent behavior detection unit 113 with a predetermined standard value to determine whether the amount of fraudulent behavior satisfies the standard. For example, the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the standard when the frequency of fraudulent behavior of the customer whose fraudulent behavior has been detected by the fraudulent behavior detection unit 113 is equal to or greater than the standard value. Furthermore, the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the standard when the frequency of fraudulent behavior of the customer whose fraudulent behavior has been detected by the fraudulent behavior detection unit 113 is less than the standard value. The determination unit 114 may compare the frequency of misconduct with the reference value for all misconduct types, or may compare the frequency of misconduct with the reference value for each misconduct type. When the determination unit 114 compares the frequency of misconduct with the reference value for each misconduct type, the reference value may differ depending on the type of misconduct.

[0063] In addition, a case will be described in which the determination unit 114 makes a determination based on points obtained by converting the number of times a customer has committed fraudulent behavior, as an example of the amount of fraudulent behavior of the customer. The determination unit 114 determines whether the amount of fraudulent behavior of a customer detected by the fraudulent behavior detection unit 113 satisfies a predetermined standard by comparing points converted from the number of times the customer has committed fraudulent behavior with a predetermined standard value. For example, the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the standard if the points converted from the number of times the customer has committed fraudulent behavior are equal to or greater than the standard value. Furthermore, the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the standard if the points converted from the number of times the customer has committed fraudulent behavior are less than the standard value.

[0064] When the determination unit 114 determines that the amount of fraudulent behavior by a customer meets the criterion, the first output unit 115 outputs a first alert to the store terminal 10. The first alert is an alert indicating that fraudulent behavior has been committed. This allows a store clerk using the store terminal 10 to recognize that a customer has committed fraudulent behavior in the store S. The store clerk can then take action against the fraudulent behavior, such as speaking to the customer who committed the fraudulent behavior or reporting it to a security company or the police.

[0065] On the other hand, if the determination unit 114 determines that the amount of fraudulent behavior by the customer does not satisfy the standard, the first output unit 115 does not output the first alert to the store terminal 10. Here, if the amount of fraudulent behavior by the customer does not satisfy the standard, it is highly likely that the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 was committed by mistake without malicious intent. In this case, by the first output unit 115 not outputting the first alert, a decrease in the work efficiency of store staff at store S is suppressed.

[0066] As described above, the determination unit 114 may determine whether the amount of fraudulent behavior satisfies the criteria for each type of fraudulent behavior detected by the fraudulent behavior detection unit 113. In this case, whether or not the first output unit 115 outputs the first alert will differ depending on the amount of fraudulent behavior and the type of fraudulent behavior. 6 is a diagram showing an example of the relationship between whether or not the first output unit 115 outputs a first alert and the type and amount of fraudulent behavior detected by the fraudulent behavior detection unit 113. Here, the determination unit 114 determines whether or not the amount of fraudulent behavior satisfies the criterion for each type of fraudulent behavior based on the number of times a customer has committed fraudulent behavior. More specifically, the determination unit 114 determines that the amount of fraudulent behavior satisfies the criterion when the fraudulent behavior detected by the fraudulent behavior detection unit 113 is highly malicious, even if it is the first fraudulent behavior.

[0067] In the example shown in Figure 6, if the fraudulent behavior detection unit 113 detects highly malicious fraudulent behavior such as shoplifting, leaving a shopping cart, leaving the store without paying, taking away unpaid merchandise, and barcode falsification, the first output unit 115 outputs a first alert even if it is the first time the customer has engaged in fraudulent behavior. On the other hand, when the fraudulent behavior detection unit 113 detects less malicious fraudulent behavior such as leaving items under the cart, skipping a scan, or fraudulent operation of the checkout device 30, if this is the first fraudulent behavior by the customer, the first output unit 115 will not output the first alert. In this case, the first output unit 115 will output the first alert only if the fraudulent behavior by the customer is the second or subsequent time.

[0068] The first alert can be exemplified by displaying predetermined text information indicating that a customer has engaged in fraudulent behavior on the display unit 12 of the store terminal 10. In addition, when the determination unit 114 determines that the amount of fraudulent behavior by the customer satisfies the criterion, the first output unit 115 outputs a command to the store terminal 10 to cause the display unit 12 to display the predetermined text information. Furthermore, the first alert may include other information in addition to the predetermined text information indicating that a customer has committed fraudulent behavior. Examples of the other information include information identifying the customer who committed the fraudulent behavior, information indicating the type of fraudulent behavior, and information indicating the severity of the fraudulent behavior. Examples of the other information include information indicating the location in store S where the fraudulent behavior was committed, and information indicating the number of times and date and time that the customer has committed fraud in the past.

[0069] Here, the maliciousness of the fraudulent behavior differs depending on the type of fraudulent behavior, as described above. The first output unit 115 can output information indicating the maliciousness of the fraudulent behavior to the store terminal 10 according to the type of fraudulent behavior detected by the fraudulent behavior detection unit 113. In general, customers who maliciously commit fraudulent behavior often do not use credit cards, membership cards, etc., in order to avoid disclosing information that could identify them. In other words, if a customer uses a credit card, membership card, etc. at the checkout device 30, it is highly likely that the customer committed fraudulent behavior by mistake without malicious intent. Therefore, when information indicating that a customer used a credit card, membership card, etc. is obtained from the checkout device 30, the first output unit 115 can output information indicating the maliciousness of the fraudulent behavior as low.

[0070] Furthermore, the first output unit 115 may output the first alert to the checkout device 30 in addition to the store terminal 10. The checkout device 30 is an example of a customer terminal used by a customer. The first alert output to the checkout device 30 may be, for example, a message displayed on the display operation unit 41 of the POS register 40 indicating that fraudulent behavior has been committed by the customer. This allows the customer to recognize that they have engaged in fraudulent behavior through the first alert displayed on the checkout device 30. For example, if a customer accidentally engages in fraudulent behavior without malicious intent, the customer's awareness of the fraudulent behavior makes it easier for the customer to resolve the fraudulent behavior.

[0071] When the store entry detection unit 112 detects the entry of a customer with a history of fraudulent behavior, the second output unit 116 outputs a second alert to the store terminal 10. The second alert is an alert indicating that a customer with a history of fraudulent behavior has entered the store. Generally, a customer with a history of fraudulent behavior, i.e., a customer who has committed fraudulent behavior in the store S in the past, is more likely to commit fraudulent behavior again in the store S than a customer without a history of fraudulent behavior. In this embodiment, when a customer with a history of fraudulent behavior is detected entering the store, a second alert is output to the store terminal 10, making it easier for a store clerk using the store terminal 10 to take action such as monitoring the behavior of the customer. This makes it more difficult for the customer to commit fraudulent behavior again in the store S.

[0072] The second alert can be exemplified by displaying predetermined text information indicating that a customer with a history of fraudulent behavior has entered the store on the display unit 12 of the store terminal 10. In addition, when the store entry detection unit 112 detects that a customer with a history of fraudulent behavior has entered the store, the second output unit 116 outputs a command to the store terminal 10 to cause the display unit 12 to display the predetermined text information.

[0073] The second output unit 116 may output the second alert to the store terminal 10 only when the store entry detection unit 112 detects the entry of a customer who has a history of fraudulent behavior of a predetermined type among multiple types of fraudulent behavior. For example, the second output unit 116 may output the second alert to the store terminal 10 only when the store entry detection unit 112 detects the entry of a customer who has a history of fraudulent behavior that indicates that fraudulent behavior of a high level has been committed among multiple types of fraudulent behavior. In this case, the second alert is less frequently output to the store terminal 10 than when the second alert is output regardless of the type of fraudulent behavior. This prevents a decrease in the work efficiency of store staff at the store S.

[0074] As described above, the store entry detection unit 112 can detect the entry of a customer at the entrance 5 of the store S. In this case, the second output unit 116 can output a second alert to the store terminal 10 at the timing when the entry of a customer at the entrance 5 of the store S is detected. When the second output unit 116 detects the entry of a customer who has previously committed shoplifting or cart abandonment, among multiple types of fraudulent behavior, it is preferable that the second output unit 116 outputs a second alert to the store terminal 10 at the timing when the customer's entry is detected at the entrance / exit 5 of the store S. Shoplifting and cart abandonment are fraudulent behaviors that mainly occur on the sales floor of the store S and occur when the customer does not use the checkout device 30. Therefore, for customers who have previously committed shoplifting or cart abandonment, by outputting a second alert at the timing when the customer's entry is detected at the entrance / exit 5, store staff can become aware of the customer's entry before they shoplift or cart abandonment occurs again.

[0075] As described above, the entry detection unit 112 can detect the entry of a customer near the checkout device 30. In this case, the second output unit 116 can output a second alert to the store terminal 10 when it detects the entry of a customer near the checkout device 30. When the second output unit 116 detects a customer who has previously engaged in fraudulent behavior near the checkout device 30 among multiple types of fraudulent behavior, it preferably outputs a second alert to the store terminal 10 near the checkout device 30. Fraudulent behavior near the checkout device 30 includes skipping scans, leaving items under the cart, leaving the store without paying, taking away unpaid items, falsifying barcodes, and fraudulent operation of the checkout device 30.

[0076] <Monitoring process> Next, a monitoring process in which the control unit 110 of the management device 100 monitors fraudulent behavior by customers in the store S and issues an alert will be described using a flowchart. 7 is a flowchart showing an example of the monitoring process performed by the control unit 110. The control unit 110 repeatedly performs the monitoring process at predetermined intervals, for example, every 1 millisecond.

[0077] In the control unit 110, the acquisition unit 111 acquires the image captured by the photographing device 20 and the information output from the accounting device 30 (step 101). In addition, the acquisition unit 111 acquires images captured by the store entrance camera 20a, the store exit camera 20b, the store camera 20c, and the checkout camera 20d that constitute the photographing device 20. The acquisition unit 111 also acquires information output from the checkout device 30, such as information about registered products, information that can identify the customer acquired by reading a credit card, and incomplete payment notifications.

[0078] Next, the control unit 110 causes the store entry detection unit 112 to detect the entry of a customer into the store S based on the captured image and information acquired in step 101 (step 102). In other words, in step 102, the store entry detection unit 112 determines whether or not a customer has entered the store S based on the captured image and information acquired in step 101. If the entry of a customer into the store S is not detected (NO in step 102), the control unit 110 ends the monitoring process.

[0079] On the other hand, if a customer's entry into store S is detected (YES in step 102), the control unit 110 determines whether the customer whose entry has been detected by the store entry detection unit 112 has a history of fraudulent behavior (step 103). In addition, the store entry detection unit 112 determines whether the customer whose entry has been detected has a history of fraudulent behavior based on the captured image and information used to detect the customer's entry in step 102 and the customer's identification information stored in the memory unit 120. If the customer detected as entering the store does not have a history of fraudulent behavior (NO in step 103), the control unit 110 proceeds to step 105, which will be described later, and continues the monitoring process.

[0080] If the customer detected as having entered the store has a history of fraudulent behavior (YES in step 103), the control unit 110 causes the second output unit 116 to output a second alert to the store terminal 10 (step 104). As described above, the second alert is an alert indicating that a customer with a history of fraudulent behavior has entered the store. At the store terminal 10, information indicating that a customer with a history of fraudulent behavior has entered the store is displayed on the display unit 12. This allows the store clerk using the store terminal 10 to recognize that a customer with a history of fraudulent behavior has entered the store.

[0081] Next, the control unit 110 causes the fraudulent behavior detection unit 113 to detect fraudulent behavior of the customer in the store S based on the captured image and information acquired in step 101 (step 105). In other words, in step 105, the fraudulent behavior detection unit 113 determines whether or not the customer has engaged in fraudulent behavior in the store S based on the captured image and information acquired in step 101. If no fraudulent behavior of the customer at store S is detected (NO in step 105), control unit 110 ends the monitoring process.

[0082] On the other hand, if fraudulent behavior of the customer at store S is detected (YES in step 105), the control unit 110 causes the determination unit 114 to determine whether the amount of fraudulent behavior of the customer satisfies a predetermined standard (step 106). In addition, the determination unit 114 determines whether the amount of fraudulent behavior of the customer satisfies the standard based on the history of fraudulent behavior of the customer stored in the storage unit 120. If the amount of fraudulent behavior of the customer does not meet the criteria (NO in step 106), the control unit 110 ends the monitoring process.

[0083] On the other hand, if the amount of fraudulent behavior by the customer meets the criteria (YES in step 106), the control unit 110 causes the first output unit 115 to output a first alert to the store terminal 10 (step 107). As described above, the first alert is an alert indicating that fraudulent behavior has been committed. In the store terminal 10, information indicating that fraudulent behavior has been committed is displayed on the display unit 12. This allows the store clerk using the store terminal 10 to recognize that fraudulent behavior has been committed by a customer in the store S. With the above, the control unit 110 ends the series of monitoring processes.

[0084] As described above, the monitoring system 1 of this embodiment includes the fraudulent behavior detection unit 113 that detects fraudulent behavior of customers regarding product purchases at the store S. The monitoring system 1 also includes a first output unit 115 that outputs a first alert to the store terminal 10, indicating that fraudulent behavior has occurred, when the amount of fraudulent behavior of a customer detected by the fraudulent behavior detection unit 113 meets a predetermined criterion. By outputting the first alert to the store terminal 10, the store clerk operating the store terminal 10 can recognize that fraudulent behavior has occurred in the store S. The store clerk can then take action against the fraudulent behavior, such as speaking to the customer who committed the fraudulent behavior or reporting it to a security company or the police.

[0085] Here, if a customer accidentally commits fraudulent behavior without any malicious intent, it is preferable not to take any action against the fraudulent behavior, such as calling out to the customer who committed the fraudulent behavior or reporting the behavior to a security company or the police, in order to prevent a decrease in the work efficiency of store staff at store S. For example, consider a case where the first output unit 115 uniformly outputs the first alert to the store terminal 10 regardless of whether the amount of fraudulent behavior meets the standard. In this case, even if the amount of fraudulent behavior by a customer is small and it is highly likely that the customer committed the fraudulent behavior by mistake without malicious intent, the first alert will be output to the store terminal 10. As a result, the number of times that store staff at the store S must deal with fraudulent behavior will increase, which may reduce the work efficiency of the store staff at the store S. In contrast, in this embodiment, when the amount of fraudulent behavior of a customer detected by the fraudulent behavior detection unit 113 meets a predetermined standard, a first alert is output to the store terminal 10, thereby preventing a decline in the work efficiency of store staff at store S. As a result, according to this embodiment, a monitoring system 1 is realized that can flexibly respond to the need to take measures in accordance with the scale of damage caused by fraudulent behavior, the maliciousness of the fraudulent behavior, and the like.

[0086] Also, for example, consider a case where fraudulent behavior of a customer is detected by measuring the number of times a customer acquires a product and the number of times the product to be purchased is registered in store S, and a first alert is output based on the detection results. In this case, it is necessary to provide a device or the like for measuring these numbers, which tends to complicate the configuration of the monitoring system 1 and the management device 100. In contrast, in this embodiment, the first alert is output when the amount of fraudulent behavior of a customer meets a predetermined standard based on the fraudulent behavior history, etc., stored in the storage unit 120. This makes it possible to output the first alert with a simpler configuration than when fraudulent behavior is detected by measuring the number of times a customer takes actions to acquire products and the number of times the customer registers products to be purchased in the store S.

[0087] Furthermore, in the monitoring system 1, the first output unit 115 outputs a first alert when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is the second or subsequent fraudulent behavior. On the other hand, the first output unit 115 does not output a first alert when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is the first fraudulent behavior. This prevents a decline in the work efficiency of store staff at store S compared to when the first output unit 115 outputs the first alert regardless of whether the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is the first or second or subsequent occurrence.

[0088] Furthermore, in the monitoring system 1, the first output unit 115 outputs a first alert if the customer whose fraudulent behavior is detected by the fraudulent behavior detection unit 113 has a history of fraudulent behavior when previously performing a payment process at the store S. On the other hand, the first output unit 115 does not output a first alert if the customer whose fraudulent behavior is detected by the fraudulent behavior detection unit 113 does not have a history of fraudulent behavior when previously performing a payment process at the store S. This makes it less likely that the first alert will be output frequently compared to when the first output unit 115 outputs the first alert regardless of whether or not the customer has a history of fraudulent behavior detected by the fraudulent behavior detection unit 113. As a result, a decrease in the work efficiency of store staff at the store S is suppressed.

[0089] The monitoring system 1 also includes a determination unit 114 that determines whether the amount of fraudulent behavior detected by the fraudulent behavior detection unit 113 satisfies a criterion. When the fraudulent behavior detection unit 113 detects fraudulent behavior of a customer, it acquires information about the type of fraudulent behavior. The determination unit 114 also determines whether the amount of fraudulent behavior satisfies a criterion for each type of fraudulent behavior. Then, the first output unit 115 outputs a first alert when the determination unit 114 determines that the criterion is satisfied. Here, fraudulent behaviors include fraudulent behaviors that are likely to be committed by customers with malicious intent and fraudulent behaviors that are likely to be committed by customers by mistake without malicious intent. In this embodiment, the determination unit 114 makes a determination for each type of fraudulent behavior, which makes it easier to output a first alert when a highly malicious fraudulent behavior is committed, compared to when a determination is made regardless of the type of fraudulent behavior.

[0090] Furthermore, in the monitoring system 1, the determination unit 114 determines that the criterion is met when the fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 is the second or subsequent fraudulent behavior. On the other hand, when the fraudulent behavior detected by the fraudulent behavior detection unit 113 is of a predetermined type, the determination unit 114 determines that the criterion is met even if the fraudulent behavior of the customer is not the second or subsequent fraudulent behavior. As a result, if the fraudulent behavior detected by the fraudulent behavior detection unit 113 is of a predetermined type, the first alert will be output even if it is the customer's first fraudulent behavior. For example, by adopting this configuration in which highly malicious fraudulent behavior such as cart pilfering and shoplifting is treated as a predetermined type of fraudulent behavior, it becomes easier to take action such as calling out to the customer who has committed the highly malicious fraudulent behavior or reporting it to a security company or the police.

[0091] The monitoring system 1 further includes an entry detection unit 112 that detects that a customer has entered the store S. The monitoring system 1 further includes a second output unit 116 that, when the entry detection unit 112 detects that a customer with a history of fraudulent behavior has entered the store, outputs a second alert to the store terminal 10 indicating that a customer with a history of fraudulent behavior has entered the store. This makes it easier for a store clerk using the store terminal 10 to recognize that a customer with a history of fraudulent behavior has entered the store before the customer commits fraudulent behavior again. This makes it easier to prevent fraudulent behavior by customers in the store S.

[0092] Furthermore, in the monitoring system 1, the store entry detection unit 112 detects customers in the vicinity of the checkout device 30 who have a history of fraudulent behavior. In this case, a second alert is output to the store terminal 10 when a customer is near the checkout device 30. This makes it easier for the store clerk using the store terminal 10 to monitor the customer operating the checkout device 30, making it easier to prevent fraudulent behavior by customers near the checkout device 30.

[0093] In addition, in the monitoring system 1, the store entry detection unit 112 detects, at the entrance 5 of the store S, customers who have a history of fraudulent behavior. Then, the second output unit 116 outputs the second alert when the entrance of a customer is detected by the store entry detection unit 112. More preferably, the second output unit 116 outputs the second alert when the entrance of a customer who has a history of a predetermined type of fraudulent behavior is detected. This makes it easier for a store clerk using the store terminal 10 to monitor customers who enter the store S before they commit fraudulent acts, making it easier to prevent fraudulent acts by customers. In particular, by outputting the second alert when the entry of a customer with a history of highly malicious fraudulent acts of a predetermined type is detected, it becomes easier to prevent highly malicious fraudulent acts from being committed again.

[0094] The monitoring system 1 further includes a storage unit 120 that stores count information, which is information regarding the number of times fraudulent behavior has been committed, for each customer and each type of fraudulent behavior. The monitoring system 1 further includes a determination unit 114 that determines whether the amount of fraudulent behavior of the customer detected by the fraudulent behavior detection unit 113 satisfies a criterion based on the count information. The first output unit 115 outputs a first alert when the determination unit 114 determines that the criterion is satisfied. The storage unit 120 also initializes the count information when the customer's behavior in store S satisfies a predetermined condition. This prevents the first alert from being output even when the likelihood of fraudulent behavior by a customer is low, compared to when the memory unit 120 does not initialize the count information, for example. This also prevents a decrease in the work efficiency of store staff at store S due to the first alert being output frequently.

[0095] Furthermore, in the monitoring system 1, the first output unit 115 outputs the first alert to the checkout device 30 in addition to the store terminal 10. In this case, the first alert displayed on the checkout device 30 allows the customer to recognize that he or she has engaged in fraudulent behavior.

[0096] Furthermore, in the monitoring system 1, the first output unit 115 outputs to the shop terminal 10, in addition to the first alert, information indicating the severity of the customer's fraudulent behavior. In this case, the store clerk using the store terminal 10 can easily take action such as calling out to the customer who committed the fraudulent behavior or reporting it to a security company or the police, based on the information indicating the severity of the fraudulent behavior output along with the first alert.

[0097] Incidentally, the accuracy with which the fraudulent behavior detection unit 113 detects fraudulent behavior by a customer may differ depending on the images captured by the image capturing device 20. In other words, the more images captured by the in-store cameras 20c in which fraudulent behavior by a customer can be confirmed, the higher the possibility that the customer is actually engaging in fraudulent behavior. On the other hand, if fraudulent behavior by a customer can be confirmed in images captured by one in-store camera 20c but not in images captured by the other in-store cameras 20c, it may be that the customer is not actually engaging in fraudulent behavior. In this embodiment, the fraudulent behavior detection unit 113 may detect fraudulent behavior of a customer in the store S only when a predetermined number or more of the camera devices 20 are capable of confirming fraudulent behavior by the customer among the images captured by the multiple camera devices 20. In this case, the first output unit 115 outputs the first alert only when fraudulent behavior by the customer can be detected. This prevents the first alert from being erroneously output even when the customer is not engaging in fraudulent behavior.

[0098] In the monitoring system 1 of this embodiment, the fraudulent behavior detection unit 113 in the control unit 110 of the management device 100 detects fraudulent behavior by customers based on images captured by the image capture device 20 and information from the checkout device 30, but this is not limited to this. For example, the image capture device 20 may detect fraudulent behavior by customers based on captured images, or the checkout device 30 may detect fraudulent behavior by customers based on their operations on the checkout device 30. In this case, the image capture device 20 or the checkout device 30 is an example of a detection unit. [Explanation of symbols]

[0099] 1...Monitoring system, 10...Store terminal, 20...Photographing device, 30...Accounting device, 40...POS register, 50...Change dispenser, 100...Management device, 110...Control unit, 111...Acquisition unit, 112...Store entry detection unit, 113...Fraudulent behavior detection unit, 114...Determination unit, 115...First output unit, 116...Second output unit

Claims

1. a detection unit for detecting fraudulent behavior of customers regarding purchases of products at a store; a determination unit that determines whether the amount of fraudulent behavior of the customer detected by the detection unit satisfies a predetermined standard; an output unit that outputs a first alert to a store terminal when the determination unit determines that the amount of fraudulent behavior of the customer satisfies the criterion; and An information processing system comprising:

2. The information processing system of claim 1, wherein the determination unit determines that the criterion is met if the fraudulent behavior of the customer detected by the detection unit is a predetermined number of times or more, and determines that the criterion is not met if the fraudulent behavior of the customer detected by the detection unit has not reached the predetermined number of times.

3. The information processing system according to claim 1, wherein the determination unit determines whether the amount of fraudulent behavior, including fraudulent behavior during past settlement processes, of a customer for whom fraudulent behavior has been detected by the detection unit satisfies the criterion.

4. When the detection unit detects fraudulent behavior of a customer, the detection unit acquires information regarding the type of fraudulent behavior; The information processing system according to claim 1 , wherein the determination unit determines whether the amount of fraudulent behavior of the customer satisfies the criterion for each type of fraudulent behavior.

5. The information processing system of claim 4, wherein the determination unit determines that the criterion is met if the fraudulent behavior of the customer detected by the detection unit is a second or subsequent fraudulent behavior, and determines that the criterion is met if the fraudulent behavior of the customer detected by the detection unit is of a predetermined type, even if the fraudulent behavior of the customer is not a second or subsequent fraudulent behavior.

6. a second detection unit for detecting a customer entering the store; a second output unit that, when a customer with a history of fraudulent behavior of a predetermined type is detected by the second detection unit, outputs a second alert to the store terminal at the time the customer is detected, indicating that a customer with the history has been detected; The information processing system according to claim 1 , further comprising:

7. a storage unit for storing count information, which is information regarding the number of times fraudulent behavior has been committed, for each customer and for each type of fraudulent behavior; 2. The information processing system according to claim 1, wherein the determination unit determines, for each type of fraudulent behavior, whether the amount of fraudulent behavior of the customer detected by the detection unit satisfies the criterion based on the number information.

8. a storage unit for storing, for each customer, frequency information relating to the number of times fraudulent behavior has been committed; the determination unit determines whether the amount of fraudulent behavior of the customer detected by the detection unit satisfies the criterion based on the number information; 2. The information processing system according to claim 1, wherein the storage unit initializes the number of times information when the customer's behavior in the store satisfies a predetermined condition.

9. 2. The information processing system according to claim 1, wherein the output unit outputs the first alert not only to the store terminal but also to a customer terminal used by a customer.

10. 2. The information processing system according to claim 1, wherein the output unit outputs information indicating the severity of the fraudulent behavior of the customer to the store terminal in addition to the first alert.

11. For computers, The ability to detect fraudulent customer behavior regarding in-store purchases; determining whether the amount of detected fraudulent customer behavior meets a predetermined standard; a function of outputting a first alert to a store terminal indicating that fraudulent behavior has been committed when the amount of fraudulent behavior of the customer satisfies the criterion; A program to make this happen.

12. A monitoring method in a monitoring system, comprising: Detect fraudulent customer behavior regarding in-store purchases, determining whether the amount of detected customer fraudulent behavior meets a predetermined criterion; If the amount of fraudulent behavior by the customer satisfies the criterion, a first alert indicating that fraudulent behavior has been committed is output to the store terminal. Monitoring method.

Images