Information processing device, information processing system, and computer-readable storage medium

Abstract

The present invention makes it possible to reduce a risk of issuing erroneous detection notification to a purchaser, and to prevent or suppress illegal behavior with efficiency and effectiveness. This information processing device is provided with a detection unit and a control unit. The detection unit detects an illegal act of a purchaser on a transaction processing device. The control unit controls the destination of notification issueing regaridng the illegal act on the basis of the accuracy of the detection of the illegal act by the detection unit.

Description

Information processing device, information processing system, and computer-readable storage medium

[0001] An embodiment of the present invention relates to an information processing device, an information processing system, and a computer-readable storage medium storing a program for causing a computer to function as an information processing device.

[0002] In recent years, due to labor shortages and rising labor costs, self-checkout systems, in which customers register and pay for their purchases themselves, have become widespread.

[0003] On the other hand, since self-checkout systems require customers to register and pay themselves, there are many cases of products not being scanned and fraudulent activity. Therefore, systems are being developed that analyze camera footage and use AI to automatically detect and alert customers to fraudulent activity.

[0004] However, such systems using AI (artificial intelligence) carry the risk of falsely detecting non-fraudulent transactions as fraudulent. Because of this risk of falsely detecting non-fraudulent transactions as fraudulent, if AI detection results are directly sent to the customer, false positives can lead to customer complaints. Therefore, due to the risk of false positives, the alert cannot be sent to the customer, but is instead sent to the store clerk's monitoring terminal, where the clerk must confirm the fraudulent activity on the attendant screen of the monitoring terminal that receives the fraudulent activity detection alert. This requires the intervention of the clerk, resulting in reduced man-hour productivity. Furthermore, in practice, it is difficult for clerks to constantly check for fraudulent activity on the attendant screen, making it difficult to effectively prevent fraudulent behavior.

[0005] Japanese Patent Application Publication No. 2021-135620

[0006] The problem that the embodiments of the present invention aim to solve is to provide an information processing device and an information processing system that can reduce the risk of false positive alerts to purchasers and efficiently and effectively prevent or deter fraudulent behavior.

[0007] FIG. 1 is a schematic diagram showing a self-checkout system as an information processing system according to the first embodiment, including a detection terminal to which an information processing device according to the first embodiment is applied. FIG. 2 is a perspective view showing an example of the external configuration of a self-checkout POS terminal included in the self-checkout system. FIG. 3 is a block diagram showing an example of the circuit configuration of the main components of the self-checkout POS terminal. FIG. 4 is a block diagram showing an example of the circuit configuration of the main components of the detection terminal. FIG. 5 is a diagram showing an example of the contents stored in a level setting memory unit of the detection terminal. FIG. 6 is a diagram showing an example of the contents stored in a type setting memory unit of the detection terminal. FIG. 7 is a diagram for explaining an example of fraudulent activity determination. FIG. 8 is a diagram showing a first portion of a sequence diagram for explaining the fraud detection operation of a customer in the self-checkout system. FIG. 9 is a diagram showing a second portion of a sequence diagram for explaining the fraud detection operation of a customer in the self-checkout system. FIG. 10 is a diagram showing a first portion of a series of flowcharts illustrating an example of the main steps of information processing executed by a processor of the detection terminal. FIG. 11 is a diagram showing a second portion of a series of flowcharts illustrating an example of the main steps of information processing executed by a processor of the detection terminal. FIG. 12 is a schematic diagram showing an example of a registration screen display including a display screen of the self-checkout POS terminal. FIG. 13 is a schematic diagram showing an example of an attendant screen of a monitoring terminal included in a self-checkout system. FIG. 14 is a schematic diagram showing an example of a display screen at a self-checkout POS terminal when fraudulent activity is detected. FIG. 15 is a schematic diagram showing a self-checkout system as an information processing system according to a second embodiment, including a detection server to which an information processing device according to a second embodiment is applied. FIG. 16 is a schematic diagram showing a self-checkout system as an information processing system according to a third embodiment. FIG. 17 is a perspective view showing an example of the external configuration of a self-checkout POS terminal included in a self-checkout system. FIG. 18 is a diagram showing a first portion of a sequence diagram for explaining the operation of detecting fraud by a customer in the self-checkout system. FIG. 19 is a diagram showing a second portion of a sequence diagram for explaining the operation of detecting fraud by a customer in the self-checkout system. FIG. 20 is a schematic diagram showing an example of a display screen at a customer display when fraudulent activity is detected.FIG. 21 is a schematic diagram for explaining sensors used in a self-checkout system as an information processing system according to the fourth embodiment. Embodiment

[0008] In one embodiment, the information processing device includes a detection unit and a control unit. The detection unit detects fraudulent activity by a purchaser against the transaction processing device. The control unit controls the destination of a fraudulent activity report based on the accuracy of the detection unit's detection of the fraudulent activity.

[0009] Hereinafter, an embodiment of an information processing apparatus will be described with reference to the drawings.

[0010] [First Embodiment] Fig. 1 is a schematic diagram showing a self-checkout system SYS as an information processing system according to the first embodiment. The self-checkout system SYS includes multiple self-service POS (Point of Sales) terminals 1, multiple cameras 2, multiple detection terminals 3 to which the information processing device according to the first embodiment is applied, a monitoring terminal 4, and a recording device 5. The self-checkout system SYS connects the self-service POS (Point of Sales) terminals 1, cameras 2, detection terminals 3, monitoring terminals 4, and recording device 5 via a communication network 6 such as a LAN (Local Area Network). Each self-service POS terminal 1 is associated with one camera 2 and one detection terminal 3, and a cash register system RS is composed of the self-service POS terminal 1, camera 2, and detection terminal 3. Each cash register system RS is installed at a store's checkout area.

[0011] The user of the self-service POS terminal 1 is a customer who has finished shopping. A customer who places purchased items in a shopping cart or similar in a sales area where items are displayed and proceeds to the checkout area operates the self-service POS terminal 1 to self-pay for the purchased items, i.e., to settle the transaction from product registration to payment. The self-service POS terminal 1 is an example of a fully self-service transaction processing device. This "transaction processing device" includes a payment device, a product registration device, and a self-service POS terminal. The payment device has a payment function and is a device where the customer himself / herself makes a payment based on registered product transaction data. The product registration device has a product registration function and is a device where the customer himself / herself registers products. The self-service POS terminal has a product registration function and a payment function and is a device where the customer himself / herself registers products and makes a payment based on the registered transaction data. The self-service POS terminal 1 transmits a POS event, which is event information indicating the customer's operation, to the corresponding detection terminal 3 via the communication network 6 based on a pre-established correspondence relationship with the detection terminal 3. The POS event is an example of a second event that indicates the behavior of a purchaser and is notified from the self-checkout POS terminal 1 to the detection terminal 3 .

[0012] The camera 2 captures images of the actions of a customer operating the self-checkout POS terminal 1, and transmits the captured camera images to the corresponding detection terminal 3 via the communication network 6 based on a pre-set correspondence between the detection terminal 3 and the camera 2. While the camera 2 transmits the camera images via the communication network 6, it is also possible to transmit the camera images directly to the detection terminal 3 via a USB (Universal Serial Bus) cable, for example. In this case, it is not necessary to set a correspondence between the camera 2 and the detection terminal 3. The camera 2 can also transmit the camera images to the recording device 5 via the communication network 6.

[0013] The detection terminal 3, to which the information processing device according to the first embodiment is applied, is equipped with an AI fraud detection engine. The detection terminal 3 determines a camera event related to the customer or the self-service POS terminal 1 based on the camera image from the camera 2. Specifically, the camera 2 is an example of a sensor that detects the customer's behavior, and the camera event is an example of a first event determined based on the output of this sensor. The detection terminal 3 then detects fraudulent activity by the customer based on the determined camera event and the POS event from the self-service POS terminal 1, and reports the detected fraud to the monitoring terminal 4 and the corresponding self-service POS terminal 1 via the communication network 6. The detection terminal 3 controls the destination of this report based on the accuracy of the fraudulent activity detection. Even if the detection terminal 3 detects fraudulent activity, the detection result has only a certain degree of reliability because the determination is made by AI, not by a human. The detection terminal 3 may detect not only zero possibility of false positive, meaning that fraudulent activity is definitely occurring, but also high possibility of false positive, meaning that it may determine that fraudulent activity is occurring even when in fact no fraudulent activity is occurring. In other words, the detection results by AI may be accurate or inaccurate depending on the behavior of the purchaser. Therefore, the detection terminal 3 calculates the "accuracy of fraudulent activity detection," which indicates the degree of accuracy of the result of detecting fraudulent activity, and controls the notification destination based on this. Specific examples of calculating this "accuracy of fraudulent activity detection" will be described later. Details of the control of notification will also be described later.

[0014] The monitoring terminal 4 is a terminal operated by a store attendant who monitors multiple self-checkout POS terminals 1 connected via the communication network 6. The monitoring terminal 4 is an example of a first device for the monitor. The monitoring terminal 4 can be wirelessly connected to the communication network 6 and can also be a tablet terminal that the store attendant can carry.

[0015] The recording device 5 is a storage device such as a NAS (Network Attached Storage) connected to the communication network 6. The recording device 5 records the camera images from each camera 2.

[0016] FIG. 2 is a perspective view showing the external configuration of the self-checkout POS terminal 1, and FIG. 3 is a block diagram showing the main circuit configuration of the self-checkout POS terminal 1. As shown in FIG.

[0017] As shown in FIG. 2 , the self-service POS terminal 1 includes a main body 101 installed on the floor, with a basket stand 102 and a bagging stand 103 on either side of the main body 101. The basket stand 102 is for customers coming from the sales floor to place baskets containing purchased items, and the bagging stand 103 is for customers to place plastic shopping bags or shopping bags (so-called "my bags") brought by customers. A display pole 104, a touch panel 105, and a camera pole 106 are attached to the top of the main body 101. Customers work by standing in front of the main body 101 in FIG. 2 so that they can see the screen of the touch panel 105. Therefore, from the customer's perspective, the basket stand 102 is on the right side of the main body 101, and the bagging stand 103 is on the left side. In the following explanation, the side where the purchaser stands is referred to as the front of the main body 101, the side where the bagging table 103 is installed is referred to as the left side of the main body 101, and the side where the basket table 102 is installed is referred to as the right side of the main body 101.

[0018] The display pole 104 has a light-emitting unit 107 at its tip that selectively emits light, for example, in blue or red. The display pole 104 displays the status of the self-service POS terminal 1, such as standby, operating, calling, or error, depending on the color of the light emitted by the light-emitting unit 107.

[0019] The touch panel 105 is composed of a display for displaying various screens related to transactions, such as registration and accounting, to the user operating the self-checkout POS terminal 1, and a touch sensor for detecting touch inputs made by the user on the screen. In the self-checkout POS terminal 1, the user is usually a purchaser. The touch panel 105 can also display information transmitted from the detection terminal 3 via the communication network 6. The self-checkout POS terminal 1 equipped with the touch panel 105 is an example of a second device for a purchaser.

[0020] Camera 2 is attached to camera pole 106. The mounting position and orientation of camera 2 are set so that the angle of view, i.e., the imaging area, includes the range of movement of the user when operating self-service POS terminal 1, including the basket stand 102 and the bagging stand 103, as shown by the dashed line in Fig. 2. More specifically, camera 2 is set so that its imaging area can capture images from above of at least the opening of the shopping basket placed on basket stand 102, the opening for inserting items into the plastic bag or my bag placed on the bagging stand 103, and the path of the items taken from the shopping basket to the point where they are inserted into the plastic bag or my bag placed on the bagging stand 103.

[0021] A reading window 109 for a scanner 108 (see FIG. 3), a card insertion slot 111 for a card reader 110 (see FIG. 3), and an issuing slot 113 for receipts printed by a printer 112 (see FIG. 3) are formed on the front of the main body 101. Furthermore, a coin insertion slot 115, a coin dispensing slot 116, a bill insertion slot 117, and a bill dispensing slot 118 for an automatic change dispenser 114 (see FIG. 3) are also formed on the front of the main body 101.

[0022] A communication cable 119 extends from the right side of the main body 101 to the outside, and a reader / writer 120 for electronic money media is connected to the tip of this communication cable 119. The reader / writer 120 is placed on a stand 121 provided on the upper right side of the main body 101.

[0023] As shown in Figure 3, the self-service POS terminal 1 includes the bagging table 103, touch panel 105, light-emitting unit 107, scanner 108, card reader 110, printer 112, automatic change dispenser 114, and reader / writer 120 described above, as well as a processor 122, main memory 123, auxiliary storage device 124, a communication interface 125, and a system transmission path 126. The system transmission path 126 includes an address bus, a data bus, control signal lines, and the like. The system transmission path 126 connects the processor 122 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 122, main memory 123, and auxiliary storage device 124 are connected via the system transmission path 126 to form the computer of the self-service POS terminal 1.

[0024] The processor 122 corresponds to the central part of the computer. The processor 122 controls each part to realize the various functions of the self-checkout POS terminal 1 in accordance with an operating system or application program. The processor 122 is, for example, a central processing unit (CPU). The processor 122 may be, for example, a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 122 may be a combination of these.

[0025] The main memory 123 corresponds to the main storage portion of the computer. The main memory 123 includes a nonvolatile memory area and a volatile memory area. The main memory 123 stores an operating system or application programs in the nonvolatile memory area. The main memory 123 may also store data required for the processor 122 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 123 uses the volatile memory area as a work area where data is rewritten by the processor 122 as appropriate. The nonvolatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory). For example, the main memory 123 stores a transaction file 127.

[0026] The transaction file 127 is a data file for storing data related to one commercial transaction processed by the self-checkout POS terminal 1. The transaction file 127 stores data such as the transaction number, purchased item data, total number of items, total amount, discount amount, and settlement amount. The transaction number is a series of numbers that is issued each time a commercial transaction is processed by the self-checkout POS terminal 1. The purchased item data is record data created for each item sold in a commercial transaction identified by the transaction number. Here, the purchased item data consists of items such as the product code, product name, price, number of items, and amount. The number of items is the number of items purchased for the product identified by the product code. The amount is the amount for the number of items purchased. The transaction file 127 can store multiple purchased item data. The total number of items is the sum of the points for each purchased item data. The total amount is the sum of the amounts for each purchased item data. The discount amount is the amount discounted from the total amount. The settlement amount is the total amount minus the discount amount.

[0027] The auxiliary storage device 124 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 124 may be an EEPROM (Electric Erasable Programmable Read-Only Memory), a HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device 124 stores data used by the processor 122 when performing various processes, data created by the processes in the processor 122, etc. The auxiliary storage device 124 may also store the application programs described above.

[0028] The communication interface 125 performs data communication in accordance with a preset communication protocol with an external device connected via the communication network 6. The external device is, for example, the detection terminal 3 or the like.

[0029] The scanner 108 reads a code symbol from a product held over the reading window 109. Each product sold in a store is affixed with a code symbol that encodes a product ID or the like to identify the product. The code symbol is, for example, a barcode. The code symbol may also be, for example, a two-dimensional data code. The scanner 108 may be of a type that reads a code symbol by scanning with laser light, or of a type that reads a code symbol from an image captured by an imaging device.

[0030] The card reader 110 reads card data recorded on a card medium such as a credit card, a point card, etc. The card reader 110 pulls the card medium inserted into the card insertion slot 111 into the main body 101, reads the card data, and then ejects the card from the card insertion slot 111.

[0031] The printer 112 prints receipt data and other information indicating the details of the transaction on receipt paper. The receipt paper on which the receipt data is printed is discharged from the issuing port 113, cut by a cutter (not shown), and issued as a receipt or proof of purchase.

[0032] The automatic change dispenser 114 includes a coin unit 128 and a bill unit 129. The coin unit 128 sorts coins inserted into the coin insertion slot 115 one by one, identifies the denomination, and stores them in a safe by denomination. The coin unit 128, for example, based on change data, removes coins of the appropriate denomination from the safe and dispenses them to the coin dispensing outlet 116. The bill unit 129 sorts banknotes inserted into the bill insertion slot 117 one by one, identifies the denomination, and stores them in the safe by denomination. The bill unit 129, for example, based on change data, removes banknotes of the appropriate denomination from the safe and dispenses them to the bill dispensing outlet 118.

[0033] The reader / writer 120 reads and rewrites electronic money recorded on an electronic money medium, such as a contactless IC card. The electronic money medium may also be an electronic device such as a smartphone or a tablet terminal.

[0034] 4 is a block diagram showing an example of the main circuit configuration of the detection terminal 3. As shown in FIG. 4, the detection terminal 3 includes a processor 31, a main memory 32, an auxiliary storage device 33, a communication interface 34, and a system transmission path 35. The system transmission path 35 includes an address bus, a data bus, a control signal line, etc. The system transmission path 35 connects the processor 31 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 31, the main memory 32, and the auxiliary storage device 33 are connected via the system transmission path 35 to form a computer of the detection terminal 3. The computer is an example of an information processing device according to the first embodiment.

[0035] The processor 31 corresponds to the central part of the computer. The processor 31 controls each part to realize various functions of the detection terminal 3 in accordance with an operating system or an application program. The processor 31 is, for example, a CPU. The processor 31 may be a multi-core / multi-threaded processor, and can execute multiple processes in parallel. The processor 31 may be, for example, an MPU, SoC, DSP, GPU, ASIC, PLD, or FPGA. Alternatively, the processor 31 may be a combination of two or more of these.

[0036] The main memory 32 corresponds to the main storage portion of the computer. The main memory 32 includes a nonvolatile memory area and a volatile memory area. The main memory 32 stores an operating system or an application program in the nonvolatile memory area. The application program includes, for example, an information processing program for causing the computer to function as the information processing device according to the first embodiment and a program for implementing an AI fraud detection engine. The nonvolatile memory area is an example of a computer-readable storage medium that stores an information processing program for causing the computer to function as the information processing device according to the first embodiment. The main memory 32 may store data necessary for the processor 31 to execute processes for controlling each component in a nonvolatile or volatile memory area. The main memory 32 uses the volatile memory area as a work area where data can be rewritten by the processor 31 as needed. The nonvolatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

[0037] The auxiliary storage device 33 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 33 can be an EEPROM, HDD, or SSD. The auxiliary storage device 33 stores data used by the processor 31 when performing various processes, data created by the processes performed by the processor 31, and the like. The auxiliary storage device 33 includes, for example, a level setting storage unit 331, a type setting storage unit 332, a camera image storage unit 333, and an event log storage unit 334. The level setting storage unit 331 stores multiple notification destination settings based on the accuracy of fraudulent activity detection, and the type setting storage unit 332 stores settings regarding which notification destination setting stored in the level setting storage unit 331 should be adopted depending on the type of fraudulent activity. The settings of the level setting storage unit 331 and the type setting storage unit 332 can be arbitrarily set according to the operations of the store or the company that operates the store. The camera image storage unit 333 stores camera images from the corresponding camera 2, and the event log storage unit 334 stores logs of POS events from the self-service POS terminal 1 and logs of camera events determined from the camera images. The level setting storage unit 331, type setting storage unit 332, camera image storage unit 333, and event log storage unit 334 can delete their stored contents as needed, for example, at any time, such as outside of store business hours. The auxiliary storage device 33 may also store the application program. In this case, the auxiliary storage device 33 is an example of a computer-readable storage medium that stores an information processing program for causing a computer to function as the information processing device according to the first embodiment.

[0038] The communication interface 34 performs data communication in accordance with a preset communication protocol with external devices connected via the communication network 6. Examples of external devices include the self-checkout POS terminal 1, the camera 2, the monitoring terminal 4, and the recording device 5.

[0039] FIG. 5 is a diagram showing an example of the contents stored in the level setting storage unit 331. As described above, the level setting storage unit 331 stores settings for notification destinations based on the accuracy of fraudulent activity detection. As shown in FIG. 5, for example, for each pattern ID, the level setting storage unit 331 stores settings for "fraud probability," "flag in video log," "notify attendant," and "alert to purchaser" for fraud levels of behavior on a five-level scale from "1" to "5." Note that while this example uses five fraud levels, the number of levels is of course not limited to this.

[0040] When the AI ​​fraud detection engine determines that fraud has occurred based on camera events based on camera footage from camera 2 and POS events from self-checkout POS terminal 1, it calculates a fraud probability, which indicates the probability that the fraud is actually occurring, as the "accuracy of fraud detection." For example, but not limited to, the degree of match with learning data from past camera footage can be calculated as the fraud probability of the determined fraud. By setting this fraud probability value as the value for each fraud level, it is possible to arbitrarily define a range of fraud levels. The fraud probability value set corresponding to each fraud level is a lower limit value. Therefore, for example, in the case of pattern ID "1," the fraud level "1: definitely fraud" has a fraud probability of "100%," the fraud level "2: very likely fraud" has a fraud probability of "75% or more but less than 100%, the fraud level "3: possible fraud" has a fraud probability of "50% or more but less than 75%, the fraud level "4: slight possibility of fraud" has a fraud probability of "25% or more but less than 50%, and the fraud level "5: not fraud" has a fraud probability of "0% or more but less than 25%." In the case of pattern ID "3," by setting the fraud level "3" to 40%, the fraud level "3: possible fraud" has a fraud probability of "40% or more but less than 75%, and the fraud level "4: slight possibility of fraud" has a fraud probability of "25% or more but less than 40%."

[0041] The "flag for video log" indicates whether or not a flag indicating fraudulent activity is recorded in the log of camera footage being recorded in the recording device 5. If set to "Yes," the flag is recorded; if set to "Off," the flag is not recorded. This flag can be used to investigate fraudulent activity at a later date. For example, if a large amount of product loss is discovered at a specific time, such as at the end of the month during inventory, it is not necessary to investigate all of the large amount of camera footage recorded in the recording device 5; only the footage for which this flag is set can be examined. Therefore, as shown in FIG. 5, it is desirable to set this flag to record all footage even if there is even the slightest possibility of fraud, that is, even if there is a high risk of false positives.

[0042] "Notify attendant" indicates whether or not to notify the monitoring terminal 4 of fraudulent activity. This can also be set arbitrarily. Upon receiving this notification, the attendant clerk monitoring the self-service POS terminal 1, who is the operator of the monitoring terminal 4, can confirm the suspicious behavior of the customer. Based on the results of this confirmation, the clerk will then decide whether to actually contact the customer. Therefore, since this notification is intended to prompt the clerk to conduct an investigation and does not immediately lead to a complaint from the customer, it is set to be performed even if there is a certain risk of false detection. The minimum fraud probability value set when notifying the attendant is set is an example of a first threshold for determining whether or not to issue a fraudulent activity alert to the monitoring terminal 4, which is an example of a first terminal.

[0043] The notification to the customer also indicates whether or not to issue a warning to the corresponding self-checkout POS terminal 1. This can also be set arbitrarily. Issuing a warning to the customer can prevent fraud. However, since false positives can lead to customer complaints, it is desirable to limit notification to cases where the risk of false positives is low. The minimum fraud probability value set when a notification to the customer is issued is an example of a second threshold for determining whether or not to issue a fraud alert to the self-checkout POS terminal 1, an example of a second terminal. Here, since the second threshold is higher than the first threshold, the second terminal includes the monitoring terminal 4, an example of a first terminal. Note that, depending on the type of fraud, the customer may already be away from the self-checkout POS terminal 1 and may not be able to see the alert. Therefore, for such types of fraud, it is ineffective to issue a fraud alert at the self-checkout POS terminal 1. Therefore, in such cases, notification to the customer can be set to "not send," such as for a fraud level of "1: Definitely Fraud" for pattern IDs "2" and "3." Therefore, the second terminal does not necessarily include the self-checkout POS terminal 1 .

[0044] FIG. 6 is a diagram illustrating an example of the contents stored in the type setting storage unit 332. As described above, the type setting storage unit 332 stores settings regarding which notification destination setting stored in the level setting storage unit 331 should be adopted depending on the type of fraudulent activity. The level setting storage unit 331 stores "fraudulent activity" and "pattern ID" linked to the fraudulent activity ID. The pattern ID is the pattern ID in the level setting storage unit 331. In other words, the level setting storage unit 331 stores settings regarding which notification destination setting should be adopted for each type of fraudulent activity. In the example of FIG. 6, as "fraudulent activity," "packing items into a bag without registering them" is set for fraudulent activity ID "1," "leaving the store without completing the transaction" is set for fraudulent activity ID "2," and "leaving unregistered items in the cart and leaving the store after completing the transaction" is set for fraudulent activity ID "3."

[0045] Specifically, these fraudulent activities can be detected by the AI ​​fraud detection engine as follows. FIG. 7 is a diagram illustrating an example of fraudulent activity detection. The AI ​​fraud detection engine determines camera events, which are events related to the customer or the self-service POS terminal 1, based on camera video from the camera 2. For example, as shown in FIG. 7 , the fraud detection engine can determine camera events such as "removing product" and "remaining product in basket" based on video footage of the area near the basket stand 102. The fraud detection engine can also determine a camera event such as "scanning product" based on video footage of the area near the reading window 109 of the scanner 108, and a camera event such as "bagging" based on video footage of the area near the bagging stand 103. The fraud detection engine can also determine camera events such as "taking remaining product and leaving store" and "taking product and leaving store" based on video footage of the area near the basket stand 102 and video footage in front of the main body 101. The fraud detection engine can then determine whether the purchaser has behaved normally or committed fraud by combining the POS event, which is the event information from the self-service POS terminal 1, with the camera events determined before and after it.

[0046] Although the example shown here is one in which fraudulent activity is determined based on a combination of a POS event and camera events determined before and after it, it goes without saying that there are also types of fraudulent activity that can be determined as legitimate based on a combination of a POS event and a camera event before it, or conversely, a combination of a POS event and a camera event after it, or a combination of two camera events before and after a POS event.

[0047] The operation of the self-checkout system SYS and the detection terminal 3 having such a configuration will be described below.

[0048] 8 and 9 are sequence diagrams illustrating the operation of detecting fraud by a customer in the self-checkout system SYS. As shown in FIG. 8, the camera 2 captures images (step S1) and transmits the captured camera footage to the corresponding detection terminal 3 and recording device 5 (step S2). For example, the camera 2 may be equipped with a human detection function and transmit the captured camera footage while a customer is captured in the captured camera footage. Alternatively, the camera 2 may be equipped with a moving object detection function instead of the human detection function and transmit the camera footage while a moving object is captured in the camera footage. Alternatively, the camera 2 may not have such a function, but instead may have a separate human detection sensor or the like installed in the self-checkout POS terminal 1, and capture and transmit the camera footage only while a person is detected in the sensor output. This allows the camera 2 to transmit the camera footage for each customer transaction. The camera 2 does not necessarily have to transmit the camera footage to both the corresponding detection terminal 3 and recording device 5. For example, the camera 2 may be configured to transmit to the corresponding detection terminal 3 but not to the recording device 5, or to transmit to the corresponding recording device 5 but not to the detection terminal 3. In other words, the camera video may be stored in either one of the devices.

[0049] The recording device 5 records the camera images transmitted from the cameras 2 (step S3). The recording device 5 records the camera images, for example, with a different file name for each camera 2. Of course, by creating separate recording folders for each camera 2, it is possible to allow the same file name to be used between cameras 2.

[0050] Similarly, the detection terminal 3 also stores the camera image transmitted from the camera 2 in the camera image storage unit 333 (step S4).

[0051] Furthermore, some kind of operation event occurs at the self-checkout POS terminal 1 by the purchaser (step S5). The self-checkout POS terminal 1 transmits a POS event indicating the content of this operation event to the corresponding detection terminal 3 and monitoring terminal 4 (step S6).

[0052] The monitoring terminal 4 records the POS event transmitted from the self-checkout POS terminal 1 (step S7).

[0053] Similarly, the detection terminal 3 also adds the POS event transmitted from the self-checkout POS terminal 1 to the event log stored in the event log storage unit 334 (step S8). The detection terminal 3 then creates a display screen to be displayed on the corresponding self-checkout POS terminal 1 (step S9). For example, a message for each POS event is prepared in advance in the detection terminal 3, and the detection terminal 3 selects a message based on the POS event log stored in the event log storage unit 334. The detection terminal 3 creates a display screen including the message and the current camera image stored in the camera image storage unit 333. The detection terminal 3 then transmits the created display screen to the corresponding self-checkout POS terminal 1 (step S10).

[0054] The self-checkout POS terminal 1 displays the display screen transmitted from the detection terminal 3 (step S11). Note that the "camera image" included in the display screen here may be the camera image itself transmitted from the camera 2 and stored in the detection terminal 3. In other words, the detection terminal 3 may transmit the camera image itself captured by the camera 2 to the self-checkout POS terminal 1.

[0055] The detection terminal 3 also uses the AI ​​fraud detection engine to determine a camera event based on the camera video stored in the camera video storage unit 333 (step S12).The detection terminal 3 then stores this determined camera event in the event log storage unit 334 (step S13).

[0056] 9, the detection terminal 3 determines fraudulent activity by a purchaser based on the POS events and / or camera events stored in the event log storage unit 334, for example, based on a POS event and camera events occurring before and after the event, using an AI fraud detection engine (step S14).The detection terminal 3 then determines the level of fraud based on the accuracy of the fraudulent activity detection, and determines where to report the fraudulent activity (step S15).

[0057] For example, if the determined fraudulent activity is "packing products without registering them" with a fraudulent activity ID of "1" shown in Fig. 6 and the determined fraud level is "4: slight possibility of fraud" shown in Fig. 5, only "flag to video log" will be performed. Therefore, the detection terminal 3 determines that the only destination to issue a report is the recording device 5. Therefore, the detection terminal 3 transmits a flag recording command to the recording device 5 (step S16).

[0058] When the flag recording command is transmitted from the detection terminal 3, the recording device 5 records a flag in the camera video acquired and recorded from the camera 2 corresponding to the detection terminal 3 (step S17).

[0059] For example, if the determined fraudulent activity is "packing products without registering them" with a fraud ID of "1" as shown in FIG. 5 and the determined fraud level is "2: Very likely fraudulent activity" as shown in FIG. 5, the system will "flag a video log" and "notify an attendant." Therefore, the detection terminal 3 determines that the monitoring terminal 4 and the recording device 5 are to be the destinations for the notification. In this case, the detection terminal 3 sends a flag recording command to the recording device 5 (step S16). The detection terminal 3 also creates a fraud notification screen (step S18) and transmits it to the monitoring terminal 4 (step S19). This fraud notification screen may include, for example, the details of the fraudulent activity and video and images for confirmation. To create such a screen, the detection terminal 3 cuts out, from the camera video stored in the camera video storage unit 333, approximately 10 seconds of video before and after the occurrence of the POS event used to determine the fraudulent activity, and resizes it as necessary to create a fraud confirmation video. The detection terminal 3 also creates a still image at a predetermined timing for each fraudulent act from the created fraud confirmation video, and uses the still image as a fraud confirmation image.

[0060] When the fraud notification screen is transmitted from the detection terminal 3, the monitoring terminal 4 displays the fraud notification screen, for example, as a pop-up (step S20).

[0061] For example, if the determined fraudulent activity is "packing products without registering them" with a fraud ID of "1" as shown in FIG. 5 and the determined fraud level is "1: definitely fraudulent" as shown in FIG. 5, the following actions are performed: "flag video log," "notify attendant," and "alert customer." Therefore, the detection terminal 3 determines that the notification destinations are the monitoring terminal 4, the recording device 5, and the self-checkout POS terminal 1. Therefore, in this case, the detection terminal 3 sends a flag recording command to the recording device 5 (step S16). The detection terminal 3 also creates a fraud notification screen (step S18) and sends it to the monitoring terminal 4 (step S19). The detection terminal 3 then creates a warning image (step S21) and sends it to the corresponding self-checkout POS terminal 1 (step S22).

[0062] When the warning image is transmitted from the detection terminal 3, the self-checkout POS terminal 1 displays the warning image (step S23).

[0063] The operation of the detection terminal 3 for realizing such an operation will be described below. Figures 10 and 11 are a series of flowcharts showing an example of the main steps of information processing executed by the processor 31 of the detection terminal 3. Unless otherwise specified, the processing operation of the processor 31 shown in Figures 10 and 11 transitions from ACTn (n is a natural number) to ACT(n+1). The steps shown in Figures 10 and 11 are just an example. There are no particular limitations on the steps as long as similar results can be obtained.

[0064] Here, it is assumed that the setting of the notification destination based on the accuracy of the detection of fraudulent activity in the level setting storage unit 331 and the setting of which notification destination setting stored in the level setting storage unit 331 to use according to the type of fraudulent activity in the type setting storage unit 332 have already been completed. These settings can be made in a variety of ways, such as by setting from the monitoring terminal 4 or by reading a setting file created separately and recorded on a recording medium such as a USB memory, and there are no particular limitations here.

[0065] In ACT 301 , the processor 31 acquires, via the communication interface 34 , the camera image transmitted from the corresponding camera 2 via the communication network 6 .

[0066] In ACT 302 , the processor 31 stores the acquired camera image in the camera image storage unit 333 .

[0067] In ACT 303, the processor 31 determines whether a POS event has occurred. Specifically, the processor 31 determines whether a POS event transmitted from the corresponding self-checkout POS terminal 1 via the communication network 6 has been acquired via the communication interface 34. If there is no POS event, the processor 31 determines NO in ACT 303 and proceeds to processing in ACT 307. If there is a POS event, the processor 31 determines YES in ACT 303 and proceeds to processing in ACT 304.

[0068] In ACT 304 , the processor 31 stores the POS event in the event log storage unit 334 .

[0069] In ACT 305 , the processor 31 creates a display screen to be displayed on the self-checkout POS terminal 1 .

[0070] In ACT 306, the processor 31 transmits the created display screen to the corresponding self-checkout POS terminal 1 via the communication network 6 through the communication interface 34. Thereafter, the processor 31 proceeds to the process of ACT 307.

[0071] The self-checkout POS terminal 1 receives this display screen and displays it. FIG. 12 is a schematic diagram showing an example of the display of a registration screen SC including the display screen 11 of the self-checkout POS terminal 1. As shown in FIG. 12, the registration screen SC is a screen used to register purchased items. The registration screen SC includes a registration list area PA1, a sales promotion list area PA2, a "Products without barcodes" touch button TB1, and a "Checkout" touch button TB2. The registration screen SC is generated by the processor 122 of the self-checkout POS terminal 1. The registration list area PA1 is an area for displaying a registration list of purchased items registered by the purchaser. By reading the code symbol attached to the product with the scanner 108, the purchased product data corresponding to the code symbol is entered into the self-checkout POS terminal 1, i.e., product registration is performed. The sales promotion list area PA2 displays multiple pieces of sales promotion information, three in the example of FIG. 12. The "Products without barcodes" touch button TB1 is used to register products without barcodes, such as fresh foods. In response to touching this touch button TB1, the self-checkout POS terminal 1 displays touch buttons corresponding to products without code symbols, so-called product buttons, and the customer can register the product by touching the product button corresponding to the purchased product. The [Checkout] touch button TB2 is used by the customer to declare the checkout of the purchased product. The self-checkout POS terminal 1 displays the received display screen 11 in a predetermined position relative to the registration screen SC, on the left side in the example of Figure 12. The display screen 11 includes a message display area 12 and a camera image display area 13. The message display area 12 displays a message corresponding to the POS event. The camera image display area 13 displays the current camera image.

[0072] In ACT 307, the processor 31 determines a camera event based on the camera footage stored in the camera footage storage unit 333 using the AI ​​fraud detection engine.

[0073] In ACT308 , the processor 31 stores the determined camera event in the event log storage unit 334 .

[0074] In ACT 309, processor 31 uses an AI fraud detection engine to determine fraudulent behavior by a purchaser based on POS events and / or camera events stored in event log storage unit 334. Thus, processor 31 is an example of a detection unit that detects fraudulent behavior by a purchaser against self-checkout POS terminal 1, which is an example of a transaction processing device.

[0075] In ACT 310, the processor 31 calculates a fraud probability that indicates the accuracy of the detection of this fraudulent activity.

[0076] In ACT 311, the processor 31 determines the fraud level based on the determined type of fraudulent activity and the fraud probability. That is, the processor 31 acquires a pattern ID corresponding to the determined type of fraudulent activity from the type setting storage unit 332, and determines the fraud level corresponding to the calculated fraud probability by referring to the setting of the pattern ID stored in the level setting storage unit 331.

[0077] In ACT 312, the processor 31 determines whether or not flag recording is necessary, that is, whether or not the determined fraud level indicates "flag to video log." If flag recording is not necessary, the processor 31 determines "NO" in ACT 312 and proceeds to processing in ACT 314. If flag recording is necessary, the processor 31 determines "YES" in ACT 312 and proceeds to processing in ACT 313.

[0078] In ACT 313, the processor 31 transmits a flag recording command to the recording device 5 via the communication interface 34 and the communication network 6. Thereafter, the processor 31 proceeds to the process of ACT 314.

[0079] In ACT 314, the processor 31 determines whether or not a fraud notification is necessary, that is, whether or not the determined fraud level indicates that "notification to an attendant" should be performed. If a fraud notification is not necessary, the processor 31 determines "NO" in ACT 314 and proceeds to processing in ACT 317. If a fraud notification is necessary, the processor 31 determines "YES" in ACT 314 and proceeds to processing in ACT 315.

[0080] In ACT 315, the processor 31 creates a fraud notification screen.

[0081] In ACT 316, the processor 31 transmits the created fraud notification screen to the monitoring terminal 4 via the communication interface 34 and the communication network 6. Thereafter, the processor 31 proceeds to the process of ACT 314.

[0082] FIG. 13 is a schematic diagram showing an example of an attendant screen of the monitoring terminal 4. The display screen of the monitoring terminal 4 is a touch panel 41 consisting of a display for displaying various screens to the attendant store clerk and a touch sensor for detecting touch inputs on the screen by the clerk. As shown in FIG. 13, the display screen includes a register list display area 42, an event log display area 43, and a notification display area 44. The register list display area 42 is an area for displaying a register list. The example in FIG. 13 shows a case where six self-service POS terminals 1, i.e., six register systems RS, are installed in the checkout area of ​​a store. Registers without a customer are displayed in a dimmed light. The event log display area 43 is an area for displaying a POS event log. Registers displaying a POS event log are highlighted in the register list display area 42. Touching a register in the register list display area 42 displays the POS event log for that register in the event log display area 43. The notification display area 44 is an area for displaying a fraud notification screen 45 from the detection terminal 3. When the fraud notification screen 45 is received, it is displayed as a pop-up.

[0083] The fraud notification screen 45 includes a notification content display 451, a fraud confirmation video 452, a fraud confirmation image 453, a video play button 454, and a check box 455. The notification content display 451 may include the register number, a message indicating the fraud content, the time of occurrence, etc. The fraud confirmation video 452 and the fraud confirmation image 453 are as described above. Touching the video play button 454 plays the fraud confirmation video 452. The check boxes 455 are designed to be checked in response to touch operations. When all the check boxes 455 on the fraud notification screen 45 are checked, the display of the fraud notification screen 45 is terminated. Conversely, the fraud notification screen 45 will continue to be displayed unless the store clerk operating the monitoring terminal 4 checks all the check boxes 455.

[0084] When all check boxes 455 on the fraud notification screen 45 are checked and the display of the fraud notification screen 45 is terminated, the monitoring terminal 4 can delete the flag from the log of the corresponding camera video recorded in the recording device 5. Of course, if there are multiple fraud notification screens 45 for one transaction, this flag deletion process is performed when the display of all fraud notification screens 45 is terminated.

[0085] In ACT 317, the processor 31 determines whether a customer notification is necessary, that is, whether the determined fraud level indicates that "notification to the customer" should be performed. If a customer notification is not necessary, the processor 31 determines "NO" in ACT 317 and proceeds to the processing of ACT 301. If a customer notification is necessary, the processor 31 determines "YES" in ACT 317 and proceeds to the processing of ACT 318.

[0086] In ACT 318, the processor 31 creates a warning image.

[0087] In ACT 319, the processor 31 transmits the created warning image to the corresponding self-checkout POS terminal 1 via the communication interface 34 and the communication network 6. After that, the processor 31 proceeds to the processing of ACT 301.

[0088] Fig. 14 is a schematic diagram showing an example of the display on the display screen 11 when fraudulent activity is detected in the self-checkout POS terminal 1. As shown in Fig. 14, when a warning image is received from the corresponding detection terminal 3, the self-checkout POS terminal 1 displays it as a pop-up warning image 14 in the message display area 12. Note that the message content of the warning image 14 shown in Fig. 14 is just an example, and it goes without saying that the message content will change depending on the type of fraudulent activity detected.

[0089] The processor 31 that executes the processes in ACT 310 to ACT 319 is an example of a control unit that controls the destination of a fraudulent activity report based on the accuracy of fraudulent activity detection.

[0090] As described above, the computer including the processor 31 of the detection terminal 3 is an information processing device according to the first embodiment, which detects fraudulent activity by a customer at the self-service POS terminal 1, which is a transaction processing device, and controls the destination of the fraudulent activity report based on the accuracy of the fraudulent activity detection. In this way, the detection terminal 3 to which the information processing device according to the first embodiment is applied controls the destination of the fraudulent activity report based on the accuracy of the fraudulent activity detection, thereby reducing the risk of false positive reports being issued to customers and enabling efficient and effective prevention or deterrence of fraudulent behavior.

[0091] Furthermore, the computer of the detection terminal 3 in the first embodiment controls the content of the fraudulent activity report depending on the report destination. Therefore, the detection terminal 3 in the first embodiment can issue a report that is appropriate for the report destination and does not cause discomfort to the purchaser, thereby reducing the risk to the purchaser. Furthermore, the report can be issued to store clerks who check for fraudulent activity, making it easier for them to check the purchaser's behavior, thereby increasing the ease with which the store clerks can check for fraudulent activity.

[0092] Furthermore, the computer of the detection terminal 3 in the first embodiment calculates a fraud probability indicating accuracy, as in the process of ACT 310, and if the fraud probability is equal to or greater than a predetermined first threshold, as in ACT 311 and ACT 314, issues a notification to a predetermined first terminal. If the fraud probability is equal to or greater than a predetermined second threshold that is higher than the first threshold, as in ACT 311 and ACT 317, issues a notification to a predetermined second terminal. Therefore, according to the detection terminal 3 in the first embodiment, unless the fraud probability is equal to or greater than the first threshold, a notification is not issued to the monitoring terminal 4, which is an example of a first terminal, so confirmation by a store clerk is not required when the probability of false detection is low. Furthermore, since a notification is only issued to the self-checkout POS terminal 1 when the fraud probability is equal to or greater than a second threshold that is higher than the first threshold, it is possible to prevent a customer from being notified despite a high risk of false detection.

[0093] Furthermore, in the computer of the detection terminal 3 in the first embodiment, the first and second thresholds and the first and second terminals are set in advance for each type of fraudulent activity and stored in the level setting storage unit 331 and the type setting storage unit 332. Therefore, according to the detection terminal 3 in the first embodiment, the first and second thresholds and the first and second terminals can be set in advance, making it possible to control alarm generation according to the standards required by the store or company, and efficiently preventing or deterring fraud.

[0094] Furthermore, the computer of the detection terminal 3 in the first embodiment determines the type of fraudulent activity based on a camera event, which is a first event determined based on camera footage output from the camera 2, which is an example of a sensor that detects the behavior of the customer, and a POS event, which is a second event indicating the behavior of the customer notified from the self-service POS terminal 1. Thus, the detection terminal 3 in the first embodiment can easily determine the type of fraudulent activity.

[0095] The self-checkout system SYS is an information processing system according to the first embodiment, and includes the detection terminal 3 according to the first embodiment, a monitoring terminal 4 as an example of a first device for a monitor that receives a fraudulent activity report from the detection terminal 3, and a self-checkout POS terminal 1 as an example of a second device for a customer that receives a fraudulent activity report from the detection terminal 3, where the first terminal includes the first device, and the second terminal includes at least the first device and may also include the second device. Thus, the self-checkout system SYS to which the information processing system according to the first embodiment is applied controls the issuance of fraudulent activity reports to the monitoring terminal 4, an example of the first device, and the self-checkout POS terminal 1, an example of the second device, based on the accuracy of fraudulent activity detection, thereby reducing the risk of false positive reports being issued to customers and enabling efficient and effective prevention or deterrence of fraudulent activity.

[0096] Second Embodiment Next, a second embodiment will be described. Note that the same reference numerals as in the first embodiment are used to designate the same configurations and operations as in the first embodiment, and descriptions thereof will be omitted.

[0097] 15 is a schematic configuration diagram showing a self-checkout system SYS as an information processing system according to the second embodiment, including a detection server 7 to which an information processing device according to the second embodiment is applied. As shown in Fig. 15, each register system RS in the self-checkout system SYS does not have a detection terminal 3 like in the first embodiment. Instead, the self-checkout system SYS is provided with a detection server 7 connected to a communication network 6. The detection server 7 is a computer configured to perform the functions of the detection terminal 3 in each register system RS.

[0098] Even with this configuration, the same effects as in the first embodiment can be achieved.

[0099] Third Embodiment Next, a third embodiment will be described. Note that the same reference numerals as in the first embodiment are used to denote the same configurations and operations as in the first embodiment, and descriptions thereof will be omitted.

[0100] FIG. 16 is a schematic diagram showing a self-checkout system SYS as an information processing system according to the third embodiment. Each register system RS included in the self-checkout system SYS includes a customer display 8 in addition to a self-checkout POS terminal 1, a camera 2, and a detection terminal 3. The customer display 8 is a small display visible to the customer operating the self-checkout POS terminal 1, such as a 7-inch display with a 7-inch screen size. The customer display 8 is an example of a second device for the customer that receives a fraudulent activity report from the detection terminal 3. The customer display 8 displays information transmitted from the detection terminal 3 via the communication network 6. The customer display 8 may also be connected directly to the detection terminal 3 without going through the communication network 6.

[0101] 17 is a perspective view showing an example of the external configuration of the self-checkout POS terminal 1 included in the self-checkout system SYS. The customer display 8 may be placed anywhere so long as it is visible to the customer. For example, as shown in FIG. 17, the customer display 8 may be placed adjacent to the touch panel 105 that displays the registration screen SC, etc., in this example, on the left side.

[0102] 18 and 19 are sequence diagrams illustrating the fraud detection operation of the self-checkout system SYS of this embodiment. Steps S1 to S9 shown in Fig. 18 are the same as those described in the first embodiment. Then, in step S10, the detection terminal 3 transmits the display screen created in step S9 to the corresponding customer display 8 in this embodiment.

[0103] The purchaser display 8 displays the display screen transmitted from the detection terminal 3 (step S11).

[0104] The subsequent steps S14 to S21 shown in Fig. 19 are also the same as those described in the first embodiment. Then, in step S22, the detection terminal 3 transmits the warning image created in step S21 to the corresponding purchaser display 8 in this embodiment.

[0105] When the warning image is transmitted from the detection terminal 3, the customer display 8 displays the warning image (step S23). Figure 20 is a schematic diagram showing an example of the display screen 81 of the customer display 8 when fraudulent activity is detected. The display screen 81 corresponds to the display screen 11 shown in Figure 14 and includes a message display area 82 corresponding to the message display area 12 on the display screen 11 and a camera image display area 83 corresponding to the camera image display area 13 on the display screen 11. When the customer display 8 receives a warning image from the corresponding detection terminal 3, it pops up a warning image 84 in the message display area 82.

[0106] In this manner, in this embodiment, when the detection terminal 3 determines to notify the consumer based on the accuracy of the fraudulent activity detection, it sets the destination of the fraudulent activity notification to the consumer display 8. Specifically, the processor 31 of the detection terminal 3 sets the destination of the display screen in ACT 306 shown in Fig. 10 and the display destination of the warning image in ACT 319 shown in Fig. 11 to the corresponding consumer display 8.

[0107] As described above, the self-checkout system SYS as an information processing system according to the third embodiment includes the detection terminal 3 in the first embodiment, the monitoring terminal 4 as an example of a first device for a monitor that receives a fraudulent activity report from the detection terminal 3, and the customer display 8 as an example of a second device for a customer that receives a fraudulent activity report from the detection terminal 3, where the first terminal includes the first device, and the second terminal includes at least the first device and may also include the second device. The self-checkout system SYS to which the information processing system according to the third embodiment is applied also controls the issuance of fraudulent activity reports to the monitoring terminal 4, an example of the first device, and the customer display 8, an example of the second device, based on the accuracy of fraudulent activity detection, thereby reducing the risk of false positive reports being issued to customers and enabling efficient and effective prevention or deterrence of fraudulent behavior.

[0108] [Fourth Embodiment] Next, a third embodiment will be described. In recent years, checkout systems have begun to be used in which, because information processing devices such as smartphones carried by customers are equipped with cameras, the cameras are used as scanners to scan purchased items and the purchase price is paid for by an electronic payment method such as a pre-registered credit card.

[0109] Therefore, an information processing program for causing such an information processing device to function as the information processing device according to the third embodiment is provided as an application program. In this case, the information processing device acquires the output of a sensor used for detecting fraudulent activity and a sensor that detects the behavior of a purchaser via short-range wireless communication such as Bluetooth (registered trademark).

[0110] 21 is a schematic diagram for explaining this sensor. In this embodiment, a surrounding area camera 21 and a cart area camera 22 attached to a shopping cart C are used as sensors.

[0111] The shopping cart C has caster members C1 for movement, a handle frame member C2, and a basket holder C3. The caster members C1 have four wheels for smooth movement on the floor. The basket holder C3 is located forward from the middle of the handle frame member C2. A shopping basket for storing merchandise can be placed on the basket holder C3 and caster members C1.

[0112] A surrounding area camera 21 is attached to the handle frame portion C2 of such a shopping cart C. The surrounding area camera 21 is, for example, a 360-degree camera, and is attached so that the entire circumference of the shopping cart C is within its angle of view, i.e., its imaging area, as shown by the dashed line in Fig. 21. As a result, no matter what position a customer takes in relation to the shopping cart C when they pick up an item from a shelf and scan it with an information processing device, the entire scene will be captured by the surrounding area camera 21.

[0113] The basket area camera 22 is attached to a pole C4 that stands upward from the handle frame portion C2 of the shopping cart C. The basket area camera 22 is attached in a position and oriented such that the basket receiving portion C3 is included in its angle of view, i.e., its photographing area, as shown by the dotted line in Fig. 21. More specifically, the basket area camera 22 is set so that its photographing area can capture an image of the inside of the shopping basket placed in the basket receiving portion C3 from diagonally above.

[0114] By using the surrounding area camera 21 and the car area camera 22 having such a shooting area as sensors, it becomes possible to use an information processing device such as a smartphone carried by a purchaser as an information processing device.

[0115] Therefore, even with this configuration, the same effects as in the first embodiment can be achieved.

[0116] Other Embodiments Although the embodiments of the information processing device have been described above, the embodiments are not limited to these.

[0117] For example, the detection terminal 3 to which the embodiment of the information processing device is applied does not need to include the fraud-confirming video in the notification screen that is sent to the monitoring terminal 4. Instead, the notification screen can include, for example, an image of the first frame of the fraud-confirming video. In this case, the monitoring terminal 4 can send a playback request to the detection terminal 3 in response to touching the video playback button 454 on the fraud notification screen 45, and the detection terminal 3 can send the fraud-confirming video to the monitoring terminal 4 in response to this playback request.

[0118] Furthermore, the detection terminal 3 is configured to create a warning image and send it to the corresponding self-checkout POS terminal 1 or customer display 8, but it may also send a warning ID instead of a warning image. In this case, by storing a warning image corresponding to the warning ID in the self-checkout POS terminal 1 or customer display 8 in advance, the self-checkout POS terminal 1 or customer display 8 can display a warning image corresponding to the warning ID sent from the detection terminal 3.

[0119] In addition, although the detection terminal 3 is configured to determine camera events, it is also possible to incorporate a memory capable of holding a certain amount of camera footage and an AI event detection engine into the camera 2 so that camera events can be determined on the camera 2 side.

[0120] Furthermore, the sensor for detecting fraudulent behavior by a purchaser using a transaction processing device such as the self-service POS terminal 1 is not limited to the camera 2, and it goes without saying that various sensors can be used, such as a weight sensor installed on the basket stand 102 and bagging stand 103 of the self-service POS terminal 1, or a temperature sensor that determines the surface temperature of the purchaser.

[0121] Furthermore, the flow of the information processing performed by the processor 31 described with reference to the flowchart is an example, and the order is not limited to this. For example, the order of ACT 312 to ACT 313, ACT 314 to ACT 316, and ACT 317 to ACT 319 in the flowchart of Fig. 11 may be reversed, or may be performed in parallel. In this way, the order of the processing may be changed or multiple processing may be performed in parallel, as long as there is no discrepancy with the preceding or subsequent processing.

[0122] The program that causes a computer to function as an information processing device according to the embodiment may be transferred in a state stored in an electronic device, or may be transferred in a state not stored in an electronic device. In the latter case, the program may be transferred via a network, or may be transferred in a state stored in a storage medium. The storage medium is a non-transitory tangible medium. The storage medium is a computer-readable medium. The storage medium may be in any form, such as a CD-ROM or a memory card, as long as it is capable of storing the program and is computer-readable.

[0123] It should be noted that the embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention, as well as within the scope of the inventions described in the claims and their equivalents.

Claims

1. An information processing device comprising: a detection unit that detects fraudulent activity by a purchaser against a transaction processing device; and a control unit that controls the destination of a report of the fraudulent activity based on the accuracy of the detection of the fraudulent activity by the detection unit.

2. The information processing device according to claim 1, wherein the control unit further controls the content of the report of the fraudulent activity according to the report destination.

3. The information processing device described in claim 2, wherein the control unit calculates a fraud probability indicating the accuracy, and if the fraud probability is equal to or greater than a predetermined first threshold, sets the destination of the report to a predetermined first terminal, and if the fraud probability is equal to or greater than a predetermined second threshold that is higher than the first threshold, sets the destination of the report to a predetermined second terminal.

4. The information processing device according to claim 3, wherein the first and second thresholds and the first and second terminals are set in advance for each type of fraudulent activity.

5. The information processing device described in claim 4, wherein the detection unit determines the type of fraudulent activity based on a first event determined based on the output of a sensor that detects the behavior of the purchaser, and a second event indicating the behavior of the purchaser notified by the transaction processing device.

6. An information processing system comprising: an information processing device that detects fraudulent activity by a purchaser against a transaction processing device and issues a report; a first device for a monitor that receives the report of the fraudulent activity from the information processing device; and a second device for the purchaser that receives the report of the fraudulent activity from the information processing device, wherein the information processing device comprises: a detection unit that detects the fraudulent activity by the purchaser against the transaction processing device; and a control unit that controls the destination of the report of the fraudulent activity based on the accuracy of the detection of the fraudulent activity by the detection unit and controls the content of the report of the fraudulent activity according to the destination, wherein the control unit calculates a fraud probability that indicates the accuracy, and if the fraud probability is equal to or greater than a predetermined first threshold, sets the destination of the report to a predetermined first terminal, and if the fraud probability is equal to or greater than a predetermined second threshold that is higher than the first threshold, sets the destination of the report to a predetermined second terminal, wherein the first terminal includes the first device, and the second terminal includes at least the first device and may include the second device.

7. A computer-readable storage medium storing a program for enabling a computer of an information processing device to perform the following functions: detect fraudulent behavior by a purchaser against a transaction processing device; and control the destination of the report of the fraudulent behavior based on the accuracy of the detection of the fraudulent behavior.

Images