Information processing device, program therefor, and information processing system

Abstract

To enable efficient and effective prevention or suppression of fraudulent acts while reducing the risk of false detection notifications to purchasers.SOLUTION: An information processing device comprises detection means and notification means. The detection means detects an action of a purchaser on a transaction processing device that requires notification. The notification means performs notification when the action detected by the detection means exceeds a threshold value of a determination criterion for the action in one transaction.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, a program therefor, and an information processing system. [Background technology]

[0002] In recent years, due to labor shortages and rising labor costs, self-checkout systems have become popular, allowing shoppers to complete everything from registering their purchases to paying for the transaction themselves. However, self-checkout systems raise concerns about fraudulent activities, such as bagging items without scanning them. To address this issue, systems have been developed that capture shoppers' behavior with cameras, analyze the footage using AI (Artificial Intelligence), and report any fraudulent activity that is detected.

[0003] However, such systems using AI video analysis may mistakenly detect non-fraudulent activity as fraudulent, and therefore, if a report is made every time fraud is detected, the risk of false positives being reported increases, leading to complaints from customers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-135620 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments of the present invention is to provide an information processing device that can efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers. [Means for solving the problem]

[0006] In one embodiment, the information processing device includes a detection means and a notification means. The detection means detects an action by a purchaser that requires notification to the transaction processing device. The notification means issues a notification when, in a single transaction, the action detected by the detection means exceeds a threshold value of a judgment criterion for that action. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a self-checkout system according to one embodiment. [Figure 2] FIG. 2 is a perspective view showing the external configuration of the self-service POS terminal. [Figure 3] FIG. 3 is a block diagram showing the main circuit configuration of the self-service POS terminal. [Figure 4] FIG. 4 is a block diagram showing an example of a main circuit configuration of the detector terminal. [Figure 5] FIG. 5 is a schematic diagram showing the main data structure of the setting table. [Figure 6] FIG. 6 is a schematic diagram showing the main data structure of the count table. [Figure 7] FIG. 7 is a sequence diagram illustrating the operation of the detection terminal for detecting and reporting a reporting action, along with the operation of the camera and self-service POS terminal of the same register system. [Figure 8] FIG. 8 is a flowchart showing the main steps of information processing executed by the processor of the detector terminal. [Figure 9] FIG. 9 is a schematic diagram showing the notification action detection operation for the notification action item "omission of scan." [Figure 10] FIG. 10 is a schematic diagram showing the notification action detection operation for the notification action item "weight error." [Figure 11] FIG. 11 is a schematic diagram showing the reporting action detection operation for the reporting action item "high-priced item error." [Figure 12] FIG. 12 is a schematic diagram showing the reporting action detection operation for the reporting action item "error in product under cart." [Figure 13]FIG. 13 is a schematic diagram showing the reporting behavior detection operation for the reporting behavior item "purchase quantity: large." [Figure 14] FIG. 14 shows an example of a registration screen displayed on the touch panel of the self-service POS terminal. [Figure 15] FIG. 15 shows an example of a registration screen of a self-service POS terminal that has received a report event for the reportable action item "omitted scanning." [Figure 16] FIG. 16 shows an example of a monitoring screen displayed on the display of the monitoring terminal. [Figure 17] FIG. 17 is a sequence diagram illustrating the operation of the detection terminal of the second embodiment regarding the detection and notification of a reporting action, along with the operation of the camera and self-checkout POS terminal of the same register system. [Figure 18] FIG. 18 is a flowchart showing the main steps of information processing executed by the processor of the detector terminal in the second embodiment. [Figure 19] FIG. 19 is an external configuration diagram of a self-service POS terminal according to another embodiment. [Figure 20] FIG. 20 is a diagram showing transitions of screens displayed on the customer display of the self-service POS terminal in another embodiment. [Figure 21] FIG. 21 is a schematic diagram showing the configuration of a self-checkout system according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of an information processing device that can efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers will be described with reference to the drawings.

[0009] [First embodiment] First, an embodiment of a self-checkout system to which an information processing device is applied will be described.

[0010] [Explanation of the self-checkout system] FIG. 1 is a schematic diagram showing a self-checkout system SYS according to one embodiment. The self-checkout system SYS includes multiple self-service POS terminals 1, multiple cameras 2, multiple detection terminals 3, a monitoring terminal 4, and a recording device 5. Specifically, the self-checkout system SYS configures a cash register system RS by associating each self-service POS terminal 1 with one camera 2 and one detection terminal 3. Each cash register system RS is installed at a checkout area of ​​a store. The self-checkout system SYS connects the self-service POS (Point of Sales) terminal 1, camera 2, and detection terminal 3 of each cash register system RS with the monitoring terminal 4 and recording device 5 via a communication network 6 such as a local area network (LAN). In this embodiment, the detection terminal 3 functions as an information processing device capable of efficiently and effectively preventing or deterring fraudulent activity while reducing the risk of false positive notifications to customers.

[0011] The user of the self-service POS terminal 1 is a customer who has finished shopping. After placing the purchased items in a shopping cart or the like in a sales area where items are displayed, the customer proceeds to the checkout area and operates the self-service POS terminal 1 to settle the transaction by himself, from registering the purchased items to paying for them. The self-service POS terminal 1 is an example of a fully self-service transaction processing device. The transaction processing device does not have to be a fully self-service device. The transaction processing device may be a payment device in which the registration of purchased items is performed on a separate device, and the customer himself performs operations related to paying for the items based on the data of the purchased items registered on that device to settle the transaction.

[0012] Based on a pre-established correspondence relationship with the detection terminal 3, the self-service POS terminal 1 transmits a POS event, which is event information indicating the operation content of the purchaser, to the corresponding detection terminal 3 via the communication network 6. The POS event is an example of a second event indicating the action content of the purchaser that is reported from the self-service POS terminal 1 to the detection terminal 3.

[0013] The camera 2 captures images of the actions of a customer operating the self-checkout POS terminal 1, and transmits the captured camera image to the corresponding detection terminal 3 via the communication network 6 based on a pre-set correspondence relationship with the detection terminal 3. Here, the image is transmitted via the communication network 6, but the image may also be transmitted directly from the camera 2 to the detection terminal 3 via, for example, a USB (Universal Serial Bus) cable. In this case, it is not necessary to set a correspondence relationship between the camera 2 and the detection terminal 3. The camera 2 can also transmit the camera image to the recording device 5 via the communication network 6.

[0014] The detection terminal 3, which functions as an information processing device, is equipped with an AI-based video analysis engine. The video analysis engine analyzes the camera video from the camera 2 to detect the behavior of the customer and identifies a camera event corresponding to that behavior. In other words, the camera 2 is an example of a sensor that detects the customer's behavior, and a camera event is an example of a first event identified based on the output of this sensor. The detection terminal 3 detects a customer's behavior that requires reporting based on the camera event identified by the video analysis engine and the POS event acquired from the self-service POS terminal 1. The detection terminal 3 may detect a reporting behavior based on a POS event, or may detect a reporting behavior based on a camera event. The detection terminal 3 reports the detected behavior, i.e., the reporting behavior, to the corresponding self-service POS terminal 1 via the communication network 6. The detection terminal 3 reports the reporting behavior not only to the corresponding self-service POS terminal 1 but also to the monitoring terminal 4.

[0015] The monitoring terminal 4 is a terminal operated by a store clerk who acts as a supervisor and monitors multiple self-service POS terminals 1 connected via the communication network 6. The monitoring terminal 4 has a display. The monitoring terminal 4 displays information necessary for monitoring multiple self-service POS terminals 1 on the display. The displayed information also includes information on reporting actions detected by the detection terminal 3.

[0016] The supervisory store clerk monitors the self-service POS terminal 1 based on the information displayed on the display. From the information on the reported behavior detected by the detection terminal 3, the clerk recognizes that a purchaser has committed an act that is suspected to be fraudulent. The supervisory store clerk is called, for example, an attendant. The monitoring terminal 4 is an example of a terminal for the supervisor. The monitoring terminal 4 may be any device that can be wirelessly connected to the communication network 6. For example, the monitoring terminal 4 may be a tablet terminal that the supervisory store clerk can carry around.

[0017] The recording device 5 is a storage device such as a NAS (Network Attached Storage) connected to a communication network 6. The recording device 5 records the camera images from each camera 2.

[0018] [Self-POS terminal explanation] FIG. 2 is a perspective view showing the exterior configuration of the self-service POS terminal 1, and FIG. 3 is a block diagram showing the main circuit configuration of the self-service POS terminal 1. As shown in FIG. 2, the self-service POS terminal 1 includes a main body 101 installed on the floor, with a basket stand 102 and a bagging stand 103 sandwiched between the main body 101. The basket stand 102 is for a customer coming from the sales floor to place a basket containing purchased items, and the bagging stand 103 is for a customer to place a plastic shopping bag or a shopping bag (so-called "my bag") brought by the customer. The main body 101 is equipped with a display pole 104, a touch panel 105, and a camera pole 106 on its upper portion. The customer works while standing in front of the main body 101 in FIG. 2 so that the screen of the touch panel 105 can be seen. Therefore, from the customer's perspective, the basket stand 102 is on the right side of the main body 101, and the bagging stand 103 is on the left side. In the following explanation, the side where the purchaser stands is referred to as the front of the main body 101, the side where the bagging table 103 is installed is referred to as the left side of the main body 101, and the side where the basket table 102 is installed is referred to as the right side of the main body 101.

[0019] The display pole 104 has a light emitting unit 107 at its tip that selectively emits light, for example, in blue or red. The display pole 104 displays the status of the self-service POS terminal 1, such as standby, operating, calling, or error, depending on the color of light emitted by the light emitting unit 107.

[0020] The touch panel 105 is composed of a display for displaying various screens related to transactions such as registration or accounting to the user operating the self-service POS terminal 1, and a touch sensor for detecting touch inputs made by the user on the screen. In the self-service POS terminal 1, the user is usually a purchaser. The touch panel 105 can also display information transmitted from the detection terminal 3 via the communication network 6.

[0021] Camera 2 is attached to the tip of camera pole 106. The angle of view, or imaging area, of camera 2 includes the basket stand 102 on the right side of main body 101 to bagging stand 103 on the left side, as shown by the dashed line in FIG. 2. In other words, the mounting position and orientation of camera 2 are set so that the imaging area includes the range of movement of the user when operating self-service POS terminal 1. More specifically, camera 2 is set so that its imaging area can capture images from above of at least the opening of a shopping basket placed on basket stand 102 or a shopping cart positioned on basket stand 102, the opening for inserting items into a plastic bag or personal bag placed on bagging stand 103, and the path of items taken from the shopping basket to be inserted into a plastic bag or personal bag placed on bagging stand 103.

[0022] The main body 101 has, on its front surface, a reading window 109 for a scanner 108 (see FIG. 3), a card insertion slot 111 for a card reader 110 (see FIG. 3), and an issuing slot 113 for receipts printed by a printer 112 (see FIG. 3). Furthermore, the main body 101 also has, on its front surface, a coin insertion slot 115, a coin dispensing slot 116, a bill insertion slot 117, and a bill dispensing slot 118 for an automatic change dispenser 114 (see FIG. 3).

[0023] In addition, a communication cable 119 extends from the right side of the main body 101 to the outside, and a payment terminal 120 for cashless payment is connected to the tip of this communication cable 119. The payment terminal 120 is placed on a stand 121 provided on the upper right side of the main body 101.

[0024] The bagging table 103 includes a scale unit 122. The scale unit 122 measures the total weight of the products placed on the bagging table 103.

[0025] As shown in FIG. 3, the self-service POS terminal 1 includes the touch panel 105, light-emitting unit 107, scanner 108, card reader 110, printer 112, automatic change dispenser 114, payment terminal 120, and scale unit 122 described above, as well as a processor 11, main memory 12, auxiliary storage device 13, clock 14, communication interface 15, and system transmission path 16. The system transmission path 16 includes an address bus, a data bus, control signal lines, and the like. The system transmission path 16 connects the processor 11 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 11, main memory 12, auxiliary storage device 13, clock 14, and communication interface 15 are connected via the system transmission path 16 to form a computer for the self-service POS terminal 1.

[0026] The processor 11 corresponds to the central part of the computer. The processor 11 controls each part to realize various functions of the self-service POS terminal 1 in accordance with an operating system or an application program. The processor 11 is, for example, a CPU (Central Processing Unit). The processor 11 may be, for example, an MPU (Micro Processing Unit), an SoC (System on a Chip), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field-Programmable Gate Array). Alternatively, the processor 11 may be a combination of two or more of these.

[0027] The main memory 12 corresponds to the main storage portion of the computer. The main memory 12 includes a nonvolatile memory area and a volatile memory area. The main memory 12 stores an operating system or application programs in the nonvolatile memory area. The main memory 12 may store data required for the processor 11 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 12 uses the volatile memory area as a work area where data is rewritten by the processor 11 as appropriate. The nonvolatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0028] The auxiliary storage device 13 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 13 may be an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device 13 stores data used by the processor 11 when performing various processes, data created by the processes in the processor 11, etc. The auxiliary storage device 13 may also store the application programs described above.

[0029] The clock 14 keeps track of the date and time. The processor 11 processes the date and time kept by the clock 14 as the current date and time.

[0030] The communication interface 15 performs data communication in accordance with a preset communication protocol with an external device connected via the communication network 6. The external device is typically the detection terminal 3. The external device may also include the monitoring terminal 4 or other devices such as a server (not shown).

[0031] Scanner 108 reads a code symbol from a commodity held over reading window 109. Each commodity sold in a store is affixed with a code symbol representing a code or the like for identifying the commodity. The code symbol is, for example, a barcode. The code symbol may also be, for example, a two-dimensional data code. Scanner 108 may be of a type that reads a code symbol by scanning with laser light, or of a type that reads a code symbol from an image captured by an imaging device.

[0032] Card reader 110 reads card data recorded on a card medium such as a credit card, a point card, etc. Card reader 110 pulls the card medium inserted into card insertion slot 111 into main body 101, reads the card data, and then ejects the card from card insertion slot 111.

[0033] The printer 112 prints receipt data and other information indicating the details of the transaction on receipt paper. The receipt paper on which the receipt data is printed is ejected from the issuing port 113, cut by a cutter (not shown), and issued as a receipt or proof of purchase.

[0034] The automatic change dispenser 114 includes a coin unit and a banknote unit. The coin unit sorts coins inserted into a coin insertion slot 115 one by one, identifies the denomination, and stores them in a safe according to the denomination. The coin unit takes out coins of the appropriate denomination from the safe based on, for example, change data, and dispenses them to a coin dispensing outlet 116. The banknote unit sorts banknotes inserted into a banknote insertion slot 117 one by one, identifies the denomination, and stores them in a safe according to the denomination. The banknote unit takes out banknotes of the appropriate denomination from the safe based on, for example, change data, and dispenses them to a banknote dispensing outlet 118.

[0035] The payment terminal 120 reads and rewrites electronic money recorded on, for example, an electronic money medium. The electronic money medium is, for example, a contactless IC card. The electronic money medium may also be an electronic device such as a smartphone or a tablet terminal.

[0036] The self-service POS terminal 1 configured as described above uses a portion of the volatile memory area in the main memory 12 as the area for the transaction file 17. The transaction file 17 is a data file for storing data related to one commercial transaction processed by the self-service POS terminal 1. The transaction file 17 stores data such as the transaction number, purchased product data, total number of items, total amount, discount amount, and settlement amount. The transaction number is a series of numbers that is issued each time a commercial transaction is processed by the self-service POS terminal 1. The purchased product data is record data created for each product sold in a commercial transaction identified by the transaction number. Here, the purchased product data consists of items such as the product code, product name, price, number of items, and amount. The number of items is the number of items purchased for the product identified by the product code. The amount is the amount for the number of items purchased. The transaction file 17 can store multiple purchased product data. The total number of items is the sum of the points of each purchased product data. The total amount is the sum of the amounts of each purchased product data. The discount amount is the amount discounted from the total amount. The settlement amount is the total amount minus the discount amount.

[0037] [Detection device configuration explanation] 4 is a block diagram showing an example of the main circuit configuration of the detection terminal 3. The detection terminal 3 includes a processor 31, a main memory 32, an auxiliary storage device 33, a clock 34, a communication interface 35, and a system transmission path 36. The system transmission path 36 includes an address bus, a data bus, a control signal line, etc. The system transmission path 36 connects the processor 31 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The computer of the detection terminal 3 is configured by connecting the processor 31, the main memory 32, the auxiliary storage device 33, the clock 34, and the communication interface 35 via the system transmission path 36. The computer of the detection terminal 3 can also be referred to as the computer of the transaction processing device.

[0038] The processor 31 corresponds to the central part of the computer. The processor 31 controls each part to realize various functions of the detection terminal 3 in accordance with an operating system or an application program. The processor 31 is, for example, a CPU. The processor 31 may be a multi-core / multi-threaded processor, and can execute multiple processes in parallel. The processor 31 may be, for example, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 31 may be a combination of two or more of these.

[0039] The main memory 32 corresponds to the main storage portion of the computer. The main memory 32 includes a nonvolatile memory area and a volatile memory area. The main memory 32 stores an operating system or application programs in the nonvolatile memory area. The main memory 32 may store data required for the processor 31 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 32 uses the volatile memory area as a work area where data is rewritten by the processor 31 as appropriate. The nonvolatile memory area is, for example, ROM. The volatile memory area is, for example, RAM.

[0040] The auxiliary storage device 33 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM, HDD, or SSD can be the auxiliary storage device 33. The auxiliary storage device 33 stores data used by the processor 31 when performing various processes, data created by the processes in the processor 31, etc. The auxiliary storage device 33 may also store the application programs described above.

[0041] The application programs stored in the main memory 32 or the auxiliary storage device 33 include, for example, an information processing program for causing the computer to function as the information processing device according to the first embodiment, and a video analysis program for implementing an AI-based video analysis engine. The method for installing these programs in the main memory 32 or the auxiliary storage device 33 is not particularly limited. The program can be installed in the main memory 32 or the auxiliary storage device 33 by recording it on a removable recording medium or by distributing it via communication over a network. The recording medium may be in any form, such as a CD-ROM or memory card, as long as it can store the program and is readable by the device.

[0042] The clock 34 keeps track of the date and time. The processor 31 processes the date and time kept by the clock 34 as the current date and time.

[0043] The communication interface 35 performs data communication in accordance with a preset communication protocol with external devices connected via the communication network 6. The external devices are, for example, the self-service POS terminal 1, the camera 2, the monitoring terminal 4, the recording device 5, etc.

[0044] In the detection terminal 3 configured as above, part of the storage area of ​​the auxiliary storage device 33 is used as an area for a setting table 331, a count table 332, a camera image storage unit 333, and an event log storage unit 334.

[0045] Setting table 331 is a data table that describes the setting data of each reporting action item and reporting threshold in association with a reporting action ID, as shown in Fig. 5. The reporting action ID is unique identification information set for each reporting action to identify the action that requires a report from the purchaser to the transaction processing device, i.e., self-service POS terminal 1, that is, the type of reporting action.

[0046] The report action items are names that represent actions that require a report from the purchaser to the self-service POS terminal 1. In this embodiment, five items are exemplified as report action items: "missed scan," "weight error," "high-priced item error," "item under cart error," and "large number of items purchased." The types of each report action will be described in detail later.

[0047] The reporting threshold is a value that serves as a criterion for determining whether or not to report a reporting action identified by the corresponding reporting action ID. The value is typically the number of occurrences. The value may also be the number of items purchased. In this embodiment, if the number of occurrences of the reporting action is less than the reporting threshold, no reporting is done, but if it is equal to or exceeds the reporting threshold, a reporting is done. The reporting destination is the transaction processing device for the purchaser who performed the action, that is, the self-service POS terminal 1. The reporting destination includes the monitoring terminal 4, which is a terminal for the monitor.

[0048] As shown in FIG. 6, the count table 332 is a data table that describes the number of occurrences and the report flag (report F) in association with the report action ID. The default for the number of occurrences before a transaction begins is "0." The number of occurrences is counted up by "1" each time a report action identified by the corresponding report action ID occurs in a transaction. The default for the report flag before a transaction begins is "0." The report flag becomes "1" when the number of occurrences of a report action identified by the corresponding report action ID in a transaction reaches the report threshold for that report action. A report action whose report flag becomes "1" is reported to the aforementioned report destination. In other words, the report flag is one-bit data that identifies whether or not the report action identified by the corresponding report action ID should be reported.

[0049] The camera image storage unit 333 sequentially stores camera images from the corresponding camera 2. The event log storage unit 334 sequentially stores a log of POS events from the self-service POS terminal 1 and a log of camera events determined from the camera images. The camera image storage unit 333 and the event log storage unit 334 can delete the stored contents at any time, such as outside the store's business hours, as needed.

[0050] Returning to the explanation of Figure 4. 4, the processor 31 of the detection terminal 3 has functions as a detection means 311, a reporting means 312, and a second reporting means 313. These functions are realized by performing information processing in accordance with the information processing program and the video analysis program described above.

[0051] Detection means 311 has a function of detecting an action by a purchaser that requires a report to the transaction processing device, i.e., self-service POS terminal 1, i.e., a reporting action. Detection means 311 detects reporting actions by type based on a first event, i.e., a camera event, determined by the video analysis engine based on the output of a sensor that detects the purchaser's behavior, i.e., camera 2, and a second event, i.e., a POS event, that indicates the purchaser's behavior and is reported from self-service POS terminal 1. Detection means 311 detects reporting actions categorized into the five categories mentioned above, for example, "omitted scan," "weight error," "high-priced item error," "item below cart error," and "large purchase."

[0052] The reporting means 312 has a function of reporting when the reporting action detected by the detection means 311 in one transaction exceeds the threshold of the judgment criteria for the reporting action, i.e., the reporting threshold. The reporting means 312 reports to the transaction processing device of the purchaser who performed the reporting action, i.e., the self-service POS terminal 1. The reporting means 312 reports when the threshold of the judgment criteria for the reporting action is exceeded.

[0053] The second reporting means 313 is a function that reports to the monitor's terminal, that is, the monitoring terminal 4, when a reporting act is detected by the detection means 311. The second reporting means 313 reports to the monitoring terminal 4 when a reporting act is detected, regardless of the reporting threshold.

[0054] [Detection terminal operation explanation] FIG. 7 is a sequence diagram for explaining the operations of the detection terminal 3 relating to the detection and reporting of a reporting action, together with the operations of the camera 2 and the self-checkout POS terminal 1 of the same register system RS.

[0055] As shown in FIG. 7, camera 2 captures images (ACT11) and outputs the captured camera footage to the corresponding detection terminal 3 (ACT12). For example, camera 2 may have a human detection function and transmit the captured camera footage while a customer is captured in the captured camera footage. Alternatively, camera 2 may have a moving object detection function instead of a human detection function and transmit the camera footage while a moving object is captured in the camera footage. Alternatively, camera 2 may not have such a function itself, but instead have a separate human detection sensor or the like installed in self-service POS terminal 1, and capture and transmit the camera footage only while a person is detected in the sensor output. This allows camera 2 to transmit the camera footage for each transaction by a customer. Camera 2 transmits the camera footage not only to detection terminal 3 but also to recording device 5. Recording device 5 records the camera footage in a data file in chronological order. For example, recording device 5 records the camera footage in a data file with a separate file name for each camera 2.

[0056] The detection terminal 3 receives the camera video, analyzes the camera video using an AI video analysis engine, detects the behavior of the customer, and determines the camera event corresponding to that behavior (ACT31). For example, when the detection terminal 3 detects from the camera video that the customer is removing a product from the cart stand 102, it determines the camera event corresponding to that behavior. Hereinafter, this camera event will be referred to as a product removal event CIa. The product removal event CIa may include the product code of the product that the customer removed from the cart stand 102, which is identified by the appearance feature data of the product.

[0057] For example, when the detection terminal 3 detects from the camera image that a purchaser holds a product over the reading window 109 of the scanner 108, it determines the camera event corresponding to that action. Hereinafter, this camera event will be referred to as a scan action event CIb. The scan action event CIb may include the product code of the product identified by the appearance feature data of the product that the purchaser holds over the reading window 109 of the scanner 108.

[0058] For example, when the detection terminal 3 detects from the camera image that a customer is putting products into a shopping bag or a personal bag placed on the bagging table 103, it determines the camera event corresponding to that action. Hereinafter, this camera event will be referred to as a product bagging event CIc. The product bagging event CIc may include the product code of the product that the customer placed in the shopping bag or personal bag placed on the bagging table 103, which is identified by the appearance feature data of the product.

[0059] When the detection terminal 3 detects from the camera image that a product has been placed in the bottom shelf of a shopping cart placed near the cart stand 102, it determines a camera event corresponding to the action of placing a product under the cart. Hereinafter, this camera event will be referred to as a "below-cart product placement event CId." The below-cart product placement event CId may include the product code of the product identified by the appearance feature data of the product placed under the cart. In this case, if two or more products are placed under the cart, the below-cart product placement event CId will include the product code of each product.

[0060] The types of camera events identified by the detection terminal 3 are not limited to the above-described four types of product pick-up event CIa, scanning action event CIb, product bagging event CIc, and under-cart product storage event CId. The detection terminal 3 may also identify other types of camera events from the camera video.

[0061] When some operation event occurs by a purchaser (ACT21), the self-checkout POS terminal 1 outputs a POS event indicating the operation content to the corresponding detection terminal 3 (ACT22). For example, when the code symbol of a product held over the reading window 109 of the scanner 108 is read, the self-checkout POS terminal 1 outputs a POS event indicating the operation content. Hereinafter, this POS event will be referred to as a registered event PIa. The registered event PIa includes the product code identified from the code symbol read by the scanner 108.

[0062] For example, when a product is placed on the bagging table 103 and the measured weight of the scale unit 122 changes, the self-service POS terminal 1 outputs a POS event indicating the operation. Hereinafter, this POS event will be referred to as a weight event PIb. The weight event PIb includes the measured weight after the change.

[0063] For example, when the [Checkout] touch button BTb displayed on the registration screen SCa (see FIG. 14) of the touch panel 105 is touched, the self-service POS terminal 1 outputs a POS event indicating the operation. Hereinafter, this POS event will be referred to as a checkout event PIc. When the purchaser has finished registering the purchased items, they touch the [Checkout] touch button BTb.

[0064] The types of POS events output from the self-checkout POS terminal 1 are not limited to the above-mentioned three types, namely, the registration event PIa, the weight event PIb, and the accounting event PIc. The self-checkout POS terminal 1 may output other types of POS events in response to the operation of the purchaser.

[0065] The detection terminal 3 detects a reporting action by the purchaser based on the POS event output from the self-service POS terminal 1 and the camera event determined based on the camera image from the camera 2 (ACT32). The detection terminal 3 detects a reporting action called, for example, "missed scan," "high-priced item error," "item under cart error," or "large purchase quantity." Alternatively, the detection terminal 3 detects a reporting action by the purchaser based on the POS event output from the self-service POS terminal 1 (ACT32). The detection terminal 3 detects a reporting action called, for example, "weight error."

[0066] In this way, detection terminal 3 detects a customer's reporting behavior based on POS events and camera events. That is, self-service POS terminal 1, which outputs POS events, and camera 2, which outputs camera footage used to determine camera events, function as output means that output information necessary to detect a customer's behavior requiring a report to the transaction processing device. It is also possible to detect a customer's reporting behavior based solely on POS events, in which case self-service POS terminal 1 functions as output means. It is also possible to detect a customer's reporting behavior based solely on camera events, in which case camera 2 functions as output means.

[0067] The detection terminal 3 determines whether or not it has detected the reporting behavior of the purchaser (ACT33). If it detects the reporting behavior (YES in ACT33), the detection terminal 3 notifies the monitor that the reporting behavior has occurred. Specifically, the detection terminal 3 outputs a reporting event to notify the monitor terminal 4, which is the operator of the monitor, that the reporting behavior has occurred (ACT34).

[0068] When the monitoring terminal 4 receives a notification event from the detection terminal 3, it notifies the content of the notification on its display. For example, the monitoring terminal 4 displays the number of cases, such as [Missed scans: 0 cases], on the monitoring screen SCb (see FIG. 16). The number of cases of [Missed scans: 0 cases] may be displayed by the monitoring terminal 4 counting up the value of the number each time a notification action (notification event) of the notification action item "Missed scan" is detected from the detection terminal 3. That is, on the monitoring screen SCb displayed on the display of the monitoring terminal 4, each time the monitoring terminal 4 receives a notification action of the notification action item "Missed scan" from the detection terminal 3, the monitoring terminal 4 may count up and display the text of the notification action on the monitoring screen SCb, for example, the number of cases of [Missed scans: 0 cases].

[0069] The detecting terminal 3 also adds "1" to the number of occurrences described in association with the reporting action ID of the reporting action in the counting table 332 (ACT 35). Then, the detecting terminal 3 compares the number of occurrences with the reporting threshold set in the setting table 331 in association with the reporting action ID of the reporting action (ACT 36).

[0070] The detection terminal 3 determines whether the number of occurrences is equal to or greater than the reporting threshold (ACT37). If the number of occurrences is equal to or greater than the reporting threshold (YES in ACT37), the detection terminal 3 notifies the purchaser that a reporting behavior has occurred. Specifically, the detection terminal 3 outputs a reporting event for the purchaser to report the reporting behavior to the self-service POS terminal 1, which is operated by the purchaser (ACT38). The self-service POS terminal 1 that has input the reporting event notifies the purchaser by displaying information such as text or an image indicating the content of the reporting behavior on the touch panel 105 (ACT23). The self-service POS terminal 1 may also notify the purchaser of the content of the reporting behavior by voice.

[0071] If the detection terminal 3 does not detect a reporting behavior in the operation of ACT 33 (NO in ACT 33), the detection terminal 3 skips the operations of ACT 34 to ACT 38. Similarly, if the detection terminal 3 detects that the number of occurrences is less than the reporting threshold in the operation of ACT 37 (NO in ACT 37), the detection terminal 3 skips the operation of ACT 38.

[0072] The detection terminal 3 repeatedly executes the operations shown in ACT31 to ACT38 from the time the purchased items of one transaction are registered at the self-service POS terminal 1 until the transaction is settled. Thus, the detection terminal 3 detects reporting behaviors that occur in one transaction by type based on POS events output from the self-service POS terminal 1 and camera events identified from the video captured by the camera 2. Then, when the number of times a reporting behavior is detected reaches or exceeds the reporting threshold for that reporting behavior, the detection terminal 3 outputs a reporting event to the self-service POS terminal 1 operated by the purchaser who performed the reporting behavior to report that the reporting behavior has been detected. Meanwhile, each time a reporting behavior is detected, the detection terminal 3 outputs a reporting event to the monitoring terminal 4, which is the monitor's terminal, to report that the reporting behavior has been detected.

[0073] [Detection terminal program explanation] 8 is a flowchart showing the main steps of the information processing executed by the processor 31 of the detection terminal 3 operating as described above. The processor 31 executes the information processing of the steps shown in the flowchart of FIG. 8 in accordance with an information processing program stored in the main memory 32 or the auxiliary storage device 33.

[0074] That is, in ACT101, the processor 31 checks whether or not camera video has been acquired from the camera 2 corresponding to the detection terminal 3. If camera video has not been acquired, the processor 31 proceeds to ACT102. In ACT102, the processor 31 checks whether or not a POS event has been acquired from the self POS terminal 1 corresponding to the detection terminal 3. If a POS event has not been acquired, the processor 31 returns to ACT101. In this way, the processor 31 waits to acquire camera video or a POS event in ACT101 and ACT102.

[0075] When camera video is acquired in the standby state of ACT101 and ACT102, the processor 31 proceeds to ACT103. In ACT103, the processor 31 stores the camera video in the camera video storage unit 333. Then, in ACT104, the processor 31 uses an AI video analysis engine to analyze the camera video stored in the camera video storage unit 333 and check whether a camera event has been identified. If a camera event has not been identified, the processor 31 returns to the standby state of ACT101 and ACT102.

[0076] If a camera event is determined in ACT 104, the processor 31 proceeds to ACT 105. The processor 31 stores the camera event in the event log storage unit 334 in ACT 105.

[0077] On the other hand, if a POS event is acquired in the standby state of ACT101 and ACT102, the processor 31 proceeds to ACT 106. The processor 31 stores the POS event in the event log storage unit 334 in ACT106.

[0078] When the camera event or POS event is stored in the event log storage unit 334 in ACT105 or ACT106, the processor 31 proceeds to ACT107. Based on the camera event and POS event stored in the event log storage unit 334 in ACT107, the processor 31 detects a reporting action in the reporting action item set in the setting table 331. The detection operation for each reporting action will be described in detail later.

[0079] The processor 31 checks whether or not a reporting action has been detected in ACT 108. If a reporting action has not been detected, the processor 31 returns to the standby state in ACT 101 and ACT 102.

[0080] When a reportable action of any of the reportable action items is detected in ACT108, the processor 31 proceeds to ACT109. The processor 31 notifies the monitor in ACT109. That is, the processor 31 transmits a report event to the monitoring terminal 4 to notify that a reportable action has been detected. The report event includes information indicating the reportable action item of the detected reportable action. The monitoring terminal 4 that receives the report event displays on its display information to notify the monitor that a reportable action of the reportable action item included in the report event has been detected in the detection terminal 3 that sent the report event (see FIG. 16).

[0081] After notifying the monitor, processor 31 proceeds to ACT110. In ACT110, processor 31 counts up by "1" the number of occurrences stored in count table 332 in association with the notification action ID of the detected notification action. Then, in ACT111, processor 31 compares the number of occurrences with the notification threshold stored in setting table 331 in association with the notification action ID. In ACT112, processor 31 determines whether the number of occurrences is equal to or greater than the notification threshold. If the number of occurrences is less than the notification threshold, processor 31 returns to the standby state of ACT101 and ACT102.

[0082] On the other hand, if the number of occurrences is equal to or greater than the notification threshold, the processor 31 proceeds to ACT113. In ACT113, the processor 31 sets the notification flag (report F) stored in the count table 332 in association with the notification action ID to "1." The processor 31 also notifies the purchaser in ACT114. That is, the processor 31 transmits a notification event to the corresponding self-service POS terminal 1 to notify that a notification action has been detected. The notification event includes information indicating the notification action item of the notification action ID associated with the notification flag that has become "1." In the self-service POS terminal 1 that receives the notification event, information to notify the purchaser that a notification action of the notification action item included in the notification event has been detected is displayed, for example, on a part of the screen of the touch panel 105 (see FIG. 15). After notifying the purchaser, the processor 31 returns to the standby state of ACT101 and ACT102.

[0083] Here, the processor 31 realizes the function of detection means 311 by the processing of ACT101 to ACT108. The processor 31 realizes the function of notification means 312 by the processing of ACT110 to ACT114. The processor 31 realizes the function of second notification means 313 by the processing of ACT109.

[0084] [Explanation of types of reporting behavior and detection operations] 9 is a schematic diagram showing the operation of detecting a reportable behavior for the reportable behavior item "failed to scan." The reportable behavior item "failed to scan" refers to the behavior of a purchaser not scanning the barcode of a purchased item that has been removed from the basket stand 102 and packed into a shopping bag or personal bag on the bagging stand 103 with the scanner 108 of the self-service POS terminal 1. For example, when a purchaser holds a purchased item over the reading window 109 of the scanner 108 but does not hold the barcode correctly over the scanner, the barcode is not read and the item is then bagged, which corresponds to the reportable behavior for the reportable behavior item "failed to scan."

[0085] For such an action, the event log storage unit 334 stores a product removal event CIa for the camera event, followed by a scanning action event CIb, and then a product bagging event CIc. On the other hand, for the POS event, only a weight event PIb is stored, and a registration event PIa is not stored.

[0086] The detection terminal 3 detects that a report action of the report action item "failed to scan" has occurred when the product removal event CIa, the scanning action event CIb, and the product bagging event CIc are stored in that order, and the weight event PIb is stored but the registered event PIa is not stored. Note that a case in which a purchaser bags the purchased products without holding them over the reading window 109 also constitutes a report action of the report action item "failed to scan." In other words, the detection terminal 3 detects that a report action of the report action item "failed to scan" has occurred even when the product removal event CIa and the product bagging event CIc are stored and the weight event PIb is stored but the registered event PIa and the scanning action event CIb are not stored.

[0087] In this embodiment, the reporting threshold for the reportable behavior item "missed scan" is set to "2." In other words, if the reportable behavior item "missed scan" is detected two or more times in one transaction, a report is sent to the self-service POS terminal 1. If it occurs only once, there is a possibility of a false detection, so a report is not sent to the self-service POS terminal 1. In other words, the report is sent to avoid stopping the purchaser's self-service operation. On the other hand, a report is sent to the monitoring terminal 4 even the first time. Therefore, a monitor who confirms the report can go to the purchaser to check whether there was a missed scan, and if there was a missed scan, instruct the purchaser to rescan, thereby resolving the missed scan.

[0088] FIG. 10 is a schematic diagram illustrating the detection of a reporting action for the reporting action item "weight error." The reporting action item "weight error" refers to a case in which the purchased items removed by a customer from the cart stand 102 and packed into a shopping bag or personal bag on the bagging stand 103 do not match the items identified by the barcodes scanned by the scanner 108 of the self-service POS terminal 1. For example, a customer removing two purchased items from a shopping cart and scanning only one of the items to bag the two items corresponds to the reporting action for the reporting action item "weight error." For example, a pack of multiple identical items is printed with a barcode corresponding to the pack price, while each individual item is printed with a barcode corresponding to the individual item price. Removing such a pack of items from the cart stand 102 and scanning the barcode corresponding to the individual item price instead of the barcode corresponding to the pack price to bag the items also corresponds to the reporting action for the reporting action item "weight error."

[0089] For such actions, the event log storage unit 334 stores a product removal event CIa for the camera event, followed by a scanning action event CIb, and then a product bagging event CIc. On the other hand, for the POS event, a registration event PIa is stored, followed by a weight event.

[0090] When a product removal event CIa, a scanning action event CIb, and a product bagging event CIc are stored in this order, and a registered event Pia and a weight event PIb are stored in this order, the detection terminal 3 acquires the set weight Wa of the product identified by the product code included in the registered event Pia (ACT201). The detection terminal 3 also acquires the measured weight included in the weight event (ACT202). The detection terminal 3 then calculates the amount of change Wb between the measured weight included in the immediately preceding weight event and the measured weight included in the current weight event (ACT203).

[0091] The detection terminal 3 calculates the difference W between the set weight Wa and the amount of change Wb (ACT 204). Then, the detection terminal 3 checks whether the difference W is equal to or greater than the judgment value Wx (ACT 205). The judgment value Wx is an arbitrary value that exceeds the allowable range ±α of the set weight Wa. If the weight of products with the same barcode is uniform, the judgment value Wx is "0" or a value close to "0". However, there are products with the same barcode, such as vegetables, fruits, and prepared foods, whose weights are not uniform. For such products, an appropriate judgment value Wx (Wx>α) that exceeds the allowable range ±α of the set weight Wa is set.

[0092] If the difference W is less than the judgment value Wx (NO in ACT205), the detection terminal 3 does not detect the report action item "weight error." On the other hand, if the difference W is equal to or greater than the judgment value Wx (YES in ACT205), the detection terminal 3 detects that a report action of the report action item "weight error" has occurred.

[0093] In this embodiment, the reporting threshold for the reporting action item "weight error" is set to "2." In other words, if the reporting action of the reporting action item "weight error" is detected twice in one transaction, a report will be sent to the self-service POS terminal 1. If it occurs only once, a malfunction of the scale unit 122 may be possible due to the position of the bagged products, so a report will not be sent to the self-service POS terminal 1. In other words, a report that would stop the purchaser from operating the self-service is avoided. Note that, since a report is sent to the monitoring terminal 4 even the first time, it is possible for a monitor to go to the purchaser and confirm the legitimacy of the product in which the weight error occurred.

[0094] FIG. 11 is a schematic diagram illustrating the detection of a reporting action for the reporting action item "high-priced item error." The reporting action item "high-priced item error" refers to an action that occurs when a purchaser removes a high-priced item from the shopping cart stand 102 and packs it into a shopping bag or personal bag on the bagging stand 103, but the item identified by the barcode scanned by the scanner 108 of the self-service POS terminal 1 is not the high-priced item. For example, items with a unit price equal to or greater than a predetermined amount are preset as high-priced items. A reporting action for the reporting action item "high-priced item error" occurs when a purchaser removes a high-priced item and other items from the shopping cart, scans only the other items, and packs them into a bag, while concealing the high-priced item. Another reporting action for the reporting action item "high-priced item error" occurs when a purchaser holds a high-priced item over the reading window 109 of the scanner 108 but packs it without reading the barcode because the barcode is not correctly held over the scanner. The latter action also corresponds to the reporting action of the reporting action item "missed scan."

[0095] For such actions, the event log storage unit 334 stores a product removal event CIa for the camera event, followed by a scanning action event CIb, and then a product bagging event CIc. On the other hand, for the POS event, a registration event PIa is stored, followed by a weight event.

[0096] The detection terminal 3 checks whether the product has been identified from the camera image of the purchaser removing the product from the cart 102, i.e., whether the product code is included in the product removal event CIa (ACT211). If the product code is included (YES in ACT211), the detection terminal 3 determines whether the product identified by the product code is a product that has been set as a high-priced product in advance (ACT212). If the product code is not included in the product removal event CIa (NO in ACT211) or if the product identified by the product code is not a high-priced product (NO in ACT212), the detection terminal 3 does not detect the reporting action item "high-priced product error."

[0097] On the other hand, if the product identified by the product code is a high-priced product (YES in ACT212), the detection terminal 3 continues to detect the report action item "high-priced product error." The detection terminal 3 acquires the product code from the registered event PIa (ACT213). The detection terminal 3 determines whether the product code matches the product code of a high-priced product (ACT214). If the product codes match (NO in ACT214), the detection terminal 3 does not detect the report action item "high-priced product error." On the other hand, if the product codes do not match (YES in ACT214), the detection terminal 3 detects that a report action for the report action item "high-priced product error" has occurred.

[0098] In this embodiment, the reporting threshold for the reportable behavior item "high-priced item error" is set to "1." In other words, when the reportable behavior item "high-priced item error" is detected once in a transaction, it is reported to the self-service POS terminal 1. Fraudulent behavior involving high-priced items causes large losses for the store. For this reason, even if it is detected once, a report is sent to alert the purchaser. This helps to deter fraudulent behavior.

[0099] 12 is a schematic diagram showing the operation of detecting a reporting action for the reporting action item "Error in product at bottom of cart." The reporting action item "Error in product at bottom of cart" refers to the action of a purchaser completing the registration of a purchased item without scanning the purchased item placed in the bottom row of the shopping cart.

[0100] For such actions, the event log storage unit 334 stores an under-cart product storage event CId for the camera event. On the other hand, for the POS event, a checkout event PIc is stored. When the detection terminal 3 acquires an under-cart product storage event CId, it recognizes the product identified by the under-cart product storage event CId (ACT 221). That is, the detection terminal 3 acquires the product code included in the under-cart product storage event CId. At this time, if the under-cart product storage event CId includes product codes for two or more items, it acquires each product code.

[0101] After that, when the detection terminal 3 acquires the accounting event PIc, it searches for the purchased items registered in the transaction file 17 of the corresponding self-service POS terminal 1 (ACT222). The detection terminal 3 then checks whether the items stored under the cart are registered in the transaction file 17 (ACT223). That is, the detection terminal 3 checks whether the purchased item data, including the item code included in the under-cart item storage event CId, is registered in the transaction file 17. If all items stored under the cart are registered in the transaction file 17 (NO in ACT223), the detection terminal 3 does not detect the report action item "under-cart item error." If even one item stored under the cart is not registered in the transaction file 17 (YES in ACT223), the detection terminal 3 detects that a report action for the report action item "under-cart item error" has occurred.

[0102] Generally, items stored in the bottom of a shopping cart are often large items that do not fit in the shopping basket. Therefore, if the detection terminal 3 is unable to acquire a product code from the bottom-of-cart product storage event CId in ACT221, it searches in ACT222 to determine whether or not any products classified as large items are registered in the transaction file 17. If no products classified as large items are registered in the transaction file 17 (YES in ACT223), the detection terminal 3 may detect that a reporting action for the reporting action item "bottom-cart product error" has occurred.

[0103] In one transaction, the reporting action of the reporting action item "cart item error" is detected only once. Therefore, in this embodiment, the reporting threshold for the reporting action item "cart item error" is set to "1."

[0104] FIG. 13 is a schematic diagram showing the operation of detecting a reportable behavior for the reportable behavior item "large purchase quantity." The reportable behavior item "large purchase quantity" is an behavior in which a purchaser purchases a large amount of merchandise in one transaction. For example, if a purchaser purchases 10 or more items, the detection terminal 3 notifies the monitoring terminal 4 and the self-service POS terminal 1 that the reportable behavior item "large purchase quantity" has been detected.

[0105] In response to such actions, a product removal event CIa, a scanning action event CIb, and a product bagging event CIc are repeatedly stored for camera events in the event log storage unit 334. On the other hand, a registration event PIa and a weight event are repeatedly stored for POS events.

[0106] The detection terminal 3 stores "0" as the default number of purchases (ACT231). Subsequently, when the product removal event CIa, the scanning action event CIb, and the product bagging event CIc, which are camera events, are stored in the event log storage unit 334, and the registration event PIa and the weight event, which are POS events, are stored, the detection terminal 3 adds "1" to the number of purchases (ACT232).

[0107] The detection terminal 3 checks whether the number of items purchased is equal to or greater than the notification threshold (ACT233). The notification threshold is the value of the number of items purchased at which it is determined that the reportable behavior item "number of items purchased: large" has been detected. For example, if a purchaser purchases 10 or more items and it is assumed that the reportable behavior item "number of items purchased: large" has been detected, the notification threshold is "10". If the number of items purchased is less than the threshold (YES in ACT233), the detection terminal 3 continues the detection operation for the reportable behavior item "number of items purchased: large".

[0108] The detection terminal 3 adds "1" to the number of purchases (ACT234) and repeats the process of comparing it with the reporting threshold (ACT235) each time a product removal event CIa, a scanning action event CIb, and a product bagging event CIc, which are camera events, are stored in the event log storage unit 334, and a registration event PIa and a weight event, which are POS events, are stored. As a result, when the number of purchases becomes equal to or greater than the reporting threshold (YES in ACT235), the detection terminal 3 detects that a reporting action of the reporting action item "Purchase number: large" has occurred.

[0109] [Explanation of the report example] 14 is an example of a registration screen SCa displayed on the touch panel 105 of the self-checkout POS terminal 1. The registration screen SCa is a screen used to register purchased products. The registration screen SC is generated by the processor 11 of the self-checkout POS terminal 1.

[0110] As shown in FIG. 14, the registration screen SCa includes a touch button BTa for [Product without barcode] and a touch button BTb for [Checkout]. The touch button BTa for [Product without barcode] is used when registering products without barcodes, such as fresh food. In response to touching this touch button BTa, the self-service POS terminal 1 displays touch buttons corresponding to products without code symbols, so-called product buttons, and the purchaser can register the product by touching the product button corresponding to the purchased product. The touch button BTb for [Checkout] is used when the purchaser, who has finished registering the purchased products, declares that he or she wants to check out the transaction.

[0111] The registration screen SCa is divided into a registration list area ARa, a text area ARb, and a camera image area ARc. The registration list area ARa is an area for displaying a list of the product names, purchase quantities, and purchase amounts of purchased items registered by the purchaser. The text area ARb is an area for displaying text such as guidance and messages for the purchaser. The camera image area ARc is an area for displaying the current camera image being captured by the corresponding camera 2.

[0112] In this embodiment, when a report event is received from the detection terminal 3, text indicating the content of the reported action is displayed in the text area ARb. FIG. 15 is an example of the registration screen SCa of the self-service POS terminal 1 that has received a report event for the reportable action item "omitted scan." As shown in FIG. 15, text indicating the reported action for the reportable action item "omitted scan," for example, "Are there any products that you haven't scanned?", is displayed in the text area ARb. Therefore, a customer who intentionally commits fraud by skipping a scan is made aware that the store is aware of their fraudulent behavior, which helps prevent further fraud.

[0113] Fig. 16 is an example of a monitoring screen SCb displayed on the display of the monitoring terminal 4. As shown in Fig. 16, the monitoring screen SCb is divided into monitoring areas STa to STh for each register system RS. The monitoring areas STa to STh display the status of the self-service POS terminal 1 of the corresponding register system RS. Furthermore, when a notification event is received from the detection terminal 3 of the corresponding register system RS, text indicating the content of the notification action is displayed in the monitoring areas STa to STh.

[0114] FIG. 16 shows the monitoring screen SCb when a report action of the report action item "missed scan" is detected once at the detection terminal 3 of the cash register system RS identified by cash register number "2," and a report action of the report action item "missed scan" is detected twice at the detection terminal 3 of the cash register system RS identified by cash register number "7." The first time the report action of the report action item "missed scan" is detected, no report is made to the self-service POS terminal 1. However, the second time the report is made, a report is also made to the self-service POS terminal 1. Therefore, in the monitoring terminal 4, the text of the report action for the first report, for example, [Missed scan: 1 case], and the text of the report action for the second report, for example, [Missed scan: 2 cases], are made distinguishable by the monitor, for example, by changing the text color, font, etc.

[0115] 16, the display of the number of [Missed scans: 0] may be configured so that the number of reported actions for the reportable action item "Missed scans" is counted up each time the detection terminal 3 of the cash register system RS detects a reported action for the reportable action item "Missed scans." That is, on the monitoring screen SCb displayed on the display of the monitoring terminal 4, each time the monitoring terminal 4 receives a reported action for the reportable action item "Missed scans" from the detection terminal 3, the monitoring terminal 4 counts up the number of reported action text on the monitoring screen SCb, for example, [Missed scans: 0]. For example, when the detection terminal 3 of the cash register system RS identified by the cash register number "2" detects a reported action for the reportable action item "Missed scans" once, it displays [Missed scans: 1], and when the detection terminal 3 subsequently detects a reported action for the reportable action item "Missed scans" once again, it displays [Missed scans: 2].

[0116] [Advantages of the first embodiment] As described above in detail, according to the first embodiment, the detection terminal 3 sets a reporting threshold for each reportable behavior item identified by a reportable behavior ID. Then, when a reportable behavior of a reportable behavior item is detected in a single transaction at or above the reporting threshold for that reportable behavior item, the detection terminal 3 reports to the corresponding self-service POS terminal 1 that the reportable behavior of that reportable behavior item has been detected. Therefore, compared to when a report is made every time fraud in a reportable behavior item is detected, the frequency of reports is reduced, thereby reducing the risk of false positive reports to the purchaser. On the other hand, when a reportable behavior exceeding the reporting threshold is detected, a report is made to the corresponding self-service POS terminal 1, enabling efficient and effective prevention or deterrence of fraudulent behavior.

[0117] Furthermore, the detection terminal 3 notifies the monitoring terminal 4 each time a reportable behavior identified in a reportable behavior item is detected. Upon receiving this notification, the monitoring terminal 4 displays on its display that a reportable behavior identified in a reportable behavior item has been detected. This allows the monitor to know in real time that a reportable behavior has occurred, making it even more effective in preventing or deterring fraudulent behavior. The report destination may be the self-service POS terminal 1, or a monitor device other than a monitoring terminal.

[0118] [Second embodiment] Next, a second embodiment of the detector terminal 3 functioning as an information processing device will be described with reference to FIGS.

[0119] 17 is a series of sequence diagrams for explaining the operations related to the detection and reporting of a reporting action in the detection terminal 3 of the second embodiment, together with the operations of the camera 2 of the same register system RS and the self-service POS terminal 1. The same reference numerals are used to denote parts common to the sequence diagram of the first embodiment shown in FIG.

[0120] 7 and 17, the second embodiment differs from the first embodiment in the processing that occurs after it is determined in ACT 37 that the number of occurrences is equal to or greater than the notification threshold. That is, in the second embodiment, when the number of occurrences is equal to or greater than the notification threshold (YES in ACT 37), the detection terminal 3 sets the notification flag (notification F) to "1" (ACT 41). At this time, the detection terminal 3 does not notify the purchaser.

[0121] When the detection terminal 3 receives a transaction event as a POS event from the self-service POS terminal 1 (YES in ACT42), it determines whether there are any reportable action items whose report flag is set to "1" (ACT43). If the report flag of at least one reportable action item is set to "1" (YES in ACT43), the detection terminal 3 notifies the purchaser (ACT44).

[0122] For example, if the report flag associated with the report action ID of the report action item "missed scan" is "1," the buyer is notified that a report action of the report action item "missed scan" has been detected. Also, if the report flag associated with the report action ID of the report action item "weight error" is also "1," the buyer is also notified that a report action of the report action item "weight error" has been detected.

[0123] 18 is a flowchart showing the main steps of the information processing executed by the processor 31 of the detector terminal 3 operating as described above. The same reference numerals are used to designate parts that are common to the flowchart of the first embodiment shown in FIG.

[0124] As is clear from comparing FIG. 8 and FIG. 18, in the second embodiment, the processing after acquiring a POS event in the standby state of ACT101 and ACT102 differs from that of the first embodiment. That is, in ACT121, processor 31 checks whether the POS event is a transaction event. That is, processor 31 checks whether the transaction event is detected by detecting that the transaction button BTb on the registration screen Sca in FIG. 14 has been pressed. If it is not a transaction event, processor 31 proceeds to ACT106. Then, processor 31 executes the processing of ACT106 through ACT113 in the same manner as described in the first embodiment. However, processing of ACT114, i.e., notifying the purchaser, is not executed after ACT113.

[0125] If processor 31 acquires an accounting event in ACT 121, it proceeds to ACT 122. In ACT 122, processor 31 checks the notification flag (notification F) in count table 332. If the result shows that the notification flags for all notification action items are "0," processor 31 ends the information processing procedure shown in the flowchart of FIG.

[0126] On the other hand, if at least one report flag is set to "1", the processor 31 proceeds to ACT123. The processor 31 issues a report to the purchaser in ACT123. That is, the processor 31 transmits a report event to the corresponding self-service POS terminal 1 to report that a report action of a report action item whose report flag is set to "1" has been detected. At this time, if two or more report flags are set to "1", the processor 31 transmits a report event to the corresponding self-service POS terminal 1 to report that a report action of each report action item has been detected. With this, the processor 31 ends the information processing of the procedure shown in the flowchart of FIG. 18.

[0127] In this way, in the second embodiment, the detection terminal 3 sets a notification threshold for each reportable behavior identified by a reportable behavior item, and when the number of detected reportable behaviors equals or exceeds the notification threshold, the detection terminal 3 notifies the detection of the reportable behavior for that reportable behavior item upon receiving a transaction event from the corresponding self-service POS terminal 1. Therefore, compared to the first embodiment, in which a notification is made when the number of detected reportable behaviors equals or exceeds the notification threshold, the frequency of notification is only one, further reducing the risk of false positive notifications to customers. Furthermore, because the notification is made after product registration is complete, it does not interfere with the product registration operation of customers who are unfamiliar with the operation of the self-service POS terminal 1. Therefore, the time required for product registration can be shortened. Meanwhile, the detection terminal 3 notifies the monitoring terminal 4 each time a reportable behavior identified by a reportable behavior item is detected. Therefore, fraud prevention or deterrence is effective.

[0128] [Other embodiments] In the above embodiment, the self-service POS terminal 1 has a text area ARb and a camera image area ARc as part of the registration screen SCa. In this regard, as shown in Fig. 19, a customer display 8 is provided in the self-service POS terminal 1. The customer display 8 is provided, for example, next to the left side of the touch panel 105 that displays the registration screen SC, etc. The position of the customer display 8 is not limited to the position shown in Fig. 19. It may be placed anywhere as long as it can be confirmed by the customer.

[0129] As shown in FIG. 20, the self-service POS terminal 1 divides the screen SCd of the customer display 8 into a text area ARb and a camera image area ARc. Then, as shown on the screen SCe in the same figure, when a notification event is received from the detection terminal 3, the self-service POS terminal 1 displays the notification action item specified by the notification event, for example, the text "Are there any products that have not been scanned?" indicating the notification action of "missed scan," in the text area ARb. By providing the customer display 8 on the self-service POS terminal 1 in this way, the customer can more accurately recognize the notification. Furthermore, the registration screen SCa of the touch panel 105 can have a larger registration list area ARa.

[0130] In the above embodiment, as shown in Fig. 1, a detection terminal 3 to which an information processing device is applied is provided for each register system RS. In this regard, as shown in Fig. 21, the detection terminal 3 is removed from the register system RS, and instead a detection server 7 is connected to the communication network 6 to configure a self-checkout system SYS. The detection server 7 is a computer configured to perform the functions of the detection terminal 3 in each register system RS. Even with this configuration, the same effects as those of the first or second embodiment can be achieved.

[0131] In the above embodiment, the detection terminal 3 is exemplified as one aspect of an information processing device that can efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers. The information processing device is not limited to the detection terminal 3. For example, the functions of the detection means 311, the notification means 312, and the second notification means 313 described in the first embodiment may be provided in other devices, such as the self-service POS terminal 1, the camera 2, the monitoring terminal 4, or the recording device 5, and each of them may function as an information processing device.

[0132] Alternatively, instead of having all of the functions in the detection terminal 3, an information processing system may be realized by having some of the functions in the self-service POS terminal 1, the camera 2, the monitoring terminal 4, or the recording device 5. For example, in the first embodiment, the detection terminal 3 functions as the detection means 311. Meanwhile, the functions of the reporting means 312 and the second reporting means 313 are provided in the self-service POS terminal 1. The self-service POS terminal 1 also has a setting table 331 and a counting table 332. With this configuration, when the detection terminal 3 detects an action by the purchaser that requires a report to the self-service POS terminal 1, that is, a reporting action, the self-service POS terminal 1 outputs the information to the self-service POS terminal 1. Upon acquiring the information on the reporting action, the self-service POS terminal 1 immediately notifies the monitoring terminal 4. Furthermore, upon acquiring the information on the reporting action, the self-service POS terminal 1 increments the number of occurrences in the counting table 332. Then, when the number of occurrences of the detected reporting action in one transaction exceeds the reporting threshold set in the setting table 331, the self-service POS terminal 1 notifies the purchaser that a reporting action has been detected, for example, by displaying a pop-up on the touch panel 105. Even with this configuration, the advantageous effect of being able to efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers can be achieved.

[0133] Needless to say, the types of reporting actions detected by the detection means 311 are not limited to the five reporting action items of "failed to scan," "weight error," "high-priced item error," "item under cart error," and "large purchase quantity." For example, if the action of bagging items without barcodes without selecting them on the touch panel 105 is detected, the reporting action of the reporting action item "no barcode item error" may be detected. For the reporting action item "no barcode item error," the reporting frequency can be reduced by setting the reporting threshold to "2." For example, if the action of touching the [Checkout] touch button while items remain in the cart 102 is detected, the reporting action of the reporting action item "missing registered item error" may be detected. Because the [Checkout] touch button is touched only once per transaction, the reporting threshold for the reporting action item "missing registered item error" is necessarily "1."

[0134] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0135] 1...self-service POS terminal, 2...camera, 3...detection terminal, 4...monitoring terminal, 5...recording device, 6...communication network, 7...detection server, 8...purchaser display, 101...main body, 102...basket stand, 103...bagging stand, 105...touch panel, 108...scanner, 109...reading window, 122...scale unit, 311...detection means, 312...reporting means, 313...second reporting means, 331...setting table, 332...counting table, 333...camera image memory unit, 334...event log memory unit.

Claims

1. a detection means for detecting an action by a purchaser that requires a notification to the transaction processing device; a reporting means for reporting when the act detected by the detection means exceeds a threshold value of a criterion for the act in one transaction; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein said notification means notifies a transaction processing device of a purchaser who has performed an action requiring notification.

3. The information processing device according to claim 2 , wherein the notification means issues a notification when a threshold value of a criterion for the behavior is exceeded.

4. 3. The information processing device according to claim 2, wherein said notification means issues a notification when said transaction processing device receives a transaction instruction from the purchaser.

5. a second reporting means for reporting to a monitor terminal when an act requiring a report is detected by the detection means; 5. The information processing apparatus according to claim 1, further comprising:

6. The information processing device described in claim 1, wherein the detection means detects the type of action requiring notification based on a first event determined based on the output of a sensor that detects the behavior of the purchaser and a second event indicating the content of the purchaser's behavior reported from the transaction processing device.

7. The computer of the information processing device, a detection means for detecting an action by a purchaser that requires notification to the transaction processing device; and a reporting means for reporting when the act detected by the detection means in one transaction exceeds a threshold value of a criterion for the act; A program to function as a

8. an output means for outputting information necessary to detect an action by a purchaser that requires notification to the transaction processing device; a detection means for detecting an action by the purchaser that requires a report based on the information output by the output means; a reporting means for reporting when the act detected by the detection means exceeds a threshold value of a criterion for the act in one transaction; An information processing system comprising:

Images