Information processing device, program therefor, and information processing system

Abstract

To enable efficient and effective prevention or suppression of fraudulent acts while reducing the risk of false detection notifications to purchasers.SOLUTION: An information processing device comprises detection means, acquisition means, and notification means. The detection means detects an action of a purchaser on a transaction processing device that requires notification. The acquisition means acquires a type of an action requiring notification detected by the detection means. The notification means performs notification when multiple types of actions requiring notification are acquired by the acquisition means in one transaction.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device, a program therefor, and an information processing system. [Background technology]

[0002] In recent years, due to labor shortages and rising labor costs, self-checkout systems have become popular, allowing shoppers to complete everything from registering their purchases to paying for the transaction themselves. However, self-checkout systems raise concerns about fraudulent activities, such as bagging items without scanning them. To address this issue, systems have been developed that capture shoppers' behavior with cameras, analyze the footage using AI (Artificial Intelligence), and report any fraudulent activity that is detected.

[0003] However, such systems using AI video analysis may mistakenly detect non-fraudulent activity as fraudulent, and therefore, if a report is made every time fraud is detected, the risk of false positives being reported increases, leading to complaints from customers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-135620 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments of the present invention is to provide an information processing device that can efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers. [Means for solving the problem]

[0006] In one embodiment, the information processing device includes a detection means, an acquisition means, and a notification means. The detection means detects an action by a purchaser requiring a notification to the transaction processing device. The acquisition means acquires the types of actions requiring a notification detected by the detection means. The notification means notifies the transaction processing device when the acquisition means acquires multiple types of actions requiring a notification in one transaction. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a self-checkout system according to one embodiment. [Figure 2] FIG. 2 is a perspective view showing the external configuration of the self-service POS terminal. [Figure 3] FIG. 3 is a block diagram showing the main circuit configuration of the self-service POS terminal. [Figure 4] FIG. 4 is a block diagram showing an example of a main circuit configuration of the detector terminal. [Figure 5] FIG. 5 is a schematic diagram showing the main data structure of the report table. [Figure 6] FIG. 6 is a sequence diagram illustrating the operation of the detection terminal for detecting and reporting a reporting action, along with the operation of the camera and self-service POS terminal of the same register system. [Figure 7] FIG. 7 is a flowchart showing the main steps of information processing executed by the processor of the detector terminal. [Figure 8] FIG. 8 is a schematic diagram showing the notification action detection operation for the notification action item "omission of scan." [Figure 9] FIG. 9 is a schematic diagram showing the notification action detection operation for the notification action item "weight error." [Figure 10] FIG. 10 is a schematic diagram showing the reporting action detection operation for the reporting action item "high-priced item error." [Figure 11] FIG. 11 is a schematic diagram showing the reporting action detection operation for the reporting action item "error in product under cart." [Figure 12] FIG. 12 is a schematic diagram showing the reporting behavior detection operation for the reporting behavior item "purchase quantity: large." [Figure 13] FIG. 13 shows an example of a registration screen displayed on the touch panel of the self-service POS terminal. [Figure 14] FIG. 14 shows an example of a registration screen of a self-service POS terminal that has received a report event for the reportable action item "omitted scanning." [Figure 15] FIG. 15 is an example of a monitoring screen displayed on the display of the monitoring terminal. [Figure 16] FIG. 16 is a sequence diagram illustrating the operation of the detection terminal of the second embodiment regarding the detection and notification of a reporting action, along with the operation of the camera and self-checkout POS terminal of the same register system. [Figure 17] FIG. 17 is a flowchart showing the main steps of information processing executed by the processor of the detector terminal in the second embodiment. [Figure 18] FIG. 18 is a block diagram showing the main circuit configuration of a detector terminal according to the third embodiment. [Figure 19] FIG. 19 is a schematic diagram showing the main data structure of the combination table. [Figure 20] FIG. 20 is a flowchart showing the main steps of information processing executed by the processor of the detector terminal in the third embodiment. [Figure 21] FIG. 21 is an external configuration diagram of a self-service POS terminal according to another embodiment. [Figure 22] FIG. 22 is a diagram showing transitions of screens displayed on the customer display of the self-service POS terminal in another embodiment. [Figure 23] FIG. 23 is a schematic diagram showing the configuration of a self-checkout system according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of an information processing device that can efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers will be described with reference to the drawings.

[0009] [First embodiment] First, an embodiment of a self-checkout system to which an information processing device is applied will be described.

[0010] [Explanation of the self-checkout system] FIG. 1 is a schematic diagram showing a self-checkout system SYS according to one embodiment. The self-checkout system SYS includes multiple self-service POS terminals 1, multiple cameras 2, multiple detection terminals 3, a monitoring terminal 4, and a recording device 5. Specifically, the self-checkout system SYS configures a cash register system RS by associating each self-service POS terminal 1 with one camera 2 and one detection terminal 3. Each cash register system RS is installed at a checkout area of ​​a store. The self-checkout system SYS connects the self-service POS (Point of Sales) terminal 1, camera 2, and detection terminal 3 of each cash register system RS with the monitoring terminal 4 and recording device 5 via a communication network 6 such as a local area network (LAN). In this embodiment, the detection terminal 3 functions as an information processing device capable of efficiently and effectively preventing or deterring fraudulent activity while reducing the risk of false positive notifications to customers.

[0011] The user of the self-service POS terminal 1 is a customer who has finished shopping. After placing the purchased items in a shopping cart or the like in a sales area where items are displayed, the customer proceeds to the checkout area and operates the self-service POS terminal 1 to settle the transaction by himself, from registering the purchased items to paying for them. The self-service POS terminal 1 is an example of a fully self-service transaction processing device. The transaction processing device does not have to be a fully self-service device. The transaction processing device may be a payment device in which the registration of purchased items is performed on a separate device, and the customer himself performs operations related to paying for the items based on the data of the purchased items registered on that device to settle the transaction.

[0012] Based on a pre-established correspondence relationship with the detection terminal 3, the self-service POS terminal 1 transmits a POS event, which is event information indicating the operation content of the purchaser, to the corresponding detection terminal 3 via the communication network 6. The POS event is an example of a second event indicating the action content of the purchaser that is reported from the self-service POS terminal 1 to the detection terminal 3.

[0013] The camera 2 captures images of the actions of a customer operating the self-checkout POS terminal 1, and transmits the captured camera image to the corresponding detection terminal 3 via the communication network 6 based on a pre-set correspondence relationship with the detection terminal 3. Here, the image is transmitted via the communication network 6, but the image may also be transmitted directly from the camera 2 to the detection terminal 3 via, for example, a USB (Universal Serial Bus) cable. In this case, it is not necessary to set a correspondence relationship between the camera 2 and the detection terminal 3. The camera 2 can also transmit the camera image to the recording device 5 via the communication network 6.

[0014] The detection terminal 3, which functions as an information processing device, is equipped with an AI-based video analysis engine. The video analysis engine analyzes the camera video from the camera 2 to detect the behavior of the customer and identifies a camera event corresponding to that behavior. In other words, the camera 2 is an example of a sensor that detects the customer's behavior, and a camera event is an example of a first event identified based on the output of this sensor. The detection terminal 3 detects a customer's behavior that requires reporting based on the camera event identified by the video analysis engine and the POS event acquired from the self-service POS terminal 1. The detection terminal 3 may detect a reporting behavior based on a POS event, or may detect a reporting behavior based on a camera event. The detection terminal 3 reports the detected behavior, i.e., the reporting behavior, to the corresponding self-service POS terminal 1 via the communication network 6. The detection terminal 3 reports the reporting behavior not only to the corresponding self-service POS terminal 1 but also to the monitoring terminal 4.

[0015] The monitoring terminal 4 is a terminal operated by a store clerk who acts as a supervisor and monitors multiple self-service POS terminals 1 connected via the communication network 6. The monitoring terminal 4 has a display. The monitoring terminal 4 displays information necessary for monitoring multiple self-service POS terminals 1 on the display. The displayed information also includes information on reporting actions detected by the detection terminal 3.

[0016] The supervisory store clerk monitors the self-service POS terminal 1 based on the information displayed on the display. From the information on the reported behavior detected by the detection terminal 3, the clerk recognizes that a purchaser has committed an act that is suspected to be fraudulent. The supervisory store clerk is called, for example, an attendant. The monitoring terminal 4 is an example of a terminal for the supervisor. The monitoring terminal 4 may be any device that can be wirelessly connected to the communication network 6. For example, the monitoring terminal 4 may be a tablet terminal that the supervisory store clerk can carry around.

[0017] The recording device 5 is a storage device such as a NAS (Network Attached Storage) connected to a communication network 6. The recording device 5 records the camera images from each camera 2.

[0018] [Self-POS terminal explanation] FIG. 2 is a perspective view showing the exterior configuration of the self-service POS terminal 1, and FIG. 3 is a block diagram showing the main circuit configuration of the self-service POS terminal 1. As shown in FIG. 2, the self-service POS terminal 1 includes a main body 101 installed on the floor, with a basket stand 102 and a bagging stand 103 sandwiched between the main body 101. The basket stand 102 is for a customer coming from the sales floor to place a basket containing purchased items, and the bagging stand 103 is for a customer to place a plastic shopping bag or a shopping bag (so-called "my bag") brought by the customer. The main body 101 is equipped with a display pole 104, a touch panel 105, and a camera pole 106 on its upper portion. The customer works while standing in front of the main body 101 in FIG. 2 so that the screen of the touch panel 105 can be seen. Therefore, from the customer's perspective, the basket stand 102 is on the right side of the main body 101, and the bagging stand 103 is on the left side. In the following explanation, the side where the purchaser stands is referred to as the front of the main body 101, the side where the bagging table 103 is installed is referred to as the left side of the main body 101, and the side where the basket table 102 is installed is referred to as the right side of the main body 101.

[0019] The display pole 104 has a light emitting unit 107 at its tip that selectively emits light, for example, in blue or red. The display pole 104 displays the status of the self-service POS terminal 1, such as standby, operating, calling, or error, depending on the color of light emitted by the light emitting unit 107.

[0020] The touch panel 105 is composed of a display for displaying various screens related to transactions such as registration or accounting to the user operating the self-service POS terminal 1, and a touch sensor for detecting touch inputs made by the user on the screen. In the self-service POS terminal 1, the user is usually a purchaser. The touch panel 105 can also display information transmitted from the detection terminal 3 via the communication network 6.

[0021] Camera 2 is attached to the tip of camera pole 106. The angle of view, or imaging area, of camera 2 includes the basket stand 102 on the right side of main body 101 to bagging stand 103 on the left side, as shown by the dashed line in FIG. 2. In other words, the mounting position and orientation of camera 2 are set so that the imaging area includes the range of movement of the user when operating self-service POS terminal 1. More specifically, camera 2 is set so that its imaging area can capture images from above of at least the opening of a shopping basket placed on basket stand 102 or a shopping cart positioned on basket stand 102, the opening for inserting items into a plastic bag or personal bag placed on bagging stand 103, and the path of items taken from the shopping basket to be inserted into a plastic bag or personal bag placed on bagging stand 103.

[0022] The main body 101 has, on its front surface, a reading window 109 for a scanner 108 (see FIG. 3), a card insertion slot 111 for a card reader 110 (see FIG. 3), and an issuing slot 113 for receipts printed by a printer 112 (see FIG. 3). Furthermore, the main body 101 also has, on its front surface, a coin insertion slot 115, a coin dispensing slot 116, a bill insertion slot 117, and a bill dispensing slot 118 for an automatic change dispenser 114 (see FIG. 3).

[0023] In addition, a communication cable 119 extends from the right side of the main body 101 to the outside, and a payment terminal 120 for cashless payment is connected to the tip of this communication cable 119. The payment terminal 120 is placed on a stand 121 provided on the upper right side of the main body 101.

[0024] The bagging table 103 includes a scale unit 122. The scale unit 122 measures the total weight of the products placed on the bagging table 103.

[0025] As shown in FIG. 3, the self-service POS terminal 1 includes the touch panel 105, light-emitting unit 107, scanner 108, card reader 110, printer 112, automatic change dispenser 114, payment terminal 120, and scale unit 122 described above, as well as a processor 11, main memory 12, auxiliary storage device 13, clock 14, communication interface 15, and system transmission path 16. The system transmission path 16 includes an address bus, a data bus, control signal lines, and the like. The system transmission path 16 connects the processor 11 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 11, main memory 12, auxiliary storage device 13, clock 14, and communication interface 15 are connected via the system transmission path 16 to form a computer for the self-service POS terminal 1.

[0026] The processor 11 corresponds to the central part of the computer. The processor 11 controls each part to realize various functions of the self-service POS terminal 1 in accordance with an operating system or an application program. The processor 11 is, for example, a CPU (Central Processing Unit). The processor 11 may be, for example, an MPU (Micro Processing Unit), an SoC (System on a Chip), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field-Programmable Gate Array). Alternatively, the processor 11 may be a combination of two or more of these.

[0027] The main memory 12 corresponds to the main storage portion of the computer. The main memory 12 includes a nonvolatile memory area and a volatile memory area. The main memory 12 stores an operating system or application programs in the nonvolatile memory area. The main memory 12 may store data required for the processor 11 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 12 uses the volatile memory area as a work area where data is rewritten by the processor 11 as appropriate. The nonvolatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0028] The auxiliary storage device 13 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 13 may be an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device 13 stores data used by the processor 11 when performing various processes, data created by the processes in the processor 11, etc. The auxiliary storage device 13 may also store the application programs described above.

[0029] The clock 14 keeps track of the date and time. The processor 11 processes the date and time kept by the clock 14 as the current date and time.

[0030] The communication interface 15 performs data communication in accordance with a preset communication protocol with an external device connected via the communication network 6. The external device is typically the detection terminal 3. The external device may also include the monitoring terminal 4 or other devices such as a server (not shown).

[0031] Scanner 108 reads a code symbol from a product held over reading window 109. Each product sold in a store is affixed with a code symbol representing a product code or the like for identifying the product. The code symbol is, for example, a barcode. The code symbol may also be, for example, a two-dimensional data code. Scanner 108 may be of a type that reads a code symbol by scanning with laser light, or of a type that reads a code symbol from an image captured by an imaging device.

[0032] Card reader 110 reads card data recorded on a card medium such as a credit card, a point card, etc. Card reader 110 pulls the card medium inserted into card insertion slot 111 into main body 101, reads the card data, and then ejects the card from card insertion slot 111.

[0033] The printer 112 prints receipt data and other information indicating the details of the transaction on receipt paper. The receipt paper on which the receipt data is printed is ejected from the issuing port 113, cut by a cutter (not shown), and issued as a receipt or proof of purchase.

[0034] The automatic change dispenser 114 includes a coin unit and a banknote unit. The coin unit sorts coins inserted into a coin insertion slot 115 one by one, identifies the denomination, and stores them in a safe according to the denomination. The coin unit takes out coins of the appropriate denomination from the safe based on, for example, change data, and dispenses them to a coin dispensing outlet 116. The banknote unit sorts banknotes inserted into a banknote insertion slot 117 one by one, identifies the denomination, and stores them in a safe according to the denomination. The banknote unit takes out banknotes of the appropriate denomination from the safe based on, for example, change data, and dispenses them to a banknote dispensing outlet 118.

[0035] The payment terminal 120 reads and rewrites electronic money recorded on, for example, an electronic money medium. The electronic money medium is, for example, a contactless IC card. The electronic money medium may also be an electronic device such as a smartphone or a tablet terminal.

[0036] The self-service POS terminal 1 configured as described above uses a portion of the volatile memory area in the main memory 12 as the area for the transaction file 17. The transaction file 17 is a data file for storing data related to one commercial transaction processed by the self-service POS terminal 1. The transaction file 17 stores data such as the transaction number, purchased product data, total number of items, total amount, discount amount, and settlement amount. The transaction number is a series of numbers that is issued each time a commercial transaction is processed by the self-service POS terminal 1. The purchased product data is record data created for each product sold in a commercial transaction identified by the transaction number. Here, the purchased product data consists of items such as the product code, product name, price, number of items, and amount. The number of items is the number of items purchased for the product identified by the product code. The amount is the amount for the number of items purchased. The transaction file 17 can store multiple purchased product data. The total number of items is the sum of the points of each purchased product data. The total amount is the sum of the amounts of each purchased product data. The discount amount is the amount discounted from the total amount. The settlement amount is the total amount minus the discount amount.

[0037] [Detection device configuration explanation] 4 is a block diagram showing an example of the main circuit configuration of the detection terminal 3. The detection terminal 3 includes a processor 31, a main memory 32, an auxiliary storage device 33, a clock 34, a communication interface 35, and a system transmission path 36. The system transmission path 36 includes an address bus, a data bus, a control signal line, etc. The system transmission path 36 connects the processor 31 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The computer of the detection terminal 3 is configured by connecting the processor 31, the main memory 32, the auxiliary storage device 33, the clock 34, and the communication interface 35 via the system transmission path 36. The computer of the detection terminal 3 can also be referred to as the computer of the transaction processing device.

[0038] The processor 31 corresponds to the central part of the computer. The processor 31 controls each part to realize various functions of the detection terminal 3 in accordance with an operating system or an application program. The processor 31 is, for example, a CPU. The processor 31 may be a multi-core / multi-threaded processor, and can execute multiple processes in parallel. The processor 31 may be, for example, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 31 may be a combination of two or more of these.

[0039] The main memory 32 corresponds to the main storage portion of the computer. The main memory 32 includes a nonvolatile memory area and a volatile memory area. The main memory 32 stores an operating system or application programs in the nonvolatile memory area. The main memory 32 may store data required for the processor 31 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 32 uses the volatile memory area as a work area where data is rewritten by the processor 31 as appropriate. The nonvolatile memory area is, for example, ROM. The volatile memory area is, for example, RAM.

[0040] The auxiliary storage device 33 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM, HDD, or SSD can be the auxiliary storage device 33. The auxiliary storage device 33 stores data used by the processor 31 when performing various processes, data created by the processes in the processor 31, etc. The auxiliary storage device 33 may also store the application programs described above.

[0041] The application programs stored in the main memory 32 or the auxiliary storage device 33 include, for example, an information processing program for causing the computer to function as the information processing device according to the first embodiment, and a video analysis program for implementing an AI-based video analysis engine. The method for installing these programs in the main memory 32 or the auxiliary storage device 33 is not particularly limited. The program can be installed in the main memory 32 or the auxiliary storage device 33 by recording it on a removable recording medium or by distributing it via communication over a network. The recording medium may be in any form, such as a CD-ROM or memory card, as long as it can store the program and is readable by the device.

[0042] The clock 34 keeps track of the date and time. The processor 31 processes the date and time kept by the clock 34 as the current date and time.

[0043] The communication interface 35 performs data communication in accordance with a preset communication protocol with external devices connected via the communication network 6. The external devices are, for example, the self-service POS terminal 1, the camera 2, the monitoring terminal 4, the recording device 5, etc.

[0044] In the detection terminal 3 configured as above, part of the storage area of ​​the auxiliary storage device 33 is used as an area for the report table 331, the camera image storage unit 332, and the event log storage unit 333.

[0045] Report table 331 is a data table that describes data of reportable action items and detection flags in association with reportable action IDs, as shown in Fig. 5. The reportable action ID is unique identification information set for each reportable action to identify the action that requires a report from the purchaser to the transaction processing device, i.e., self-service POS terminal 1, that is, the type of reportable action.

[0046] The report action items are names that represent actions that require a report from the purchaser to the self-service POS terminal 1. In this embodiment, five items are exemplified as report action items: "missed scan," "weight error," "high-priced item error," "item under cart error," and "large number of items purchased." The types of each report action will be described in detail later.

[0047] The detection flag is 1-bit data for identifying whether or not a reportable action of a reportable action item identified by a corresponding reportable action ID has been detected. In this embodiment, a detection flag indicating that a reportable action has not been detected is set to "0," and a detection flag indicating that a reportable action has been detected is set to "1."

[0048] The camera image storage unit 332 sequentially stores camera images from the corresponding camera 2. The event log storage unit 333 sequentially stores a log of POS events from the self-service POS terminal 1 and a log of camera events determined from the camera images. The camera image storage unit 332 and the event log storage unit 333 can delete the stored contents at any time, such as outside the store's business hours, as needed.

[0049] Returning to the explanation of Figure 4. 4, the processor 31 of the detection terminal 3 has functions as a detection means 311, an acquisition means 312, a reporting means 313, and a second reporting means 314. These functions are realized by performing information processing in accordance with the information processing program and the video analysis program described above.

[0050] Detection means 311 has a function of detecting an action by a purchaser that requires a report to the transaction processing device, i.e., self-service POS terminal 1, i.e., a reporting action. Detection means 311 detects reporting actions by type based on a first event, i.e., a camera event, determined by the video analysis engine based on the output of a sensor that detects the purchaser's behavior, i.e., camera 2, and a second event, i.e., a POS event, that indicates the purchaser's behavior and is reported from self-service POS terminal 1. Detection means 311 detects reporting actions categorized into the five categories mentioned above, for example, "omitted scan," "weight error," "high-priced item error," "item below cart error," and "large purchase."

[0051] The acquisition means 312 has a function of acquiring the type of notification action detected by the detection means 311. The acquisition means 312 stores information indicating the acquired type of notification action. The information indicating the type of notification action is each detection flag in the notification table 331. That is, the acquisition means 312 updates the detection flag in the notification table 331 corresponding to the type of notification action detected by the detection means 311 from "0" to "1" and stores the information indicating the type of notification action. For example, when the notification action detected by the detection means 311 is a notification action of the notification action item "omission of scan," the acquisition means 312 updates the detection flag Fa from "0" to "1" and stores that the detected notification action is the notification action item "omission of scan." When the notification action detected by the detection means 311 is a notification action of the notification action item "weight error," the acquisition means 312 updates the detection flag Fb from "0" to "1" and stores that the detected notification action is the notification action item "weight error." When the reporting action detected by the detection means 311 is a reporting action for the reporting action item "high-priced item error," the acquisition means 312 updates the detection flag Fc from "0" to "1" and stores that the detected reporting action is the reporting action item "high-priced item error." When the reporting action detected by the detection means 311 is a reporting action for the reporting action item "item below cart error," the acquisition means 312 updates the detection flag Fd from "0" to "1" and stores that the detected reporting action is the reporting action item "item below cart error." When the reporting action detected by the detection means 311 is a reporting action for the reporting action item "large purchase items," i.e., when the number of purchase items reaches the number that should be reported as a reporting action, the acquisition means 312 updates the detection flag Fe from "0" to "1" and stores that the detected reporting action is the reporting action item "large purchase items."

[0052] The reporting means 313 has a function of reporting when the acquisition means 312 acquires multiple types of actions requiring reporting in one transaction. That is, the reporting means 313 issues a report when two or more types of detection flags become "1" in one transaction. For example, when the detection flag Fa and the detection flag Fb become "1," the reporting means 313 reports that the reportable action items "missing scan" and "weight error" have been detected. The reporting means 313 issues a report to the transaction processing device of the purchaser who performed the reportable action, i.e., the self-service POS terminal 1. The reporting means 313 issues a report when the threshold value of the judgment criteria for the reportable action is exceeded.

[0053] The second reporting means 314 is a function that reports to the monitor's terminal, that is, the monitoring terminal 4, when a reporting action is detected by the detection means 311. The second reporting means 314 reports to the monitoring terminal 4 when a reporting action is detected, regardless of the type of reporting action.

[0054] [Detection terminal operation explanation] FIG. 6 is a sequence diagram for explaining the operation of the detection terminal 3 regarding the detection and notification of a reporting action, together with the operation of the camera 2 and the self-checkout POS terminal 1 of the same register system RS.

[0055] As shown in FIG. 6, camera 2 captures images (ACT11) and outputs the captured camera footage to the corresponding detection terminal 3 (ACT12). For example, camera 2 may have a human detection function and transmit the captured camera footage while a customer is captured in the camera footage. Alternatively, camera 2 may have a moving object detection function instead of a human detection function and transmit the camera footage while a moving object is captured in the camera footage. Alternatively, camera 2 may not have such a function itself, but instead have a separate human detection sensor or the like installed in self-service POS terminal 1, and capture and transmit the camera footage only while a person is detected in the sensor output. This allows camera 2 to transmit the camera footage for each transaction by a customer. Camera 2 transmits the camera footage not only to detection terminal 3 but also to recording device 5. Recording device 5 records the camera footage in a data file in chronological order. For example, recording device 5 records the camera footage in a data file with a separate file name for each camera 2.

[0056] The detection terminal 3 receives the camera video, analyzes the camera video using an AI video analysis engine, detects the behavior of the customer, and determines the camera event corresponding to that behavior (ACT31). For example, when the detection terminal 3 detects from the camera video that the customer is removing a product from the cart stand 102, it determines the camera event corresponding to that behavior. Hereinafter, this camera event will be referred to as a product removal event CIa. The product removal event CIa may include the product code of the product that the customer removed from the cart stand 102, which is identified by the appearance feature data of the product.

[0057] For example, when the detection terminal 3 detects from the camera image that a purchaser holds a product over the reading window 109 of the scanner 108, it determines the camera event corresponding to that action. Hereinafter, this camera event will be referred to as a scan action event CIb. The scan action event CIb may include the product code of the product identified by the appearance feature data of the product that the purchaser holds over the reading window 109 of the scanner 108.

[0058] For example, when the detection terminal 3 detects from the camera image that a customer is putting products into a shopping bag or a personal bag placed on the bagging table 103, it determines the camera event corresponding to that action. Hereinafter, this camera event will be referred to as a product bagging event CIc. The product bagging event CIc may include the product code of the product that the customer placed in the shopping bag or personal bag placed on the bagging table 103, which is identified by the appearance feature data of the product.

[0059] When the detection terminal 3 detects from the camera image that a product has been placed in the bottom shelf of a shopping cart placed near the cart stand 102, it determines a camera event corresponding to the action of placing a product under the cart. Hereinafter, this camera event will be referred to as a "below-cart product placement event CId." The below-cart product placement event CId may include the product code of the product identified by the appearance feature data of the product placed under the cart. In this case, if two or more products are placed under the cart, the below-cart product placement event CId will include the product code of each product.

[0060] The types of camera events identified by the detection terminal 3 are not limited to the above-mentioned four types of product pick-up event CIa, scanning action event CIb, product bagging event CIc, and under-cart product storage event CId. The detection terminal 3 may also identify other types of camera events from the camera video.

[0061] When some operation event occurs by a purchaser (ACT21), the self-checkout POS terminal 1 outputs a POS event indicating the operation content to the corresponding detection terminal 3 (ACT22). For example, when the code symbol of a product held over the reading window 109 of the scanner 108 is read, the self-checkout POS terminal 1 outputs a POS event indicating the operation content. Hereinafter, this POS event will be referred to as a registered event PIa. The registered event PIa includes the product code identified from the code symbol read by the scanner 108.

[0062] For example, when a product is placed on the bagging table 103 and the measured weight of the scale unit 122 changes, the self-service POS terminal 1 outputs a POS event indicating the operation. Hereinafter, this POS event will be referred to as a weight event PIb. The weight event PIb includes the measured weight after the change.

[0063] For example, when the [Checkout] touch button BTb displayed on the registration screen SCa (see FIG. 14) of the touch panel 105 is touched, the self-service POS terminal 1 outputs a POS event indicating the operation. Hereinafter, this POS event will be referred to as a checkout event PIc. When the purchaser has finished registering the purchased items, they touch the [Checkout] touch button BTb.

[0064] The types of POS events output from the self-checkout POS terminal 1 are not limited to the above-mentioned three types, namely, the registration event PIa, the weight event PIb, and the accounting event PIc. The self-checkout POS terminal 1 may output other types of POS events in response to the operation of the purchaser.

[0065] The detection terminal 3 detects a reporting action by the purchaser based on the POS event output from the self-service POS terminal 1 and the camera event determined based on the camera image from the camera 2 (ACT32). The detection terminal 3 detects a reporting action called, for example, "missed scan," "high-priced item error," "item under cart error," or "large purchase quantity." Alternatively, the detection terminal 3 detects a reporting action by the purchaser based on the POS event output from the self-service POS terminal 1 (ACT32). The detection terminal 3 detects a reporting action called, for example, "weight error." In this way, detection terminal 3 detects a customer's reporting behavior based on POS events and camera events. That is, self-service POS terminal 1, which outputs POS events, and camera 2, which outputs camera footage used to determine camera events, function as output means that output information necessary to detect a customer's behavior requiring a report to the transaction processing device. It is also possible to detect a customer's reporting behavior based solely on POS events, in which case self-service POS terminal 1 functions as output means. It is also possible to detect a customer's reporting behavior based solely on camera events, in which case camera 2 functions as output means.

[0066] The detection terminal 3 determines whether or not it has detected the reporting behavior of the purchaser (ACT33). If it detects the reporting behavior (YES in ACT33), the detection terminal 3 notifies the monitor that the reporting behavior has occurred. Specifically, the detection terminal 3 outputs a reporting event to notify the monitor terminal 4, which is the operator of the monitor, that the reporting behavior has occurred (ACT34).

[0067] The detecting terminal 3 also updates the detection flag described in association with the reporting action ID of the reporting action in the reporting table 331 from "0" to "1" (ACT 35). If the detection flag is already "1", it maintains that state.

[0068] The detection terminal 3 searches the notification table 331 and counts the number n of detection flags that have become "1" (ACT 36). The detection terminal 3 then determines whether the number n of detection flags that have become "1" is equal to or greater than a predetermined threshold N (ACT 37). The threshold N is a threshold that serves as a criterion for determining whether to notify the self-service POS terminal 1 of the notification action. In this embodiment, the threshold N is set to "2." When the number n of detection flags that have become "1" becomes equal to or greater than the threshold N (YES in ACT 37), the detection terminal 3 notifies the purchaser that the notification action has occurred. Specifically, the detection terminal 3 outputs a notification event for the purchaser to notify the self-service POS terminal 1, which is the operator, of the notification action (ACT 38). The self-service POS terminal 1 that has received the notification event notifies the purchaser by, for example, displaying information such as text or an image indicating the content of the notification action on the touch panel 105 (ACT 23). The self-service POS terminal 1 may also notify the purchaser of the content of the notification action by voice.

[0069] If the detection terminal 3 does not detect a reporting behavior in the operation of ACT 33 (NO in ACT 33), the detection terminal 3 skips the operations of ACT 34 to ACT 38. Similarly, if the detection terminal 3 performs the operation of ACT 37 and the number n of detection flags that have become "1" is less than the threshold value N (NO in ACT 37), the detection terminal 3 skips the operation of ACT 38.

[0070] The detection terminal 3 repeatedly executes the operations shown in ACT31 to ACT38 from the time the purchased items of one transaction are registered at the self-service POS terminal 1 until the transaction is settled. Thus, the detection terminal 3 detects the types of reporting actions that occur in one transaction based on the POS events output from the self-service POS terminal 1 and the camera events identified from the video captured by the camera 2. When the detection terminal 3 detects threshold N or more types of reporting actions, it outputs a reporting event to the self-service POS terminal 1 operated by the purchaser who performed the reporting action, to notify that multiple types of reporting actions have been detected. Meanwhile, each time a reporting action is detected, it outputs a reporting event to the monitoring terminal 4, which is the monitor's terminal, to notify that the reporting action has been detected.

[0071] [Detection terminal program explanation] 7 is a flowchart showing the main steps of the information processing executed by the processor 31 of the detecting terminal 3 operating as described above. The processor 31 executes the information processing of the steps shown in the flowchart of FIG. 7 in accordance with an information processing program stored in the main memory 32 or the auxiliary storage device 33.

[0072] That is, in ACT101, the processor 31 checks whether or not camera video has been acquired from the camera 2 corresponding to the detection terminal 3. If camera video has not been acquired, the processor 31 proceeds to ACT102. In ACT102, the processor 31 checks whether or not a POS event has been acquired from the self POS terminal 1 corresponding to the detection terminal 3. If a POS event has not been acquired, the processor 31 returns to ACT101. In this way, the processor 31 waits to acquire camera video or a POS event in ACT101 and ACT102.

[0073] When camera video is acquired in the standby state of ACT101 and ACT102, the processor 31 proceeds to ACT103. In ACT103, the processor 31 stores the camera video in the camera video storage unit 332. Then, in ACT104, the processor 31 uses an AI video analysis engine to analyze the camera video stored in the camera video storage unit 332 and check whether a camera event has been identified. If a camera event has not been identified, the processor 31 returns to the standby state of ACT101 and ACT102.

[0074] If a camera event is determined in ACT 104, the processor 31 proceeds to ACT 105. In ACT 105, the processor 31 stores the camera event in the event log storage unit 333.

[0075] On the other hand, if a POS event is acquired in the standby state of ACT101 and ACT102, the processor 31 proceeds to ACT 106. The processor 31 stores the POS event in the event log storage unit 333 in ACT106.

[0076] When the camera event or POS event is stored in the event log storage unit 333 in ACT105 or ACT106, the processor 31 proceeds to ACT107. Based on the camera event and POS event stored in the event log storage unit 333 in ACT107, the processor 31 detects a reporting action in the reporting action items set in the reporting table 331. The detection operation for each reporting action will be described in detail later.

[0077] The processor 31 checks whether or not a reporting action has been detected in ACT 108. If a reporting action has not been detected, the processor 31 returns to the standby state in ACT 101 and ACT 102.

[0078] When a reportable action of any of the reportable action items is detected in ACT108, the processor 31 proceeds to ACT109. The processor 31 notifies the monitor in ACT109. That is, the processor 31 transmits a report event to the monitoring terminal 4 to notify that a reportable action has been detected. The report event includes information indicating the reportable action item of the detected reportable action. The monitoring terminal 4 that receives the report event displays on its display information to notify the monitor that a reportable action of the reportable action item included in the report event has been detected in the detection terminal 3 that transmitted the report event (see FIG. 15).

[0079] After notifying the monitor, processor 31 proceeds to ACT110. Processor 31 acquires the type of the detected reporting behavior in ACT110. Then, processor 31 updates the detection flag described in the reporting table 331 in association with the reporting behavior ID of that type from "0" to "1" in ACT111. Note that if the detection flag is already "1," processor 31 maintains that state.

[0080] The processor 31 counts the number n of detection flags that have become "1" in ACT 112. Then, the processor 31 determines whether the number n of detection flags that have become "1" is equal to or greater than a predetermined threshold N (N=2 in this embodiment) in ACT 113. If the number n of detection flags that have become "1" is less than the threshold N, the processor 31 returns to the standby state of ACT 101 and ACT 102.

[0081] On the other hand, if the number n of detection flags that have become "1" is equal to or greater than the threshold value N, the processor 31 proceeds to ACT114. In ACT114, the processor 31 notifies the purchaser. That is, the processor 31 transmits a notification event to the corresponding self-service POS terminal 1 to notify that a notification action has been detected. The notification event includes information indicating the notification action items of two or more notification action IDs that are respectively associated with the two or more detection flags that have become "1". In the self-service POS terminal 1 that receives the notification event, for example, information to notify the purchaser that notification actions of two or more notification action items included in the notification event have been detected is displayed on a part of the screen of the touch panel 105 (see FIG. 14). After notifying the purchaser, the processor 31 returns to the standby state of ACT101 and ACT102.

[0082] Here, the processor 31 realizes the function of detection means 311 by processing ACT101 to ACT108. The processor 31 realizes the function of acquisition means 312 by processing ACT110 and ACT111. The processor 31 realizes the function of notification means 313 by processing ACT112 to ACT114. The processor 31 realizes the function of second notification means 314 by processing ACT109.

[0083] [Explanation of types of reporting behavior and detection operations] 8 is a schematic diagram showing the operation of detecting a reportable behavior for the reportable behavior item "failed to scan." The reportable behavior item "failed to scan" refers to the behavior of a purchaser not scanning the barcode of a purchased item that has been removed from the basket stand 102 and packed into a shopping bag or personal bag on the bagging stand 103 with the scanner 108 of the self-service POS terminal 1. For example, when a purchaser holds a purchased item over the reading window 109 of the scanner 108 but does not hold the barcode correctly, and the barcode is not read when the item is packed, this behavior corresponds to the reportable behavior for the reportable behavior item "failed to scan."

[0084] For such an action, the event log storage unit 333 stores a product removal event CIa for the camera event, followed by a scanning action event CIb, and then a product bagging event CIc. On the other hand, for the POS event, only a weight event PIb is stored, and a registration event PIa is not stored.

[0085] The detection terminal 3 detects that a report action of the report action item "failed to scan" has occurred when the product removal event CIa, the scanning action event CIb, and the product bagging event CIc are stored in that order, and the weight event PIb is stored but the registered event PIa is not stored. Note that a case in which a purchaser bags the purchased products without holding them over the reading window 109 also constitutes a report action of the report action item "failed to scan." In other words, the detection terminal 3 detects that a report action of the report action item "failed to scan" has occurred even when the product removal event CIa and the product bagging event CIc are stored and the weight event PIb is stored but the registered event PIa and the scanning action event CIb are not stored.

[0086] When a reportable action of the reportable action item "omitted scan" is detected, the detection flag Fa becomes "1." Then, when the number n of detection flags that have become "1" becomes equal to or exceeds the threshold N, the detection terminal 3 issues a report to the purchaser. At this time, the report event sent to the self-service POS terminal 1 includes information indicating the reportable action item "omitted scan" and information indicating the reportable action item of the reportable action ID associated with the other detection flag that has become "1."

[0087] FIG. 9 is a schematic diagram illustrating the detection of a reporting action for the reporting action item "weight error." The reporting action item "weight error" refers to a case in which the purchased items removed by a customer from the cart stand 102 and packed into a shopping bag or personal bag on the bagging stand 103 do not match the items identified by the barcodes scanned by the scanner 108 of the self-service POS terminal 1. For example, a customer removing two purchased items from a shopping cart and scanning only one of the items to bag the two items corresponds to the reporting action for the reporting action item "weight error." For example, a pack of multiple identical items is printed with a barcode corresponding to the pack price, while each individual item is printed with a barcode corresponding to the individual item price. Removing such a pack of items from the cart stand 102 and scanning the barcode corresponding to the individual item price instead of the barcode corresponding to the pack price to bag the items also corresponds to the reporting action for the reporting action item "weight error."

[0088] For such actions, the event log storage unit 333 stores a product removal event CIa for the camera event, followed by a scanning action event CIb, and then a product bagging event CIc. On the other hand, for the POS event, a registration event PIa is stored, followed by a weight event.

[0089] When a product removal event CIa, a scanning action event CIb, and a product bagging event CIc are stored in this order, and a registered event Pia and a weight event PIb are stored in this order, the detection terminal 3 acquires the set weight Wa of the product identified by the product code included in the registered event Pia (ACT201). The detection terminal 3 also acquires the measured weight included in the weight event (ACT202). The detection terminal 3 then calculates the amount of change Wb between the measured weight included in the immediately preceding weight event and the measured weight included in the current weight event (ACT203).

[0090] The detection terminal 3 calculates the difference W between the set weight Wa and the amount of change Wb (ACT 204). Then, the detection terminal 3 checks whether the difference W is equal to or greater than the judgment value Wx (ACT 205). The judgment value Wx is an arbitrary value that exceeds the allowable range ±α of the set weight Wa. If the weight of products with the same barcode is uniform, the judgment value Wx is "0" or a value close to "0". However, there are products with the same barcode, such as vegetables, fruits, and prepared foods, whose weights are not uniform. For such products, an appropriate judgment value Wx (Wx>α) that exceeds the allowable range ±α of the set weight Wa is set.

[0091] If the difference W is less than the judgment value Wx (NO in ACT205), the detection terminal 3 does not detect the report action item "weight error." On the other hand, if the difference W is equal to or greater than the judgment value Wx (YES in ACT205), the detection terminal 3 detects that a report action of the report action item "weight error" has occurred.

[0092] When a report action of the report action item "weight error" is detected, the detection flag Fb becomes "1." Then, when the number n of detection flags that have become "1" becomes equal to or exceeds the threshold N, the detection terminal 3 notifies the purchaser. At this time, the report event sent to the self-service POS terminal 1 includes information indicating the report action item "weight error" and information indicating the report action item of the report action ID associated with the other detection flag that has become "1."

[0093] FIG. 10 is a schematic diagram illustrating the detection of a reporting action for the reporting action item "high-priced item error." The reporting action item "high-priced item error" refers to an action that occurs when a purchaser removes a high-priced item from the shopping cart stand 102 and packs it into a shopping bag or personal bag on the bagging stand 103, but the item identified by the barcode scanned by the scanner 108 of the self-service POS terminal 1 is not the high-priced item. For example, items with a unit price equal to or greater than a predetermined amount are preset as high-priced items. A reporting action for the reporting action item "high-priced item error" occurs when a purchaser removes a high-priced item and other items from the shopping cart, scans only the other items, and packs them into a bag, while concealing the high-priced item. Another reporting action for the reporting action item "high-priced item error" occurs when a purchaser holds a high-priced item over the reading window 109 of the scanner 108 but packs it without reading the barcode because the barcode is not correctly held over the scanner. The latter action also corresponds to the reporting action of the reporting action item "missed scan."

[0094] For such actions, the event log storage unit 333 stores a product removal event CIa for the camera event, followed by a scanning action event CIb, and then a product bagging event CIc. On the other hand, for the POS event, a registration event PIa is stored, followed by a weight event.

[0095] The detection terminal 3 checks whether the product has been identified from the camera image of the purchaser removing the product from the cart 102, i.e., whether the product code is included in the product removal event CIa (ACT211). If the product code is included (YES in ACT211), the detection terminal 3 determines whether the product identified by the product code is a product that has been set as a high-priced product in advance (ACT212). If the product code is not included in the product removal event CIa (NO in ACT211) or if the product identified by the product code is not a high-priced product (NO in ACT212), the detection terminal 3 does not detect the reporting action item "high-priced product error."

[0096] On the other hand, if the product identified by the product code is a high-priced product (YES in ACT212), the detection terminal 3 continues to detect the report action item "high-priced product error." The detection terminal 3 acquires the product code from the registered event PIa (ACT213). The detection terminal 3 determines whether the product code matches the product code of a high-priced product (ACT214). If the product codes match (NO in ACT214), the detection terminal 3 does not detect the report action item "high-priced product error." On the other hand, if the product codes do not match (YES in ACT214), the detection terminal 3 detects that a report action for the report action item "high-priced product error" has occurred.

[0097] When a report action of the report action item "high-priced item error" is detected, the detection flag Fc becomes "1." Then, when the number n of detection flags that have become "1" becomes equal to or exceeds the threshold N, the detection terminal 3 notifies the purchaser. At this time, the report event sent to the self-service POS terminal 1 includes information indicating the report action item "high-priced item error" and information indicating the report action item of the report action ID associated with the other detection flag that has become "1."

[0098] 11 is a schematic diagram showing the operation of detecting a reporting action for the reporting action item "Error in the bottom of the cart." The reporting action item "Error in the bottom of the cart" refers to the action of a purchaser completing the registration of a purchased item without scanning the purchased item placed in the bottom row of the shopping cart.

[0099] For such actions, the event log storage unit 333 stores an under-cart product storage event CId for the camera event. On the other hand, for the POS event, a checkout event PIc is stored. When the detection terminal 3 acquires an under-cart product storage event CId, it recognizes the product identified by the under-cart product storage event CId (ACT221). That is, the detection terminal 3 acquires the product code included in the under-cart product storage event CId. At this time, if the under-cart product storage event CId includes product codes for two or more items, it acquires each product code.

[0100] After that, when the detection terminal 3 acquires the accounting event PIc, it searches for the purchased items registered in the transaction file 17 of the corresponding self-service POS terminal 1 (ACT222). The detection terminal 3 then checks whether the items stored under the cart are registered in the transaction file 17 (ACT223). That is, the detection terminal 3 checks whether the purchased item data, including the item code included in the under-cart item storage event CId, is registered in the transaction file 17. If all items stored under the cart are registered in the transaction file 17 (NO in ACT223), the detection terminal 3 does not detect the report action item "under-cart item error." If even one item stored under the cart is not registered in the transaction file 17 (YES in ACT223), the detection terminal 3 detects that a report action for the report action item "under-cart item error" has occurred.

[0101] Generally, items stored in the bottom of a shopping cart are often large items that do not fit in the shopping basket. Therefore, if the detection terminal 3 is unable to acquire a product code from the bottom-of-cart product storage event CId in ACT221, it searches in ACT222 to determine whether or not any products classified as large items are registered in the transaction file 17. If no products classified as large items are registered in the transaction file 17 (YES in ACT223), the detection terminal 3 may detect that a reporting action for the reporting action item "bottom-cart product error" has occurred.

[0102] When a report action of the report action item "item error in cart" is detected, the detection flag Fc becomes "1." Then, when the number n of detection flags that have become "1" becomes equal to or exceeds the threshold N, the detection terminal 3 notifies the purchaser. At this time, the report event sent to the self-service POS terminal 1 includes information indicating the report action item "item error in cart" and information indicating the report action item of the report action ID associated with the other detection flag that has become "1."

[0103] FIG. 12 is a schematic diagram showing the operation of detecting a reportable behavior for the reportable behavior item "large purchase quantity." The reportable behavior item "large purchase quantity" is an behavior in which a purchaser purchases a large amount of merchandise in one transaction. For example, if a purchaser purchases 10 or more items, the detection terminal 3 notifies the monitoring terminal 4 and the self-service POS terminal 1 that the reportable behavior item "large purchase quantity" has been detected.

[0104] In response to such actions, a product pick-up event CIa, a scanning action event CIb, and a product bagging event CIc are repeatedly stored for camera events in the event log storage unit 333. On the other hand, a registration event PIa and a weight event are repeatedly stored for POS events.

[0105] The detection terminal 3 stores "0" as the default purchase number (ACT231). Subsequently, when the product removal event CIa, the scanning action event CIb, and the product bagging event CIc, which are camera events, are stored in the event log storage unit 333, and the registration event PIa and the weight event, which are POS events, are stored, the detection terminal 3 adds "1" to the purchase number (ACT232).

[0106] The detection terminal 3 checks whether the number of items purchased is equal to or greater than the notification threshold (ACT233). The notification threshold is the value of the number of items purchased at which it is determined that the reportable behavior item "number of items purchased: large" has been detected. For example, if a purchaser purchases 10 or more items and it is assumed that the reportable behavior item "number of items purchased: large" has been detected, the notification threshold is "10". If the number of items purchased is less than the threshold (YES in ACT233), the detection terminal 3 continues the detection operation for the reportable behavior item "number of items purchased: large".

[0107] The detection terminal 3 adds "1" to the number of purchases (ACT234) and repeats the process of comparing it with the reporting threshold (ACT235) each time a product removal event CIa, a scanning action event CIb, and a product bagging event CIc, which are camera events, are stored in the event log storage unit 333, and a registration event PIa and a weight event, which are POS events, are stored in the event log storage unit 333. As a result, when the number of purchases becomes equal to or greater than the reporting threshold (YES in ACT235), the detection terminal 3 detects that a reporting action of the reporting action item "Purchase number: large" has occurred.

[0108] When a reportable action of the reportable action item "large number of items purchased" is detected, the detection flag Fc becomes "1". Then, when the number n of detection flags that have become "1" becomes equal to or exceeds the threshold N, the detection terminal 3 issues a report to the purchaser. At this time, the report event sent to the self-service POS terminal 1 includes information indicating the reportable action item "large number of items purchased" and information indicating the reportable action items of the reportable action IDs associated with the other detection flags that have become "1".

[0109] [Explanation of the report example] 13 is an example of a registration screen SCa displayed on the touch panel 105 of the self-checkout POS terminal 1. The registration screen SCa is a screen used to register purchased products. The registration screen SCa is generated by the processor 11 of the self-checkout POS terminal 1.

[0110] As shown in FIG. 13, the registration screen SCa includes a touch button BTa for [Product without barcode] and a touch button BTb for [Checkout]. The touch button BTa for [Product without barcode] is used when registering products without barcodes, such as fresh food. In response to touching this touch button BTa, the self-service POS terminal 1 displays touch buttons corresponding to products without code symbols, so-called product buttons, and the purchaser can register the product by touching the product button corresponding to the purchased product. The touch button BTb for [Checkout] is used when the purchaser, who has finished registering the purchased products, declares that he or she wants to check out the transaction.

[0111] The registration screen SCa is divided into a registration list area ARa, a text area ARb, and a camera image area ARc. The registration list area ARa is an area for displaying a list of the product names, purchase quantities, and purchase amounts of purchased items registered by the purchaser. The text area ARb is an area for displaying text such as guidance and messages for the purchaser. The camera image area ARc is an area for displaying the current camera image being captured by the corresponding camera 2.

[0112] In this embodiment, when a notification event is received from the detection terminal 3, text indicating the content of the notification action is displayed in the text area ARb. FIG. 14 shows an example of the registration screen SCa of the self-service POS terminal 1 that receives a notification event when two types of notification actions, the notification action item "omitted scan" and the notification action item "error in item under cart," are detected. As shown in FIG. 14, text indicating the notification action for the notification action item "omitted scan," such as "Are there any items you haven't scanned?", and text indicating the notification action for the notification action item "error in item under cart," such as "Did you scan the items under cart?", are displayed in the text area ARb. Therefore, a customer who intentionally commits fraud by failing to scan or by not scanning an item under the cart is made aware that the store is aware of their fraudulent behavior, which helps prevent further fraud.

[0113] Fig. 15 is an example of a monitoring screen SCb displayed on the display of the monitoring terminal 4. As shown in Fig. 15, the monitoring screen SCb is divided into monitoring areas STa to STh for each register system RS. The monitoring areas STa to STh display the status of the self-service POS terminal 1 of the corresponding register system RS. Furthermore, when a notification event is received from the detection terminal 3 of the corresponding register system RS, text indicating the content of the notification action is displayed in the monitoring areas STa to STh.

[0114] FIG. 15 shows the monitoring screen SCb when the detection terminal 3 of the register system RS identified by register number "2" detects a report action for the report action item "missed scan," and the detection terminal 3 of the register system RS identified by register number "7" detects report actions for the report action item "missed scan" and the report action item "item under cart error." If the detection terminal 3 of the register system RS detects only one type of report action, no report is sent to the self-service POS terminal 1. However, if two or more types of report actions are detected, a report is also sent to the self-service POS terminal 1. Therefore, the monitoring terminal 4 makes it possible for the monitor to distinguish between the text of the report action for the register system RS for which only one type of report action has been detected, such as "missed scan," and the text of the report action for the register system RS for which two or more types of report actions have been detected, such as "missed scan" and "item under cart error," by changing the text color, font, etc., for example.

[0115] [Effects of the first embodiment] As described above in detail, according to the first embodiment, the detection terminal 3 stores a detection flag for each reportable act item identified by a reportable act ID. Then, when at least two or more reportable acts are detected in a single transaction and the respective detection flags become "1," the detection terminal 3 notifies the corresponding self-service POS terminal 1 that the two or more reportable acts have been detected. Therefore, compared to when a report is sent every time fraud in a reportable act item is detected, the frequency of reports is reduced, thereby reducing the risk of false positive reports to the purchaser. On the other hand, when two or more reportable acts are detected, a report is sent to the corresponding self-service POS terminal 1, enabling efficient and effective prevention or deterrence of fraudulent acts.

[0116] Furthermore, the detection terminal 3 notifies the monitoring terminal 4 each time a reportable behavior identified in a reportable behavior item is detected. Upon receiving this notification, the monitoring terminal 4 displays on its display that a reportable behavior identified in a reportable behavior item has been detected. This allows the monitor to know in real time that a reportable behavior has occurred, making it even more effective in preventing or deterring fraudulent behavior. The report destination may be the self-service POS terminal 1, or a monitor device other than a monitoring terminal.

[0117] [Second embodiment] Next, a second embodiment of the detector terminal 3 functioning as an information processing device will be described with reference to FIGS.

[0118] 16 is a series of sequence diagrams for explaining the operations related to the detection and reporting of a reporting action in the detection terminal 3 of the second embodiment, together with the operations of the camera 2 of the same register system RS and the self-service POS terminal 1. The same reference numerals are used to denote parts common to the sequence diagram of the first embodiment shown in FIG.

[0119] 6 and 16, the second embodiment differs from the first embodiment in the processing that occurs after it is determined in ACT 37 that the number of occurrences is equal to or greater than the notification threshold. That is, in the second embodiment, when the number n of detection flags that have become "1" becomes equal to or greater than the threshold N (YES in ACT 37), the detection terminal 3 sets the notification flag (report F) to "1" (ACT 41). At this time, the detection terminal 3 does not notify the purchaser.

[0120] When the detection terminal 3 receives a transaction event as a POS event from the self-service POS terminal 1 (YES in ACT42), it determines whether the notification flag is set to "1" (ACT43). If the notification flag is set to "1" (YES in ACT43), the detection terminal 3 notifies the purchaser (ACT44).

[0121] For example, if the detection flag Fa associated with the report action ID of the report action item "missed scan" and the detection flag Fd associated with the report action ID of the report action item "error in item under cart" are both "1", and the report flag becomes "1" in ACT41, the purchaser will be notified that the report actions of the report action items "missed scan" and "error in item under cart" have been detected.

[0122] 17 is a flowchart showing the main steps of the information processing executed by the processor 31 of the detector terminal 3 operating as described above. The same reference numerals are used to designate parts that are common to the flowchart of the first embodiment shown in FIG.

[0123] As is clear from comparing FIG. 7 and FIG. 17, in the second embodiment, the processing after acquiring a POS event in the standby state of ACT101 and ACT102 differs from that of the first embodiment. That is, in ACT121, processor 31 checks whether the POS event is a transaction event. That is, processor 31 checks whether the transaction event is detected by detecting that the transaction button BTb on the registration screen Sca in FIG. 14 has been pressed. If the transaction event is not detected, processor 31 proceeds to ACT106. Then, processor 31 executes the processing of ACT106 to ACT113 in the same manner as described in the first embodiment. If the number n of detection flags that have become "1" in ACT113 is equal to or greater than the threshold N, processor 31 proceeds to ACT122. In ACT122, processor 31 sets the notification flag to "1." The notification flag is one-bit data that identifies whether or not to report the notification behavior detected by the detection terminal 3. The default value of the notification flag is "0," indicating that no notification is to be performed. If the number n of detection flags that have become "1" in ACT113 is equal to or greater than the threshold N, the value becomes "1" indicating that a report should be made. The processor 31 that has set the report flag to "1" returns to the standby state of ACT101 and ACT102.

[0124] If the processor 31 acquires a accounting event in ACT 121, the processor 31 proceeds to ACT 123. In ACT 123, the processor 31 checks whether the notification flag is set to "1." If the result shows that the notification flag is set to "0," the processor 31 ends the information processing procedure shown in the flowchart of FIG. 17.

[0125] On the other hand, if the notification flag is set to "1", the processor 31 proceeds to ACT 124. The processor 31 notifies the purchaser in ACT 124. That is, the processor 31 transmits a notification event to the corresponding self-service POS terminal 1 to notify that notification actions of two or more types of notification action items for which the detection flag is set to "1" have been detected. With this, the processor 31 ends the information processing of the procedure shown in the flowchart of FIG. 17.

[0126] In this second embodiment, when the detection terminal 3 detects at least two or more reportable behavior items in a single transaction and the detection flag for each item becomes "1," the detection terminal 3 updates the report flag from the default "0" to "1." Then, when a transaction event is received from the corresponding self-service POS terminal 1, the detection terminal 3 notifies the detection of two or more reportable behavior items, provided that the report flag is "1." Therefore, compared to the first embodiment, in which a report is issued upon detection of at least two or more reportable behavior items, the frequency of the report is only one, further reducing the risk of false positives to the customer. Furthermore, because the report is issued after product registration is complete, it does not interfere with the product registration operation of customers unfamiliar with the self-service POS terminal 1. Therefore, the time required for product registration can be shortened. Meanwhile, the detection terminal 3 notifies the monitoring terminal 4 each time a reportable behavior identified by a reportable behavior item is detected. This effectively prevents or deters fraudulent activity.

[0127] [Third embodiment] Next, a third embodiment of the detector terminal 3 functioning as an information processing device will be described with reference to FIGS.

[0128] 18 is a block diagram showing the main circuit configuration of the detection terminal 3 according to the third embodiment. The same reference numerals are used to designate parts that are common to the block diagram of the detection terminal 3 according to the first embodiment shown in FIG.

[0129] 4 and 18, in the third embodiment, an area for a combination table 334 is added to the storage area of ​​the auxiliary storage device 33. As shown in Fig. 19, the combination table 334 is a data table in which a first detection flag (first detection F) and a second detection flag (second detection F) are set in association with different notification conditions (condition A, condition B, condition C, ...).

[0130] Here, the number of reporting conditions is arbitrary. Only one type of condition may be set in combination table 334, or four or more types of conditions may be set in combination table 334. Furthermore, the combination of the first detection flag and the second detection flag corresponding to one condition is also arbitrary. Furthermore, it is also possible to set only the first detection flag and not the second detection flag.

[0131] 19, in association with reporting condition A, the detection flag Fa for the reporting action item "omitted scan" is set as the first detection flag, and the detection flag Fd for the reporting action item "error with item under cart" is set as the second detection flag. Also, in association with reporting condition B, the detection flag Fb for the reporting action item "weight error" is set as the first detection flag, and the detection flag Fd for the reporting action item "error with item under cart" is set as the second detection flag. On the other hand, for reporting condition C, only the detection flag Fc for the reporting action item "error with high-priced item" is set as the first detection flag, and the second detection flag is not set.

[0132] 18, the processor 31 of the detection terminal 3 equipped with such a combination table 334 has a function as a third notification means 315 in addition to the detection means 311, acquisition means 312, and second notification means 134. The third notification means 315 has a function of issuing a notification when the combination of types acquired by the acquisition means 312 in one transaction satisfies a preset condition. In other words, the third notification means 315 issues a notification to the self-checkout POS terminal 1 when the combination of types acquired by the acquisition means 312 satisfies the combination of the first detection flag and the second detection flag set in the combination table 334.

[0133] Specifically, if a report action of the report action item "missed scan" and a report action of the report action item "error in item under cart" are detected in one transaction, condition A is satisfied. Therefore, the third reporting means 315 reports to the self-service POS terminal 1. Similarly, if a report action of the report action item "weight error" and a report action of the report action item "error in item under cart" are detected in one transaction, condition B is satisfied. Therefore, the third reporting means 315 reports to the self-service POS terminal 1. On the other hand, if a report action of the report action item "error in high-priced item" is detected in one transaction, that alone satisfies condition C. Therefore, the third reporting means 315 reports to the self-service POS terminal 1.

[0134] 20 is a flowchart showing the main steps of information processing executed by the processor 31 of the detector terminal 3 in the third embodiment. The same reference numerals are used to designate parts that are common to the flowchart of the first embodiment shown in FIG.

[0135] As is clear from comparing FIG. 7 and FIG. 20, the third embodiment differs from the first embodiment in the processing that occurs after updating the detection flag of the detected notification behavior item to "1" in ACT111. That is, after updating the detection flag of the detected notification behavior item to "1" in ACT111, the processor 31 proceeds to ACT131. In ACT131, the processor 31 searches for notification conditions set in the combination table 334 based on the detection flag in the notification table 331. Specifically, the processor 31 checks the first detection flag and the second detection flag for each notification condition. For a notification condition for which the second detection flag is not set (for example, condition C), only the first detection flag is checked.

[0136] The processor 31 checks whether the condition is met in ACT132. If the first detection flag and the second detection flag are both set to "1", the processor 31 determines that the condition is met. For a notification condition in which the second detection flag is not set, if the first detection flag is set to "1", the processor 31 determines that the condition is met. If the condition is not met, the processor 31 returns to the standby state in ACT101 and ACT102.

[0137] If the condition is met, the processor 31 proceeds to ACT133. In ACT133, the processor 31 notifies the purchaser. That is, the processor 31 sends a notification event to the corresponding self-service POS terminal 1 to notify that a notification action has been detected. The notification event includes information indicating the notification action item of each notification action ID associated with the first detection flag and the second detection flag of the notification condition for which the condition is met. If the condition is met only with the first detection flag, the notification event includes information indicating the notification action item of the notification action ID associated with that first detection flag. After notifying the purchaser, the processor 31 returns to the standby state of ACT101 and ACT102.

[0138] Here, the processor 31 realizes the function of the third reporting means 315 by the processing of ACT131 to ACT133.

[0139] In the third embodiment having such a configuration, when a combination of reportable act items in which a reportable act is detected in one transaction satisfies the conditions of the combination of the first detection flag and the second detection flag set in the combination table 334, the detection terminal 3 reports to the corresponding self-service POS terminal 1. Therefore, in the third embodiment as well, the frequency of reports is reduced compared to when a report is made every time fraud in a reportable act item is detected, thereby reducing the risk of a false positive report to the purchaser.

[0140] Furthermore, in the third embodiment, it is possible to set only the first detection flag in the combination table 334, without setting the second detection flag. When such an operation is adopted, the detection terminal 3 notifies the corresponding self-checkout POS terminal 1 when it detects a reportable behavior of a reportable behavior item corresponding to the first detection flag. Therefore, for example, when a reportable behavior that causes serious damage to the store, such as a "high-priced item error," is detected, it is possible to set it to notify even if other reportable behaviors are not detected. As a result, fraudulent behavior that causes serious damage to the store can be reliably deterred.

[0141] In the third embodiment, as in the second embodiment, a notification may be sent to the self-checkout POS terminal 1 in response to a transaction instruction after the notification condition is met.

[0142] [Other embodiments] In the first embodiment, the reporting means 313 issues a report when two or more types of detection flags become "1" in one transaction. In this regard, the reporting means 313 may not issue a report when two types of detection flags become "1," but may issue a report when three or more types of detection flags become "1." In other words, the threshold N, which is the criterion for determining whether or not to report a reporting action to the self-checkout POS terminal 1, may be set to "3" or a greater value.

[0143] In the third embodiment, a notification is issued when the combination of detection flags that has become "1" matches the combination of conditions set in the combination table 334. In this regard, for example, a notification may be issued when the notification actions corresponding to the first detection flag and the second detection flag are detected a set number of times or more.

[0144] In the above embodiment, the self-service POS terminal 1 has a text area ARb and a camera image area ARc as part of the registration screen SCa. In this regard, as shown in FIG. 21, a customer display 8 is provided in the self-service POS terminal 1. The customer display 8 is provided, for example, next to the left side of the touch panel 105 that displays the registration screen SC, etc. The position of the customer display 8 is not limited to the position shown in FIG. 21. It may be placed anywhere as long as it can be confirmed by the customer.

[0145] As shown in FIG. 22, the self-service POS terminal 1 divides the screen SCd of the customer display 8 into a text area ARb and a camera image area ARc. Then, as shown on the screen SCe in the same figure, when a notification event is received from the detection terminal 3, the self-service POS terminal 1 displays in the text area ARb the notification action item specified by the notification event, such as the text "Are there any items not scanned?" indicating the notification action of "missed scan" and the text "did you scan the items under the cart?" indicating the notification action of the notification action item "cart item error." By providing the customer display 8 on the self-service POS terminal 1 in this way, the customer can more accurately recognize notifications. Furthermore, the registration screen SCa of the touch panel 105 can have a larger registration list area ARa. In the above embodiment, the detection terminal 3 is exemplified as one aspect of an information processing device that can efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers. The information processing device is not limited to the detection terminal 3. For example, the functions of the detection means 311, acquisition means 312, notification means 313, and second notification means 314 described in the first embodiment may be provided in other devices, such as the self-service POS terminal 1, the camera 2, the monitoring terminal 4, or the recording device 5, and each of these may function as an information processing device. Alternatively, instead of having all of the functions in the detection terminal 3, an information processing system may be realized by having some of the functions in the self-service POS terminal 1, the camera 2, the monitoring terminal 4, or the recording device 5. For example, in the first embodiment, the detection terminal 3 functions as the detection means 311. Meanwhile, the functions of the acquisition means 312, the reporting means 313, and the second reporting means 314 are provided by the self-service POS terminal 1. The self-service POS terminal 1 also has a reporting table 331. With this configuration, when the detection terminal 3 detects an action by the purchaser that requires a report to the self-service POS terminal 1, that is, a reporting action, it outputs the information to the self-service POS terminal 1. The self-service POS terminal 1 acquires the type of reporting action from the information and reports it to the monitoring terminal 4 at that time. Furthermore, when the self-service POS terminal 1 acquires the information on the reporting action, it identifies the type and sets the detection flag in the reporting table 331 to “1.” If the detection flags for multiple types of reporting actions are set to “1” in one transaction, the self-service POS terminal 1 notifies the user by displaying a pop-up on the touch panel 105, for example, to indicate that a reporting action has been detected. Even with this configuration, the advantageous effect of being able to efficiently and effectively prevent or deter fraudulent activities while reducing the risk of false positive notifications to purchasers can be achieved.

[0146] In the above embodiment, as shown in Fig. 1, a detection terminal 3 to which an information processing device is applied is provided for each register system RS. In this regard, as shown in Fig. 23, the detection terminal 3 is removed from the register system RS, and instead a detection server 7 is connected to the communication network 6 to configure a self-checkout system SYS. The detection server 7 is a computer configured to perform the functions of the detection terminal 3 in each register system RS. Even with this configuration, the same effects as those of the first or second embodiment can be achieved.

[0147] It goes without saying that the types of reporting actions detected by the detection means 311 are not limited to the five reporting action items of "omitted scanning," "weight error," "high-priced item error," "item under cart error," and "large purchase quantity." For example, when an action of bagging an item without a barcode without selecting it on the touch panel 105 is detected, the reporting action of the reporting action item "no barcode item error" may be detected. For example, when an action of touching the [checkout] touch button while an item remains in the cart stand 102 is detected, the reporting action of the reporting action item "omitted item registration error" may be detected.

[0148] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0149] 1...self-service POS terminal, 2...camera, 3...detection terminal, 4...monitoring terminal, 5...recording device, 6...communication network, 7...detection server, 8...purchaser display, 101...main body, 102...basket stand, 103...bagging stand, 105...touch panel, 108...scanner, 109...reading window, 122...scale unit, 311...detection means, 312...acquisition means, 313...reporting means, 314...second reporting means, 315...third reporting means, 331...reporting table, 332...camera image memory unit, 333...event log memory unit, 334...combination table.

Claims

1. a detection means for detecting an action by a purchaser that requires a notification to the transaction processing device; an acquisition means for acquiring the type of the action requiring notification detected by the detection means; a reporting means for reporting when the acquiring means acquires multiple types of actions requiring the notification in one transaction; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein said notification means notifies said transaction processing device of a purchaser who has performed a plurality of types of actions requiring notification.

3. The information processing device according to claim 2 , wherein the notification means issues a notification when a plurality of types of actions requiring notification are acquired.

4. 3. The information processing device according to claim 2, wherein said notification means issues a notification when said transaction processing device receives a transaction instruction from the purchaser.

5. a second reporting means for reporting to a monitor terminal when an act requiring a report is detected by the detection means; 5. The information processing apparatus according to claim 1, further comprising:

6. a detection means for detecting an action by a purchaser that requires a notification to the transaction processing device; an acquisition means for acquiring the type of the action requiring notification detected by the detection means; a third notification means for notifying that the combination of types acquired by the acquisition means satisfies a preset condition in one transaction; An information processing device comprising:

7. An information processing device as described in claim 1 or 6, wherein the detection means detects the type of action requiring notification based on a first event determined based on the output of a sensor that detects the behavior of the purchaser and a second event indicating the content of the purchaser's behavior reported from the transaction processing device.

8. The computer of the information processing device, detection means for detecting an action by a purchaser that requires notification to the transaction processing device; an acquisition means for acquiring the type of the action requiring notification detected by the detection means; and a reporting means for reporting when the acquiring means acquires multiple types of actions requiring the notification in one transaction; A program to function as a

9. an output means for outputting information necessary to detect an action by a purchaser that requires notification to the transaction processing device; a detection means for detecting an action by the purchaser that requires a report based on the information output by the output means; an acquisition means for acquiring the type of the action requiring notification detected by the detection means; a reporting means for reporting when the acquiring means acquires multiple types of actions requiring the notification in one transaction; An information processing system comprising:

Images