Information processing apparatus, storage medium, and information processing system

Abstract

The invention discloses an information processing apparatus, a storage medium, and an information processing system, and aims to reduce the risk of false detection notification to a purchaser and effectively prevent or suppress unauthorized behaviors with effectiveness. The information processing apparatus includes a detection section and a notification section. The detection unit detects a notification-requiring behavior of a purchaser to a transaction processing device. The notification unit performs notification in a transaction when the behavior detected by the detection unit exceeds a threshold value of a criterion for determination of the behavior.

Description

[0001] This application claims priority from Japanese Application No. JP 2024-070000, filed April 23, 2024, and the contents of the aforementioned application are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to an information processing apparatus and storage medium, and an information processing system. BACKGROUND

[0003] In recent years, due to a shortage of manpower or a rise in labor costs, a self-checkout system in which a purchaser who has made a purchase performs self-checkout from registration of a purchased product to payment of transaction money is becoming widespread. On the other hand, in the self-checkout system, there is a concern that an improper act such as bagging is performed without scanning a product. Therefore, a system that performs analysis of an image of an action of a purchaser by capturing the action with a camera using AI (Artificial Intelligence), and reports in the case where an improper act is detected has been developed.

[0004] However, in the system using image analysis based on AI, it is possible that an act that is not an improper act is mistakenly detected as an improper act. Therefore, when a report is made every time an improper act is detected, the risk of a mistaken detection report increases, which leads to a complaint from a purchaser. SUMMARY

[0005] In view of the above problem, the present application aims to provide an information processing apparatus and storage medium, and an information processing system that can reduce the risk of a mistaken detection report to a purchaser and can effectively prevent or suppress an improper act that has an effect.

[0006] To solve the above problem, an information processing apparatus according to an embodiment includes a detection unit and a reporting unit. The detection unit detects an act of a purchaser that requires a report to a transaction processing apparatus. The reporting unit reports in the case where the act detected by the detection unit exceeds a threshold value of a determination criterion for the act in one transaction.

[0007] According to the above information processing apparatus, since the frequency of reporting is reduced compared to the case where a report is made every time an act that requires a report is detected, it is possible to reduce the risk of a mistaken detection report to a purchaser.

[0008] In the above information processing apparatus, the reporting unit reports to the transaction processing apparatus for the purchaser who has performed the act that requires the report.

[0009] According to the information processing device described above, in a case where a behavior requiring notification above a threshold value is detected, the corresponding transaction processing device is notified, and thus, a fraudulent behavior having an actual effect can be effectively prevented or suppressed.

[0010] In the information processing device described above, the notification unit notifies at a time point at which a threshold value of a determination criterion for the behavior is exceeded.

[0011] According to the information processing device described above, when a threshold value or more is detected, notification can be made immediately.

[0012] In the information processing device described above, the notification unit notifies at a time point at which a purchaser's checkout instruction to the transaction processing device is accepted.

[0013] According to the information processing device described above, since the notification frequency is only once, the risk of a false detection notification to the purchaser can be further reduced.

[0014] In the information processing device described above, a second notification unit that notifies a monitor's terminal at a time point at which the detection unit detects a behavior requiring notification is further included.

[0015] According to the information processing device described above, the monitor can grasp in real time a case where a behavior requiring notification has occurred.

[0016] In the information processing device described above, the detection unit detects a behavior requiring the notification by category based on a first event identified based on an output of a sensor that detects the purchaser's action and a second event indicating a content of the purchaser's action notified by the transaction processing device.

[0017] According to the information processing device described above, a behavior requiring notification can be detected by category based on both a first event based on an output of a sensor and a second event notified by a transaction processing device.

[0018] In the information processing device described above, a setting list in which a threshold value of a determination criterion is stored for each category of a behavior requiring notification is further included, and the notification unit notifies in a case where a threshold value of a determination criterion of the setting list for a category detected by the detection unit is exceeded.

[0019] According to the information processing device described above, a threshold value can be set for each category of a behavior requiring notification.

[0020] In the information processing device described above, the first event is identified by analyzing a camera image of a camera as the sensor by an image analysis engine, and the second event is a content of the purchaser's action corresponding to the purchaser's operation to the transaction processing device notified by the transaction processing device.

[0021] According to the information processing apparatus described above, it is possible to perform category detection of a behavior requiring notification based on a camera image and an operation of a transaction processing apparatus.

[0022] Another aspect of the present application is a storage medium storing a program that causes a computer of an information processing apparatus to function as: a detection section that detects a behavior requiring notification by a purchaser with respect to a transaction processing apparatus; and a notification section that notifies in a transaction when a behavior detected by the detection section exceeds a threshold value of a criterion for the behavior.

[0023] According to the storage medium described above, it is possible to provide a computer with a function that reduces the risk of false detection notification to a purchaser.

[0024] Another aspect of the present application is an information processing system including: an output section that outputs information required to detect a behavior requiring notification by a purchaser with respect to a transaction processing apparatus; a detection section that detects the behavior requiring notification by the purchaser based on information output by the output section; and a notification section that notifies in a transaction when a behavior detected by the detection section exceeds a threshold value of a criterion for the behavior.

[0025] According to the information processing system described above, since the frequency of notification is reduced compared to a case where notification is performed each time a behavior requiring notification is detected, it is possible to reduce the risk of false detection notification to a purchaser. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a diagram showing the outline configuration of a self-checkout system according to an embodiment.

[0027] Figure 2 is a perspective view showing the external configuration of a self-POS terminal.

[0028] Figure 3 is a block diagram showing the main circuit configuration of a self-POS terminal.

[0029] Figure 4 is a block diagram showing an example of the main circuit configuration of a detection terminal.

[0030] Figure 5 is a diagram showing the main data structure of a setting list.

[0031] Figure 6 is a diagram showing the main data structure of a count list.

[0032] Figure 7is a series of timing charts for explaining actions involved in detection and notification of a notification behavior in the detection terminal together with actions of a camera and a self-service POS terminal of the same cash register system.

[0033] Figure 8 is a flowchart showing main steps of information processing performed by a processor of the detection terminal.

[0034] Figure 9 is a schematic diagram showing a notification behavior detection action of a notification behavior item "scanning omission".

[0035] Figure 10 is a schematic diagram showing a notification behavior detection action of a notification behavior item "weight error".

[0036] Figure 11 is a schematic diagram showing a notification behavior detection action of a notification behavior item "high-priced goods error".

[0037] Figure 12 is a schematic diagram showing a notification behavior detection action of a notification behavior item "goods under the cart error".

[0038] Figure 13 is a schematic diagram showing a notification behavior detection action of a notification behavior item "number of purchases: large".

[0039] Figure 14 is an example of a registration screen displayed on a touch panel of the self-service POS terminal.

[0040] Figure 15 is an example of a registration screen of the self-service POS terminal which has received a notification event of the notification behavior item "scanning omission".

[0041] Figure 16 is an example of a monitoring screen displayed on a display of the monitoring terminal.

[0042] Figure 17 is a series of timing charts for explaining actions involved in detection and notification of a notification behavior in the detection terminal of the second embodiment together with actions of a camera and a self-service POS terminal of the same cash register system.

[0043] Figure 18 is a flowchart showing main steps of information processing performed by a processor of the detection terminal in the second embodiment.

[0044] Figure 19 is an external configuration diagram of the self-service POS terminal in other embodiments.

[0045] Figure 20 is a migration diagram of a screen displayed on a purchaser display of the self-service POS terminal according to other embodiments.

[0046] Figure 21 is a schematic configuration diagram of a self-checkout system representing other embodiments.

[0047] Explanation of Reference Signs

[0048] 1…self-checkout POS terminal, 2…camera, 3…detection terminal, 4…monitoring terminal, 5…recording device, 6…communication network, 7…detection server, 8…purchaser display, 101…main body, 102…basket table, 103…bagging table, 105…touch panel, 108…scanner, 109…reading window, 122…scale unit, 311…detection section, 312…notification section, 313…second notification section, 331…setting list, 332…counting list, 333…camera image storage section, 334…event log storage section. DETAILED DESCRIPTION

[0049] Hereinafter, an embodiment of an information processing device capable of reducing the risk of false detection notification to a purchaser and effectively preventing or suppressing effective fraudulent behavior will be described using the drawings.

[0050] [First Embodiment]

[0051] First, an embodiment of a self-checkout system to which the information processing device is applied will be described.

[0052] [Description of Self-Checkout System]

[0053] Figure 1 is a schematic configuration diagram of a self-checkout system SYS of an embodiment. The self-checkout system SYS includes a plurality of self-checkout POS terminals 1, a plurality of cameras 2, a plurality of detection terminals 3, a monitoring terminal 4, and a recording device 5. Specifically, the self-checkout system SYS configures a register system RS by corresponding one camera 2 and one detection terminal 3 to each self-checkout POS terminal 1. Each register system RS is provided at a checkout place of a store. The self-checkout system SYS connects the self-checkout POS (Point of Sales) terminals 1, the cameras 2, and the detection terminals 3, the monitoring terminal 4, and the recording device 5 of each register system RS through a communication network 6 such as a LAN (Local Area Network). In the present embodiment, the detection terminal 3 functions as an information processing device capable of reducing the risk of false detection notification to a purchaser and effectively preventing or suppressing effective fraudulent behavior.

[0054] The purchaser who has finished shopping becomes a user of the self-service POS terminal 1. The purchaser who has put the purchased goods purchased at the store where the goods are displayed in a shopping basket or the like and moved to the checkout place operates the self-service POS terminal 1, and the self-service performs settlement of the transaction relating to the registration of the purchased goods to the payment of the money. The self-service POS terminal 1 is an example of a transaction processing device that is a full self-service corresponding device. The transaction processing device can also be a device that is not a full self-service corresponding device. The transaction processing device can also be a settlement device in which the registration of the purchased goods is performed in another device, and the purchaser himself or herself performs an operation relating to the payment of the money based on the data of the purchased goods registered by the device to settle the transaction.

[0055] The self-service POS terminal 1 transmits event information, that is, a POS event, indicating the operation content of the purchaser to the corresponding detection terminal 3 via the communication network 6 based on the correspondence relationship with the detection terminal 3 set in advance. The POS event is an example of the second event indicating the action content of the purchaser reported from the self-service POS terminal 1 to the detection terminal 3.

[0056] The camera 2 captures the action of the purchaser who operates the self-service POS terminal 1, and transmits the captured camera image to the corresponding detection terminal 3 via the communication network 6 according to the correspondence relationship with the detection terminal 3 set in advance. Note that, here, although via the communication network 6, the camera image can also be transmitted directly from the camera 2 to the detection terminal 3 by a USB (Universal Serial Bus) cable, for example. In this case, it is not necessary to set the correspondence relationship with the detection terminal 3 in the camera 2. The camera 2 can also transmit the camera image to the recording device 5 via the communication network 6.

[0057] The detection terminal 3 having a function as an information processing device is provided with an image analysis engine based on AI. The image analysis engine analyzes the camera image from the camera 2, detects the action of the purchaser, and identifies the camera event corresponding to the action. That is, the camera 2 is an example of a sensor that detects the action of the purchaser, and the camera event is an example of the first event identified based on the output of the sensor. The detection terminal 3 detects the behavior of the purchaser that needs to be reported based on the camera event identified by the image analysis engine and the POS event acquired from the self-service POS terminal 1. The detection terminal 3 sometimes also detects the behavior to be reported based on the POS event or the behavior to be reported based on the camera event. The detection terminal 3 reports the detected behavior, that is, the behavior to be reported, to the corresponding self-service POS terminal 1 via the communication network 6. The detection terminal 3 reports the behavior to be reported not only to the corresponding self-service POS terminal 1 but also to the monitoring terminal 4.

[0058] The monitoring terminal 4 is a terminal operated by a clerk who is a monitor of the plurality of self-service POS terminals 1 connected through the communication network 6. The monitoring terminal 4 has a display. The monitoring terminal 4 displays information required when monitoring the plurality of self-service POS terminals 1 on the display. The displayed information also includes information of the reported behavior detected by the detection terminal 3.

[0059] The clerk as a monitor monitors the self-service POS terminal 1 according to the information displayed on the display. The clerk as a monitor recognizes that the behavior considered to be an irregular behavior has been performed by the purchaser according to the information of the reported behavior detected by the detection terminal 3. The clerk as a monitor is, for example, called a server. The monitoring terminal 4 is an example of a terminal for a monitor. The monitoring terminal 4 can be any device as long as it can be connected to the communication network 6 by wireless. For example, it can be a tablet terminal which the clerk as a monitor can carry.

[0060] The recording device 5 is a storage device such as a NAS (Network Attached Storage) connected to the communication network 6. The recording device 5 records the camera images from each camera 2.

[0061] [Explanation of self-service POS terminal]

[0062] Figure 2 is a perspective view showing the external configuration of the self-service POS terminal 1, Figure 3 is a block diagram showing the main part circuit configuration of the self-service POS terminal 1. As shown in Figure 2 , the self-service POS terminal 1 has a main body 101 provided on the floor, and a basket table 102 and a bagging table 103 are provided with the main body 101 therebetween. The basket table 102 is used to place a basket or the like in which the purchaser from the store has put purchased goods, and the bagging table 103 is used to place a plastic bag or a shopping bag brought by the purchaser, that is, a so-called eco-bag. The main body 101 has a display pole 104, a touch panel 105, and a camera pole 106 mounted on the upper portion thereof. The purchaser stands in front of the main body 101 to perform an operation so as to be able to see the screen of the touch panel 105. Figure 2 Therefore, from the perspective of the purchaser, the basket table 102 is on the right side of the main body 101, and the bagging table 103 is on the left side thereof. In the following explanation, the side on which the purchaser stands is set as the front side of the main body 101, the side on which the bagging table 103 is provided is set as the left side of the main body 101, and the side on which the basket table 102 is provided is set as the right side of the main body 101.

[0063] The display pole 104 has, for example, a light emitting portion 107 which selectively emits blue and red light at the front end portion thereof. The display pole 104 displays the state of the self-service POS terminal 1, for example, in standby, in operation, in call, in error, and the like, according to the light emission color of the light emitting portion 107.

[0064] Touch panel 105 consists of a display for displaying various screens related to transactions such as registration and checkout to the user operating self-service POS terminal 1, and a touch sensor for detecting user touch input on the screen. In self-service POS terminal 1, the user is typically a purchaser. Touch panel 105 can also display information transmitted from detection terminal 3 via communication network 6.

[0065] The camera rod 106 is provided with a camera 2 at its front end. The field of view of the camera 2 is the shooting area, such as Figure 2 As shown by the dashed line, the camera 2 extends from the basket table 102 on the right side of the main body 101 to the bagging table 103 on the left side. Specifically, the camera 2 is positioned and positioned so that it fits within the user's range of motion when operating the self-service POS terminal 1. More specifically, the camera 2 is configured so that its imaging area can capture, from above, at least the opening of a shopping basket or a shopping cart positioned on the basket table 102, the opening for inserting merchandise into a plastic bag or reusable bag placed on the bagging table 103, and the path of merchandise as it is removed from the shopping basket and inserted into the plastic bag or reusable bag placed on the bagging table 103.

[0066] The main body 101 has a scanner 108 formed on its front surface (see Figure 3 ) reading window 109, card reader 110 (refer to Figure 3 ) card insertion port 111, and the printer 112 (see Figure 3 ) is provided at a dispensing port 113 for printing receipts. In addition, an automatic change machine 114 (see Figure 3 )'s coin input port 115, coin dispensing port 116, banknote input port 117 and banknote dispensing port 118.

[0067] A communication cable 119 extends outward from the right side of the main body 101. A cashless payment terminal 120 is connected to the distal end of the communication cable 119. The payment terminal 120 is placed on a stand 121 provided on the upper right side of the main body 101.

[0068] The bagging table 103 includes a scale unit 122. The scale unit 122 measures the total weight of commodities and the like placed on the bagging table 103.

[0069] like Figure 3As shown, the self-service POS terminal 1 is provided with, in addition to the above-described touch panel 105, light emitting section 107, scanner 108, card reader 110, printer 112, automatic change machine 114, settlement terminal 120, scale unit 122, a processor 11, main memory 12, auxiliary storage device 13, clock 14, communication interface 15, and system transmission path 16. The system transmission path 16 includes an address bus, a data bus, a control signal line, and the like. The system transmission path 16 connects the processor 11 and the other sections directly or via a signal input / output circuit, and transmits data signals exchanged therebetween. The processor 11, main memory 12, auxiliary storage device 13, clock 14, and communication interface 15 are connected by the system transmission path 16, thereby constituting a computer of the self-service POS terminal 1.

[0070] The processor 11 corresponds to a central part of the above-described computer. The processor 11 controls the sections in accordance with an operating system or an application program to realize various functions as the self-service POS terminal 1. The processor 11 can be, for example, a CPU (Central Processing Unit). The processor 11 includes, for example, an MPU (Micro Processing Unit), SoC (System on a Chip), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field-Programmable Gate Array), or the like. Alternatively, the processor 11 can be a processor composed of a plurality of combinations of these.

[0071] The main memory 12 corresponds to a main storage part of the above-described computer. The main memory 12 includes a non-volatile storage area and a volatile storage area. The main memory 12 stores an operating system or an application program in the non-volatile storage area. The main memory 12 can store data required by the processor 11 to execute processing for controlling the sections in the non-volatile or volatile storage area. The main memory 12 uses the volatile storage area as a work area in which data is appropriately rewritten by the processor 11. The non-volatile storage area is, for example, a ROM (Read Only Memory). The volatile storage area is, for example, a RAM (Random Access Memory).

[0072] The auxiliary storage device 13 corresponds to an auxiliary storage portion of the above-described computer. For example, an EEPROM (registered trademark) (Electric Erasable Programmable Read Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive) or the like can be the auxiliary storage device 13. The auxiliary storage device 13 stores data used when the processor 11 performs various processing, data generated by the processing of the processor 11, and the like. The auxiliary storage device 13 can also store the above-described application program.

[0073] The clock 14 counts the date and time. The processor 11 processes the date and time counted by the clock 14 as a current date and time.

[0074] The communication interface 15 performs data communication with an external device connected through the communication network 6 in accordance with a communication protocol set in advance. The external device is typically the detection terminal 3. The external device can include other devices, such as the monitoring terminal 4 or a server not illustrated.

[0075] The scanner 108 reads a code symbol of a product from an alignment reading window 109. A code symbol indicating a product code or the like for identifying the product is attached to each product sold at a store. The code symbol is, for example, a bar code. The code symbol can be, for example, a two-dimensional data code. The scanner 108 can be a type that reads a code symbol by scanning with laser light, or a type that reads a code symbol from an image captured by an imaging device.

[0076] The card reader 110 reads card data recorded in a card medium such as a credit card or a point card. The card reader 110 introduces a card medium inserted into a card insertion port 111 into the main body 101, reads card data, and discharges the card medium from the card insertion port 111.

[0077] The printer 112 prints, on a sheet of paper, receipt data indicating a product transaction content or the like. The sheet of paper on which the receipt data is printed is discharged from a dispensing port 113, cut by a cutter not illustrated, and dispensed as a receipt or a voucher.

[0078] The automatic change machine 114 includes a coin unit and a bill unit. The coin unit sorts coins one by one to identify the denomination of the coins, and stores the coins in a cash box by denomination. The coin unit, for example, takes out coins of a corresponding denomination from the cash box in accordance with change data, and dispenses the coins from a coin dispensing port 116. The bill unit sorts bills one by one to identify the denomination of the bills, and stores the bills in a cash box by denomination. The bill unit, for example, takes out bills of a corresponding denomination from the cash box in accordance with change data, and dispenses the bills from a bill dispensing port 118.

[0079] The payment terminal 120 reads and rewrites electronic money recorded in an electronic money medium, for example. The electronic money medium is, for example, a contactless IC card. The electronic money medium may also be an electronic device such as a smartphone or a tablet terminal.

[0080] The self-service POS terminal 1 with this configuration uses a portion of the volatile memory area within the main memory 12 as an area for the transaction file 17. The transaction file 17 is a data file used to store data related to a single commodity transaction processed by the self-service POS terminal 1. The transaction file 17 stores data such as the transaction number, purchased commodity data, total number of items, total amount, discount amount, and settlement amount. A transaction number is a serial number issued each time a commodity transaction is processed by the self-service POS terminal 1. Purchased commodity data is recorded for each commodity bought or sold in a transaction identified by a transaction number. The purchased commodity data consists of items such as the commodity code, commodity name, price, number of items, and amount. The number of items is the number of items purchased for the commodity identified by the commodity code. The amount is the amount corresponding to the number of items purchased. The transaction file 17 can store multiple pieces of purchased commodity data. The total number of items is the sum of the numbers of items in each piece of purchased commodity data. The total amount is the sum of the amounts in each piece of purchased commodity data. The discount amount is the amount obtained by subtracting the discount amount from the total amount. The settlement amount is the amount obtained by subtracting the discount amount from the total amount.

[0081] [Composition of the detection terminal]

[0082] Figure 4 This is a block diagram showing an example of the main circuit configuration of the detection terminal 3. The detection terminal 3 includes a processor 31, main memory 32, auxiliary storage device 33, clock 34, communication interface 35, and system transmission path 36. System transmission path 36 includes an address bus, a data bus, control signal lines, and other components. The system transmission path 36 connects the processor 31 to other components directly or via signal input / output circuits, transmitting data signals between them. The processor 31, main memory 32, auxiliary storage device 33, clock 34, and communication interface 35 are connected via the system transmission path 36, forming the computer of the detection terminal 3. The computer of the detection terminal 3 can be referred to as the computer of the transaction processing device.

[0083] The processor 31 is equivalent to the core of the computer. Based on the operating system or application program, the processor 31 controls various components to implement the various functions of the detection terminal 3. The processor 31 is, for example, a CPU. The processor 31 can be a multi-core / multi-threaded processor capable of executing multiple processes in parallel. The processor 31 can be, for example, an MPU, a SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 31 can be a combination of multiple processors.

[0084] The main memory 32 corresponds to the main storage portion of the above-described computer. The main memory 32 includes a non-volatile memory area and a volatile memory area. The main memory 32 stores an operating system or an application program in the non-volatile memory area. The main memory 32 can store data necessary for the processor 31 to execute processing for controlling each portion in the non-volatile or volatile memory area. The main memory 32 uses the volatile memory area as a work area in which data is appropriately overwritten by the processor 31. The non-volatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

[0085] The auxiliary storage device 33 corresponds to the auxiliary storage portion of the above-described computer. For example, an EEPROM, an HDD, or an SSD, or the like can be the auxiliary storage device 33. The auxiliary storage device 33 holds data used when the processor 31 performs various processing, data generated by the processing of the processor 31, and the like. The auxiliary storage device 13 can also store the above-described application program.

[0086] The application program stored in the main memory 32 or the auxiliary storage device 33 includes, for example, an information processing program for causing the above-described computer to function as the information processing apparatus of the first embodiment, and an image analysis program that realizes an AI-based image analysis engine. The method of installing these programs in the main memory 32 or the auxiliary storage device 33 is not particularly limited. The programs can be recorded in a removable storage medium, or distributed by communication via a network, and installed in the main memory 32 or the auxiliary storage device 33. If the storage medium is capable of storing the programs like a CD-ROM, a memory card, or the like, and the apparatus is capable of reading, it does not matter what form it is.

[0087] The clock 34 counts the date and time. The processor 31 processes the date and time counted by the clock 34 as the current date and time.

[0088] The communication interface 35 performs data communication with external devices connected through the communication network 6 in accordance with a communication protocol set in advance. The external devices are, for example, the self-service POS terminal 1, the camera 2, the monitoring terminal 4, the recording apparatus 5, and the like.

[0089] The detection terminal 3 of this configuration sets a part of the storage area of the auxiliary storage device 33 as the area of the setting list 331, the count list 332, the camera image storage 333, and the event log storage 334.

[0090] As Figure 5As shown, setting list 331 is a data list that lists various setting data items, such as notification action items and notification thresholds, in association with notification action IDs. Setting list 331 stores setting data for each type (category) of notification-required action. The notification action ID is unique identification information assigned to each notification action to identify the purchaser's notification-required action, i.e., the type of notification action, to the transaction processing device, i.e., the self-service POS terminal 1.

[0091] Reportable action items are the names of actions that require notification by the purchaser to the self-service POS terminal 1. In this embodiment, five examples of reportable actions are shown: "Missed Scan," "Weight Error," "High-Priced Item Error," "Item Under Cart Error," and "Purchase Quantity: Large." The types of reportable actions will be described in detail later.

[0092] The notification threshold is a value that determines whether or not to notify the notification behavior (behavior that needs to be notified) specified by the corresponding notification behavior ID. In other words, a threshold value of the determination standard is stored for each category of behavior that needs to be notified. The value is typically the number of occurrences. The value may also be the number of purchased items. In this embodiment, if the number of occurrences of the notification behavior is less than the notification threshold, no notification is made, but when it reaches or exceeds the notification threshold, a notification is made. The notification destination is the transaction processing device for the purchaser who performed the behavior, that is, the self-service POS terminal 1. The notification destination includes the monitoring terminal 4, which is a terminal used by the monitor.

[0093] like Figure 6 As shown, the count list 332 is a data list that records the number of occurrences and the notification flag (notification F) in association with the notification behavior ID. The default value of the number of occurrences before the start of the transaction is "0". Each time the notification behavior specified by the corresponding notification behavior ID occurs in a transaction, the number of occurrences is incremented by "1". The default value of the notification flag before the start of the transaction is "0". When the number of occurrences of the notification behavior specified by the corresponding notification behavior ID in a transaction reaches the notification threshold of the notification behavior, the notification flag is "1". The notification behavior with a notification flag of "1" is notified to the above-mentioned notification destination. That is, the notification flag is a 1-bit data used to identify whether the notification behavior specified by the corresponding notification behavior ID is notified.

[0094] The camera image storage unit 333 sequentially stores camera images from the corresponding camera 2. The event log storage unit 334 sequentially stores a log of POS events from the self-service POS terminal 1 and a log of camera events identified based on the camera images. The camera image storage unit 333 and the event log storage unit 334 can delete their stored content as needed, for example, outside of the store's business hours.

[0095] Return Figure 4 the explanation.

[0096] As Figure 4 shown, the processor 31 of the detection terminal 3 has the functions of a detection section 311, a notification section 312, and a second notification section 313. These functions are realized by information processing according to the above-mentioned information processing program and the image analysis program.

[0097] The detection section 311 has the function of detecting the behavior of the purchaser, i.e., the notification behavior, which is the behavior for which notification is required by the transaction processing device, i.e., the self-service POS terminal 1. The detection section 311 distinguishes and detects the kind of the notification behavior on the basis of the first event, i.e., the camera event, and the second event, i.e., the POS event. The first event is discriminated by the camera image of the sensor, i.e., the camera 2, which analyzes the detection purchaser's action. The second event is notified from the self-service POS terminal 1 of the purchaser's action content corresponding to the operation of the purchaser to the POS terminal 1. The distinguishing and detecting the kind of the notification behavior is called category detection. The detection section 311 detects the notification behavior of, for example, the above-mentioned five items of "scanning omission", "weight error", "high-priced goods error", "cart lower goods error", and "purchase quantity: large".

[0098] The notification section 312 is a function of notifying in a transaction when the notification behavior detected by the detection section 311 exceeds the threshold value of the determination criterion for the notification behavior, i.e., the threshold value of the determination criterion for the category of the behavior for which notification is required by the set list 331 (notification threshold value). The notification section 312 notifies the transaction processing device, i.e., the self-service POS terminal 1, of the purchaser who performed the notification behavior. The notification section 312 notifies at the point in time when the threshold value of the determination criterion for the notification behavior is exceeded.

[0099] The second notification section 313 is a function of notifying the monitor's terminal, i.e., the monitoring terminal 4, at the point in time when the notification behavior is detected by the detection section 311. The second notification section 313 notifies the monitoring terminal 4 at the point in time when the notification behavior is detected, regardless of the notification threshold value.

[0100] [Explanation of the action of the detection terminal]

[0101] Figure 7 is a series of timing charts for explaining the action involved in the detection and notification of the notification behavior in the detection terminal 3 together with the actions of the camera 2 and the self-service POS terminal 1 of the same cash register system RS.

[0102] As Figure 7As shown, the camera 2 takes a picture (ACT 11), and outputs the taken camera image to the corresponding detection terminal 3 (ACT 12). For example, the camera 2 has a person detection function, and transmits the camera image in which the purchaser is reflected during the period in which the camera image is taken. Alternatively, the camera 2 can have a moving body detection function instead of the person detection function, and transmit the camera image in which a moving body is reflected during the period in which the camera image is taken. Alternatively, the camera 2 itself can not have such a function, and a person detection sensor or the like can be provided on the self-checkout POS terminal 1, and the camera 2 can take a picture only during the period in which a person is detected by the sensor, and transmit the camera image. Thus, the camera 2 can transmit the camera image in units of transactions of the purchaser. Note that the camera 2 transmits the camera image not only to the detection terminal 3, but also to the recording device 5. The recording device 5 records the camera image in a data file in chronological order. The recording device 5, for example, distinguishes the file name for each camera 2, and records the camera image in the data file.

[0103] The detection terminal 3 that receives the camera image analyzes the camera image using an AI-based image analysis engine, detects the action of the purchaser, and identifies a camera event corresponding to the action (ACT 31). For example, the detection terminal 3 identifies a camera event corresponding to the action when the action of the purchaser taking the product from the shelf 102 is detected from the camera image. Hereinafter, this camera event is referred to as a product taking event CIa. The product taking event CIa sometimes also includes the product code of the product taken by the purchaser from the shelf 102, which is specified by the appearance feature data of the product.

[0104] For example, the detection terminal 3 identifies a camera event corresponding to the action when the action of the purchaser aligning the product on the reading window 109 of the scanner 108 is detected from the camera image. Hereinafter, this camera event is referred to as a scanning action event CIb. The scanning action event CIb sometimes also includes the product code of the product aligned by the purchaser on the reading window 109 of the scanner 108, which is specified by the appearance feature data of the product.

[0105] For example, the detection terminal 3 identifies a camera event corresponding to the action when the action of the purchaser storing the product in the plastic bag or the eco-bag placed on the bagging table 103 is detected from the camera image. Hereinafter, this camera event is referred to as a product bagging event CIc. The product bagging event CIc sometimes also includes the product code of the product stored by the purchaser in the plastic bag or the eco-bag placed on the bagging table 103, which is specified by the appearance feature data of the product.

[0106] When detection terminal 3 detects from camera images that merchandise has been stored in the lower section of a shopping cart placed near basket 102, it identifies a camera event corresponding to the storage of merchandise beneath the cart. This camera event is hereinafter referred to as a "merchandise storage beneath the cart" event CId. The "merchandise storage beneath the cart" event CId may also include a product code, specified by the appearance feature data of the merchandise stored beneath the cart. In this case, if two or more types of merchandise are placed beneath the cart, the "merchandise storage beneath the cart" event CId includes the product code for each merchandise.

[0107] It should be noted that the types of camera events identified by the detection terminal 3 are not limited to the aforementioned four types: the product removal event CIa, the scanning action event CIb, the product bagging event CIc, and the product storage event CId under the cart. The detection terminal 3 can also identify other types of camera events based on camera images.

[0108] When a purchaser performs an operation (ACT 21), the self-service POS terminal 1 outputs a POS event indicating the details of the operation to the corresponding detection terminal 3 (ACT 22). For example, when the self-service POS terminal 1 reads a code symbol of a product positioned against the reading window 109 of the scanner 108, it outputs a POS event indicating the details of the operation. This POS event is hereinafter referred to as a registration event PIa. The registration event PIa includes the product code specified by the code symbol read by the scanner 108.

[0109] For example, if a product is placed on the bagging station 103 and the weight measured by the scale unit 122 changes, the self-service POS terminal 1 outputs a POS event indicating the content of the operation. Hereinafter, this POS event is referred to as a weight event PIb. The weight event PIb includes the changed weight.

[0110] For example, the self-service POS terminal 1 displays the registration screen SCa (see Figure 14 ) is touched, a POS event indicating the details of the operation is output. Hereinafter, this POS event is referred to as a checkout event PIc. After completing the registration of the purchased goods, the purchaser touches the [Checkout] touch button BTb.

[0111] It should be noted that the types of POS events outputted from the self-service POS terminal 1 are not limited to the aforementioned three types of registration event PIa, weight event PIb, and checkout event PIc. The self-service POS terminal 1 may output other types of POS events according to the purchaser's operation.

[0112] The detection terminal 3 detects the purchaser's notification behavior (ACT 32) based on the POS events output from the self-service POS terminal 1 and the camera events identified from the camera image of the camera 2. For example, the detection terminal 3 detects notification behaviors such as "missed scan," "wrong high-priced item," "wrong item under the cart," or "large quantity purchased." Alternatively, the detection terminal 3 detects the purchaser's notification behavior (ACT 32) based on the POS events output from the self-service POS terminal 1. For example, the detection terminal 3 detects notification behaviors such as "weight error."

[0113] In this way, detection terminal 3 detects the purchaser's notification behavior based on POS events and camera events. Specifically, self-service POS terminal 1, which outputs POS events, and camera 2, which outputs camera images used to identify camera events, function as output units. This output unit outputs the information necessary to detect the purchaser's notification behavior to the transaction processing device. It should be noted that it is also possible to detect the purchaser's notification behavior based solely on POS events. In this case, self-service POS terminal 1 functions as an output unit. Furthermore, since it is also possible to detect the purchaser's notification behavior based solely on camera events, in this case, camera 2 also functions as an output unit.

[0114] The detection terminal 3 determines whether the buyer's notification behavior has been detected (ACT 33). If the notification behavior has been detected ("YES" in ACT 33), the detection terminal 3 notifies the monitor of the occurrence of the notification behavior. Specifically, the detection terminal 3 outputs a notification event to notify the monitoring terminal 4 of the occurrence of the notification behavior (ACT 34).

[0115] It should be noted that, when the monitoring terminal 4 receives a notification event from the detection terminal 3, it notifies the display of the notification content. For example, the monitoring terminal 4 displays the notification content on the monitoring screen SCb (refer to Figure 16 ), the number of scans is displayed as [Scanning Omissions: 0]. The number of scans [Scanning Omissions: 0] can also be displayed by the monitoring terminal 4 incrementing the value of the number each time a notification action (notification event) with the notification action item "Scanning Omission" is detected from the detection terminal 3. That is, on the monitoring screen SCb displayed on the display of the monitoring terminal 4, the monitoring terminal 4 can increment the text of the notification action, such as [Scanning Omissions: 0], on the monitoring screen SCb, and display it each time a notification action with the notification action item "Scanning Omission" is received from the detection terminal 3.

[0116] Further, the detection terminal 3 adds "1" to the number of occurrences described in the count list 332 in association with the notification behavior ID of the notification behavior (ACT 35). Then, the detection terminal 3 compares the number of occurrences with the notification threshold value set in the setting list 331 in association with the notification behavior ID of the notification behavior (ACT 36).

[0117] The detection terminal 3 determines whether the number of occurrences is equal to or greater than the notification threshold value (ACT 37). In a case where the number of occurrences is equal to or greater than the notification threshold value (YES in ACT 37), the detection terminal 3 notifies the purchaser that the notification behavior has occurred. Specifically, the detection terminal 3 outputs a notification event for notifying the notification behavior to the self-service POS terminal 1 operated by the purchaser as the operator (ACT 38). The self-service POS terminal 1 to which the notification event is inputted notifies the purchaser of the notification behavior, for example, by displaying information such as text or images indicating the content of the notification behavior on the touch panel 105 (ACT 23). The self-service POS terminal 1 can notify the purchaser of the content of the notification behavior by voice.

[0118] Note that in a case where the notification behavior is not detected in the action of ACT 33 of the detection terminal 3 (NO in ACT 33), the detection terminal 3 skips the actions of ACT 34 to ACT 38. Also, in a case where the number of occurrences is less than the notification threshold value in the action of ACT 37 of the detection terminal 3 (NO in ACT 37), the detection terminal 3 skips the action of ACT 38.

[0119] The detection terminal 3 repeatedly performs the actions shown in ACT 31 to ACT 38 during a period before the self-service POS terminal 1 registers the purchased goods of a transaction and settles the transaction. In this way, the detection terminal 3 detects the notification behavior occurring in a transaction on a category basis based on the POS event outputted from the self-service POS terminal 1 and the camera event recognized from the image captured by the camera 2. Then, when the number of detections of the notification behavior reaches equal to or greater than the notification threshold value of the notification behavior, the detection terminal 3 outputs a notification event for notifying that the notification behavior is detected to the self-service POS terminal 1 operated by the purchaser who performed the notification behavior. On the other hand, with respect to the monitoring terminal 4 as a monitor terminal, a notification event for notifying that the notification behavior is detected is outputted each time the notification behavior is detected.

[0120] [Explanation of the program of the detection terminal]

[0121] Figure 8 is a flowchart showing the main part order of the information processing performed by the processor 31 of the detection terminal 3 which performs the actions as described above. The processor 31 performs the information processing of the steps shown in the flowchart of Figure 8 according to the information processing program stored in the main memory 32 or the auxiliary storage device 33.

[0122] Specifically, in ACT 101, the processor 31 checks whether a camera image has been acquired from the camera 2 corresponding to the detection terminal 3. If no camera image has been acquired, the processor 31 proceeds to ACT 102. In ACT 102, the processor 31 checks whether a POS event has been acquired from the self-service POS terminal 1 corresponding to the detection terminal 3. If no POS event has been acquired, the processor 31 returns to ACT 101. Thus, in ACTs 101 and 102, the processor 31 waits for the acquisition of a camera image or the acquisition of a POS event.

[0123] If the processor 31 acquires a camera image while in the waiting state of ACT 101 and ACT 102, it proceeds to ACT 103. In ACT 103, the processor 31 stores the camera image in the camera image storage unit 333. Then, in ACT 104, the processor 31 analyzes the camera image stored in the camera image storage unit 333 using an AI-based image analysis engine to determine whether a camera event can be identified. If a camera event cannot be identified, the processor 31 returns to the waiting state of ACT 101 and ACT 102.

[0124] When the processor 31 recognizes a camera event in ACT 104 , the process proceeds to ACT 105 . As ACT 105 , the processor 31 stores the camera event in the event log storage unit 334 .

[0125] On the other hand, when a POS event is acquired in the reception waiting state of ACT 101 and ACT 102 , the processor 31 proceeds to ACT 106 . The processor 31 stores the POS event in the event log storage unit 334 as ACT 106 .

[0126] When a camera event or a POS event is stored in the event log storage unit 334 in ACT 105 or ACT 106, the processor 31 proceeds to ACT 107. In ACT 107, the processor 31 detects a notification action for the notification action item set in the setting list 331 based on the camera event and the POS event stored in the event log storage unit 334. The detection operation of each notification action will be described in detail later.

[0127] The processor 31 checks whether the notification action can be detected as ACT 108. If the notification action is not detected, the processor 31 returns to the waiting state of ACT 101 and ACT 102.

[0128] When the reporting behavior of any one of the reporting behavior items is detected in the ACT 108, the processor 31 proceeds to the ACT 109. As the ACT 109, the processor 31 performs the monitor reporting. That is, the processor 31 transmits a reporting event for reporting the detection of the reporting behavior to the monitor terminal 4. The reporting event includes information indicating the reporting behavior item of the detected reporting behavior. In the monitor terminal 4 that receives the reporting event, on the display in the detection terminal 3 of the reporting event transmission source, information for reporting the detection of the reporting behavior of the reporting behavior item included in the reporting event to the monitor is displayed (refer to FIG. 6). Figure 16 ).

[0129] The processor 31 that has finished the monitor reporting proceeds to the ACT 110. As the ACT 110, the processor 31 increases the occurrence count stored in the count list 332 in association with the reporting behavior ID of the detected reporting behavior by the count "1". Then, as the ACT 111, the processor 31 compares the occurrence count with the reporting threshold stored in the setting list 331 in association with the reporting behavior ID. As the ACT 112, the processor 31 determines whether the occurrence count is equal to or greater than the reporting threshold. In the case where the occurrence count is less than the reporting threshold, the processor 31 returns to the waiting acceptance state of the ACT 101 and the ACT 102.

[0130] On the other hand, in the case where the occurrence count is equal to or greater than the reporting threshold, the processor 31 proceeds to the ACT 113. As the ACT 113, the processor 31 sets the reporting flag (reporting F) stored in the count list 332 in association with the reporting behavior ID to "1". In addition, as the ACT 114, the processor 31 performs the purchaser reporting. That is, the processor 31 transmits a reporting event for reporting the detection of the reporting behavior to the corresponding self-service POS terminal 1. In the reporting event, information indicating the reporting behavior item of the reporting behavior ID associated with the reporting flag that becomes "1" is included. In the self-service POS terminal 1 that receives the reporting event, for example, on a part of the screen of the touch panel 105, information for reporting the detection of the reporting behavior of the reporting behavior item included in the reporting event to the purchaser is displayed (refer to FIG. 7). Figure 15 ). The processor 31 that has performed the purchaser reporting returns to the waiting acceptance state of the ACT 101 and the ACT 102.

[0131] Here, the processor 31 realizes the function as the detection section 311 through the processing of the ACT 101 to the ACT 108. The processor 31 realizes the function as the reporting section 312 through the processing of the ACT 110 to the ACT 114. The processor 31 realizes the function as the second reporting section 313 through the processing of the ACT 109.

[0132] [Explanation of the Kind of Reporting Behavior and Detection Action]

[0133] Figure 9 is a schematic view of a notification behavior detection action indicating the notification behavior item "scan omission". The notification behavior item "scan omission" refers to a behavior in which the purchaser does not scan the bar code of the purchased goods taken out from the basket table 102 and packed in the plastic bag or the eco bag at the bagging table 103 by the scanner 108 of the self-service POS terminal 1. For example, a behavior in which the purchaser aligns the purchased goods to the reading window 109 of the scanner 108, but does not read the bar code because the bar code is not aligned correctly and packs the goods, corresponds to the notification behavior of the notification behavior item "scan omission".

[0134] For such a behavior, in the event log storage section 334, the goods taking-out event CIa is stored first, then the scan action event CIb is stored, and then the goods packing event CIc is stored. On the other hand, for the POS event, only the weight event PIb is stored, and the registration event PIa is not stored.

[0135] The detection terminal 3 detects the presence of the notification behavior of the notification behavior item "scan omission" in a case where the goods taking-out event CIa, the scan action event CIb, and the goods packing event CIc are stored in this order, and the weight event PIb is stored, but the registration event PIa is not stored. Note that, in a case where the purchaser packs the goods without aligning the goods to the reading window 109, the behavior also becomes the notification behavior of the notification behavior item "scan omission". That is, the detection terminal 3 detects the presence of the notification behavior of the notification behavior item "scan omission" in a case where the goods taking-out event CIa and the goods packing event CIc are stored, and the weight event PIb is stored, but the registration event PIa and the scan action event CIb are not stored.

[0136] In the present embodiment, the notification threshold for the notification behavior item "scan omission" is set to "2". That is, for the notification behavior of the notification behavior item "scan omission", if the behavior is detected two or more times in one transaction, the self-service POS terminal 1 is notified. If the behavior is detected only once, the behavior is considered to be a false detection, and thus the self-service POS terminal 1 is not notified. That is, by the notification, the self-service operation of the purchaser is prevented from being stopped. On the other hand, the monitoring terminal 4 is also notified of the behavior the first time. Thus, the monitor who is notified confirms whether the scan omission is present or not by going to the side of the purchaser, and in a case where the scan omission is confirmed, instructs the purchaser to scan again, and thus the scan omission can be eliminated.

[0137] Figure 10is a diagram showing a notification behavior detection action indicating the notification behavior item "weight error". The notification behavior item "weight error" refers to a behavior in which a purchaser takes out a purchased product from the basket table 102 and packs it in a plastic bag or an eco bag at the bagging table 103, and the product does not match the product specified by the bar code scanned by the scanner 108 of the self-service POS terminal 1. For example, a behavior in which a purchaser takes out two kinds of purchased products from a shopping basket, scans only one of the products, and packs both of the products is a notification behavior of the notification behavior item "weight error". For example, on a product that is a package sale of a plurality of the same product, a bar code corresponding to the package sale price is printed, and on the other hand, a bar code corresponding to the single product price is printed on each product. A behavior in which such a package sale product is taken out from the basket table 102, a bar code corresponding to the single product price is scanned instead of a bar code corresponding to the package sale price, and the product is packed is also a notification behavior of the notification behavior item "weight error".

[0138] For such a behavior, in the event log storage section 334, the product taking-out event CIa is stored first, then the scanning action event CIb is stored, and then the product packing event CIc is stored. On the other hand, with respect to the POS event, the registration event PIa is stored first, and then the weight event PIb is stored.

[0139] The detection terminal 3 acquires the set weight Wa of the product specified by the product code included in the registration event PIa (ACT 201) in a case where the product taking-out event CIa, the scanning action event CIb, and the product packing event CIc are stored in this order, and the registration event PIa and the weight event PIb are stored in this order. In addition, the detection terminal 3 acquires the measured weight included in the weight event (ACT 202). Then, the detection terminal 3 calculates a change amount Wb of the measured weight included in the previous weight event and the measured weight included in this weight event (ACT 203).

[0140] The detection terminal 3 calculates a difference W of the set weight Wa and the change amount Wb (ACT 204). Then, the detection terminal 3 confirms whether the difference W is equal to or greater than a determination value Wx (ACT 205). The determination value Wx is an arbitrary value that exceeds the allowable range ± a of the set weight Wa. In a case where the weights of products with the same bar code are uniform, the determination value Wx is "0" or a value close to "0". However, even for products with the same bar code such as vegetables, fruits, and home cooking, there are products whose weights are not uniform. For such products, an appropriate determination value Wx (Wx > a) that exceeds the allowable range ± a of the set weight Wa is set.

[0141] In the case where the difference W is smaller than the determination value Wx (NO in ACT 205), the detection terminal 3 does not perform detection of the reporting behavior item "weight error". In contrast, in the case where the difference W is equal to or greater than the determination value Wx (YES in ACT 205), the detection terminal 3 detects the reporting behavior of the reporting behavior item "weight error".

[0142] In the present embodiment, the reporting threshold value for the reporting behavior item "weight error" is set to "2". That is, in the case where the reporting behavior of the reporting behavior item "weight error" is detected twice in one transaction, the self-checkout POS terminal 1 is reported. If it is only once, the malfunction of the scale unit 122 can be considered in view of the position of the bagged goods and the like, and thus the self-checkout POS terminal 1 is not reported. That is, by the reporting, the self-operation of the purchaser is prevented from being stopped. Note that since the first time is also reported to the monitoring terminal 4, the monitor can confirm the legitimacy of the goods in which the weight error has occurred.

[0143] Figure 11 is a schematic view of the reporting behavior detection operation of the reporting behavior item "high-priced goods error". The reporting behavior item "high-priced goods error" refers to the behavior in which, in the case where the purchased goods taken out from the basket table 102 and bagged in the plastic bag or the eco-bag of the bagging table 103 are high-priced goods, the goods specified by the bar code scanned by the scanner 108 of the self-checkout POS terminal 1 are not the high-priced goods. For example, the goods whose unit price is equal to or greater than a predetermined amount are set as the high-priced goods in advance. The behavior in which the purchaser takes out the high-priced goods and other goods from the shopping basket, scans only the other goods to bag, and hides the high-priced goods without bagging, corresponds to the reporting behavior of the reporting behavior item "high-priced goods error". Alternatively, the behavior in which the purchaser directs the high-priced goods to the reading window 109 of the scanner 108, but does not read the bar code to bag because the bar code is not correctly directed, also corresponds to the reporting behavior of the reporting behavior item "high-priced goods error". Note that the latter behavior also corresponds to the reporting behavior of the reporting behavior item "scanning omission".

[0144] In the case of such behavior, in the event log storage section 334, the goods taking-out event CIa is stored with respect to the camera event, and then the scanning action event CIb is stored, and then the goods bagging event CIc is stored. On the other hand, with respect to the POS event, the registration event PIa is stored, and then the weight event is stored.

[0145] The detection terminal 3 confirms whether the product is designated, i.e., whether the product code is included in the product removal event CIa, based on the camera image of the behavior of the purchaser removing the product from the basket table 102 (ACT211). In the case where the product code is included (YES in ACT211), the detection terminal 3 determines whether the product designated by the product code is a product that is set in advance as a high-priced product (ACT212). In the case where the product code is not included in the product removal event CIa (NO in ACT211), or in the case where the product designated by the product code is not a high-priced product (NO in ACT212), the detection terminal 3 does not perform detection of the reporting behavior item "high-priced product error".

[0146] In contrast, in the case where the product designated by the product code is a high-priced product (YES in ACT212), the detection terminal 3 continues detection of the reporting behavior item "high-priced product error". The detection terminal 3 acquires the product code from the registration event PIa (ACT213). The detection terminal 3 discriminates whether the product code coincides with the product code of the high-priced product (ACT214). In the case where the product codes coincide (NO in ACT214), the detection terminal 3 does not perform detection of the reporting behavior item "high-priced product error". In contrast, in the case where the product codes do not coincide (YES in ACT214), the detection terminal 3 detects the reporting behavior of the reporting behavior item "high-priced product error".

[0147] In the present embodiment, the reporting threshold value for the reporting behavior item "high-priced product error" is set to "1". That is, for the reporting behavior of the reporting behavior item "high-priced product error", reporting is made to the self-checkout POS terminal 1 when the behavior is detected once in one transaction. For the irregular behavior of the high-priced product, the store suffers a large loss. Therefore, reporting is made even when the behavior is detected once, to arouse the attention of the purchaser. By doing so, it is possible to suppress irregular behavior.

[0148] Figure 12 is a schematic view showing a reporting behavior detection operation of the reporting behavior item "cart lower product error". The reporting behavior item "cart lower product error" refers to a behavior in which the purchaser ends registration of the purchase of a product without scanning the product received in the lower stage of the shopping cart.

[0149] For such an action, in the event log storage section 334, the cart-under goods storage event CId is stored with respect to the camera event. On the other hand, for the POS event, the checkout event PIc is stored. When the cart-under goods storage event CId is acquired by the detection terminal 3, the goods specified by the cart-under goods storage event CId are recognized (ACT 221). That is, the detection terminal 3 acquires the goods codes included in the cart-under goods storage event CId. At this time, in the case where two or more goods codes are included in the cart-under goods storage event CId, the respective goods codes are acquired.

[0150] After that, when the checkout event PIc is acquired by the detection terminal 3, the purchased goods registered in the transaction file 17 of the corresponding self-checkout POS terminal 1 are searched (ACT 222). Then, the detection terminal 3 confirms whether the goods stored under the cart are registered in the transaction file 17 (ACT 223). That is, the detection terminal 3 confirms whether the purchase data of the goods codes included in the cart-under goods storage event CId are registered in the transaction file 17. In the case where all the goods stored under the cart are registered in the transaction file 17 (NO in ACT 223), the detection terminal 3 does not perform the detection of the reporting action item "cart-under goods error". In the case where even one kind of the goods stored under the cart is not registered in the transaction file 17 (YES in ACT 223), the detection terminal 3 detects the reporting action in which the reporting action item "cart-under goods error" exists.

[0151] Note that, generally, the goods stored in the lower section of the shopping cart are large goods which are not put in the shopping basket. Therefore, in the case where the detection terminal 3 cannot acquire the goods codes from the cart-under goods storage event CId in ACT 221, it searches whether the goods classified as large goods in ACT 222 are registered in the transaction file 17. Then, in the case where the goods classified as large goods are not registered in the transaction file 17 (YES in ACT 223), the detection terminal 3 can also detect the reporting action in which the reporting action item "cart-under goods error" exists.

[0152] In one transaction, the reporting action of the reporting action item "cart-under goods error" is detected only once. Therefore, in the present embodiment, the reporting threshold value with respect to the reporting action item "cart-under goods error" is set to "1".

[0153] Figure 13is a schematic view of a notification behavior detection action indicating the notification behavior item "number of purchases: large". The notification behavior item "number of purchases: large" refers to the behavior of a purchaser purchasing a large number of purchased goods in one transaction. For example, in the case where the purchaser purchases 10 or more kinds, the detection terminal 3 notifies the monitoring terminal 4 and the self-service POS terminal 1 that the notification behavior item "number of purchases: large" has been detected.

[0154] For such behavior, with respect to the camera events, the product removal event CIa, the scanning action event CIb, and the product bagging event CIc are repeatedly stored in the event log storage section 334. On the other hand, with respect to the POS events, the registration event PIa and the weight event are repeatedly stored.

[0155] The detection terminal 3 stores "0" as the default number of purchases (ACT231). Next, when the product removal event CIa, the scanning action event CIb, and the product bagging event CIc of the camera events and the registration event PIa and the weight event of the POS events are stored in the event log storage section 334, the detection terminal 3 adds "1" to the number of purchases (ACT232).

[0156] The detection terminal 3 confirms whether the number of purchases is equal to or greater than the notification threshold value (ACT233). The notification threshold value is the value of the number of purchases that is determined to have detected the notification behavior item "number of purchases: large". For example, assume that the notification threshold value is "10" when the notification behavior item "number of purchases: large" is notified of detection in the case where the purchaser purchases 10 or more kinds. In the case where the number of purchases is less than the threshold value (YES in ACT233), the detection terminal 3 continues the detection action of the notification behavior item "number of purchases: large".

[0157] The detection terminal 3 stores the product removal event CIa, the scanning action event CIb, and the product bagging event CIc of the camera events in the event log storage section 334, and each time the registration event PIa and the weight event of the POS events are stored, adds "1" to the number of purchases (ACT234), and repeatedly performs the process of comparing with the notification threshold value (ACT235). As a result, when the number of purchases becomes equal to or greater than the notification threshold value (YES in ACT235), the detection terminal 3 detects the notification behavior of the notification behavior item "number of purchases: large".

[0158] [Explanation of the notification example]

[0159] Figure 14 is an example of a registration screen Sca displayed on the touch panel 105 of the self-service POS terminal 1. The registration screen Sca is a screen for the registration of purchased goods. The registration screen Sca is generated by the processor 11 of the self-service POS terminal 1.

[0160] AsFigure 14 As shown in FIG. 6, the registration screen SCa includes a touch button BTa of [No barcode item] and a touch button BTb of [Check out]. The touch button BTa of [No barcode item] is used when registering an item such as a fresh food item that has no barcode. According to a touch of this touch button BTa, in the self-POS terminal 1, a touch button, so-called item button, corresponding to an item to which no code symbol is attached is displayed, and the purchaser registers an item by touching an item button corresponding to a purchased item. The touch button BTb of [Check out] is used when the purchaser who has completed the registration of a purchased item declares the check out of a transaction.

[0161] The registration screen SCa is divided into a registration list area ARa, a text area ARb, and a camera image area Arc. The registration list area ARa is an area for displaying a list of item names, purchase quantities, and purchase amounts of purchased items registered by the purchaser. The text area ARb is an area for displaying text such as a guide to the purchaser, a message, and the like. The camera image area Arc is an area for displaying a camera image at a current time point taken by the corresponding camera 2.

[0162] In the present embodiment, when a notification event is received from the detection terminal 3, text indicating the content of the notification behavior is displayed in the text area ARb. Figure 15 is an example of the registration screen SCa of the self-POS terminal 1 that has received a notification event of the notification behavior item "scanning omission". As shown in FIG. 7, in the text area ARb, text indicating the notification behavior of the notification behavior item "scanning omission", for example, "Are there any items that have not been scanned?" is displayed. Figure 15 As shown in FIG. 7, in the text area ARb, text indicating the notification behavior of the notification behavior item "scanning omission", for example, "Are there any items that have not been scanned?" is displayed. Therefore, a purchaser who intentionally performs an improper behavior of scanning omission will notice that the improper behavior is known by the store side, and thus, the improper behavior will be prevented.

[0163] Figure 16 is an example of a monitoring screen SCb displayed on the display of the monitoring terminal 4. As shown in FIG. 8, the monitoring screen SCb is divided into monitoring areas STa to STn according to the cash register systems RS. In the monitoring areas STa to STn, the states of the self-POS terminals 1 of the respective cash register systems RS are displayed. In addition, when a notification event is received from the detection terminal 3 of the corresponding cash register system RS, text indicating the content of the notification behavior is displayed in the monitoring areas STa to STn. Figure 16

[0164] Figure 16 ​This is a monitoring screen SCb when the detection terminal 3 of the cash register system RS identified by cash register number "2" detects a notification action for the notification action item "missed scanning" once, and the detection terminal 3 of the cash register system RS identified by cash register number "7" detects a notification action for the notification action item "missed scanning" twice. The notification action for the notification action item "missed scanning" is not reported to the self-service POS terminal 1 the first time. However, the self-service POS terminal 1 is notified the second time. Therefore, on the monitoring terminal 4, the text of the notification action for the first notification, such as [Missed scanning: 1 item], and the text of the notification action for the second notification, such as [Missed scanning: 2 items], are changed, for example, by changing the text color or font, so that the monitor can distinguish.

[0165] It should be noted that in Figure 16 In the example, the display of the number "[Missed Scanning: 0 items]" can also be configured such that each time a notification action for the notification action item "Missed Scanning" is detected at the detection terminal 3 of the cash register system RS, the value of the number is incremented and displayed. Specifically, on the monitoring screen SCb displayed on the display of the monitoring terminal 4, each time the monitoring terminal 4 receives a notification action for the notification action item "Missed Scanning" from the detection terminal 3, the text of the notification action on the monitoring screen SCb, such as "Missed Scanning: 0 items," is incremented. For example, at the detection terminal 3 of the cash register system RS identified by cash register number "2," when a notification action for the notification action item "Missed Scanning" is detected once, the display will read "Missed Scanning: 1 item." Subsequently, when a notification action for the notification action item "Missed Scanning" is detected again, the count is incremented to "Missed Scanning: 2 items" and displayed.

[0166] [Effects of the First Embodiment]

[0167] As described above, according to the first embodiment, the detection terminal 3 sets a notification threshold for each notification behavior item identified by a notification behavior ID. Then, during a transaction, when the detection behavior for a notification behavior item is greater than or equal to the notification threshold for that notification behavior item, the detection terminal 3 notifies the corresponding self-service POS terminal 1 of the detection of the notification behavior item. Therefore, compared to a scenario where fraudulent behavior is detected each time a notification is made, the frequency of notifications is reduced, thereby reducing the risk of false detection notifications for the purchaser. Furthermore, when a notification behavior is detected that exceeds the notification threshold, the corresponding self-service POS terminal 1 is notified, effectively preventing or suppressing fraudulent activity.

[0168] Further, the detection terminal 3 notifies the monitoring terminal 4 of the notification behavior each time the notification behavior identified by the notification behavior item is detected. Upon receiving the notification, the monitoring terminal 4 displays on the display that the notification behavior identified by the notification behavior item is detected. Thus, the monitor can grasp in real time that the notification behavior has occurred, and thus, has greater effectiveness in preventing or suppressing the irregular behavior. Note that the notification destination can also be the self-service POS terminal 1, or can also be a monitoring device other than the monitoring terminal, and the like.

[0169] [Second Embodiment]

[0170] Next, a second embodiment of the detection terminal 3 functioning as an information processing device will be described using Figure 17 and Figure 18

[0171] Figure 17 A series of timing charts in which the detection and notification of the notification behavior in the detection terminal 3 of the second embodiment are described together with the actions of the camera 2 of the same cash register system RS and the self-service POS terminal 1. To the parts common to the timing chart of the first embodiment shown in FIG. 9, the same symbols are assigned. Figure 7

[0172] As will be understood from a comparison between Figure 7 and Figure 17 , in the second embodiment, in ACT 37, the processing after the number of occurrences is determined to be equal to or greater than the notification threshold is different from the first embodiment. That is, in the second embodiment, the detection terminal 3 sets the notification flag (notification F) to "1" (ACT 41) when the number of occurrences is equal to or greater than the notification threshold (YES in ACT 37). At this time, the detection terminal 3 does not notify the purchaser.

[0173] The detection terminal 3 determines whether there is a notification behavior item for which the notification flag is "1" (ACT 43) when the checkout event as a POS event is received from the self-service POS terminal 1 (YES in ACT 42). When the notification flag of at least one notification behavior item is "1" (YES in ACT 43), the detection terminal 3 notifies the purchaser (ACT 44).

[0174] For example, in a case where the notification flag associated with the notification behavior ID of the notification behavior item "scan omission" is "1", the purchaser is notified of the detection of the notification behavior of the notification behavior item "scan omission". Further, in a case where the notification flag associated with the notification behavior ID of the notification behavior item "weight error" is also "1", the purchaser is notified of the detection of the notification behavior of the notification behavior item "weight error" together.

[0175] Figure 18 ​​is a flowchart of the essential steps of the information processing performed by the processor 31 of the detection terminal 3 to indicate the action as described above. To the Figure 8 The portions common to the flowcharts of the first embodiment shown in Figs. 10 and 11 are given the same symbols.

[0176] As Figure 8 As Figure 18 In the second embodiment, the processing after the acquisition of the POS event in the waiting acceptance state of ACT 101 and ACT 102 is different from the first embodiment. That is, the processor 31 confirms whether the POS event is a check-out event as ACT 121. That is, the processor 31 confirms the check-out event by detecting that the check-out button BTb of the registration screen Sca in Figure 14 is pressed. In the case where it is not a check-out event, the processor 31 proceeds to ACT 106. Then, the processor 31 performs the processing of ACT 106 to ACT 113 similarly to the processing explained in the first embodiment. However, the processing of ACT 114 is not performed after ACT 113, that is, the notification to the purchaser is not performed.

[0177] When the processor 31 has acquired the check-out event in ACT 121, it proceeds to ACT 122. As ACT 122, the processor 31 checks the notification flags (notification F) of the count list 332. As a result, in the case where the notification flags of all the notification behavior items are "0", the processor 31 ends the information processing of the steps shown in the flowchart of Figure 18

[0178] On the other hand, in the case where at least one of the notification flags is "1", the processor 31 proceeds to ACT 123. As ACT 123, the processor 31 performs the notification to the purchaser. That is, the processor 31 transmits a notification event for notifying the notification behavior of the notification behavior item in which the notification flag is "1" to the corresponding self-service POS terminal 1. At this time, in the case where two or more of the notification flags are "1", the notification event for notifying the notification behavior of each of the notification behavior items in which the notification flags are "1" is transmitted to the corresponding self-service POS terminal 1. Then, the processor 31 ends the information processing of the steps shown in the flowchart of Figure 18

[0179] ​​Thus, in the second embodiment, the detection terminal 3 sets a notification threshold for each notification behavior identified by the notification behavior item. When a notification behavior is detected to be above the notification threshold, the detection terminal 3 notifies the monitoring terminal 4 of the detection of the notification behavior item when a checkout event is received from the corresponding self-service POS terminal 1. Therefore, compared to the first embodiment in which notification is made only when a notification behavior is detected to be above the notification threshold, the frequency of notification is only once, thus further reducing the risk of false detection notifications to the purchaser. In addition, since notification is made after the product registration is completed, it does not hinder the product registration operation of customers who are not accustomed to operating the self-service POS terminal 1. Therefore, the time required for product registration can be shortened. On the other hand, the detection terminal 3 notifies the monitoring terminal 4 each time it detects a notification behavior identified by the notification behavior item. Therefore, it is effective in preventing or suppressing improper behavior.

[0180] [Other embodiments]

[0181] In the above embodiment, the self-service POS terminal 1 sets a portion of the registration screen SCa as the text area ARb and the camera image area ARc. Figure 19 As shown, the self-service POS terminal 1 is provided with a purchaser display 8. The purchaser display 8 is arranged on the left side of the touch panel 105 that displays the registration screen SC, etc. It should be noted that the position of the purchaser display 8 is not limited to Figure 19 The location shown. It can be placed anywhere as long as the purchaser can confirm the location.

[0182] like Figure 20 As shown, the self-service POS terminal 1 divides the screen SCd of the purchaser display 8 into a text area ARb and a camera image area ARc. Figure 20 As shown in the screen SCe, when a notification event is received from the detection terminal 3, the text area ARb displays the notification action item specified by the notification event, for example, the text "Are there any unscanned items?" indicating the notification action of "missed scanning." Thus, by providing the purchaser display 8 on the self-service POS terminal 1, the purchaser can more accurately identify the notification. Furthermore, the registration screen SCa of the touch panel 105 can occupy a larger registration list area ARa.

[0183] In the above embodiment, if Figure 1 As shown in FIG. 1 , a detection terminal 3 using an information processing device is provided for each cash register system RS. Figure 21As shown, the detection terminal 3 is excluded from the cash register system RS, and instead the detection server 7 is connected to the communication network 6, and a self-checkout system SYS is configured. The detection server 7 is a computer configured to realize the functions of the detection terminal 3 in each cash register system RS. As such a configuration, the same effects as the first or second embodiment can be achieved.

[0184] In the above-described embodiments, the detection terminal 3 is exemplified as a form of the information processing apparatus capable of reducing the risk of false detection notification to the purchaser and effectively preventing or suppressing the effective fraudulent behavior. The information processing apparatus is not limited to the detection terminal 3. For example, other apparatuses such as the self-checkout POS terminal 1, the camera 2, the monitoring terminal 4, or the recording apparatus 5 can be provided with the functions of the detection unit 311, the notification unit 312, and the second notification unit 313 described in the first embodiment, and can function as the information processing apparatus, respectively.

[0185] Alternatively, instead of the detection terminal 3 having all the functions, the self-checkout POS terminal 1, the camera 2, the monitoring terminal 4, or the recording apparatus 5, or the like can be provided with a part of the functions to realize the information processing system. For example, in the first embodiment, the detection terminal 3 has the function as the detection unit 311. On the other hand, the self-checkout POS terminal 1 has the functions as the notification unit 312 and the second notification unit 313. In addition, the self-checkout POS terminal 1 has the setting list 331 and the count list 332. According to this configuration, when the detection terminal 3 detects the behavior of the purchaser that requires notification to the self-checkout POS terminal 1, that is, the notification behavior, the information is output to the self-checkout POS terminal 1. The self-checkout POS terminal 1 notifies the monitoring terminal 4 at the time point when the information of the notification behavior is acquired. In addition, the self-checkout POS terminal 1 adds the number of occurrences of the count list 332 when the information of the notification behavior is acquired. Then, in one transaction, in a case where the number of occurrences of the detected notification behavior exceeds the notification threshold value set in the setting list 331, the notification is performed by, for example, popping up a display of the detected notification behavior on the touch panel 105. Even with such a configuration, the same effect of reducing the risk of false detection notification to the purchaser while effectively preventing or suppressing the effective fraudulent behavior can be achieved.

[0186] Of course, the types of the reporting behavior detected by the detection section 311 are not limited to the five items of the reporting behavior "scan omission", "weight error", "high-priced goods error", "cart under goods error", and "purchase number: large". For example, when a behavior of bagging a product without a bar code attached without performing a selection operation in the touch panel 105 is detected, a reporting behavior of the reporting behavior item "no bar code product error" can also be detected. For the reporting behavior item "no bar code product error", the reporting threshold value can be set to "2" to reduce the reporting frequency. For example, when a behavior of touching the [check out] touch button with a product remaining on the basket table 102 is detected, a reporting behavior of the reporting behavior item "registration omission product error" can also be detected. Since the case of touching the [check out] touch button is only once in one transaction, the reporting threshold value of the reporting behavior item "registration omission product error" is necessarily "1".

[0187] Further, although several embodiments of the present application have been described, these embodiments are presented by way of example only, and are not intended to limit the scope of the application. These novel embodiments can be carried out in various other forms, and various omissions, substitutions and changes can be made without departing from the scope of the application. These embodiments and their modifications are included within the scope of the application, and are included within the scope of the application and equivalents thereof as recited in the claims.

Claims

1. An information processing apparatus, characterized by, comprising: a detection section that detects a behavior of a purchaser that requires notification to a transaction processing device; and a notification section that, in a transaction, makes a notification when a behavior detected by the detection section exceeds a threshold value of a criterion for the behavior.

2. The information processing device according to claim 1, wherein the notification section makes a notification to a transaction processing device of a purchaser who performed the behavior that requires the notification.

3. The information processing device according to claim 2, wherein the notification section makes a notification at a point in time at which the threshold value of the criterion for the behavior is exceeded.

4. The information processing device according to claim 2, wherein the notification section makes a notification at a point in time at which a checkout instruction of the transaction processing device by the purchaser is accepted.

5. The information processing apparatus according to any one of claims 1 to 4, wherein further comprising: a second notification section that makes a notification to a terminal for a monitor at a point in time at which the behavior that requires the notification is detected by the detection section.

6. The information processing device according to claim 1, wherein the detection section detects a behavior that requires the notification by category based on a first event that is recognized based on an output of a sensor that detects an action of the purchaser and a second event that indicates a content of an action of the purchaser that is notified by the transaction processing device.

7. The information processing apparatus according to claim 6, wherein further comprising: a setting list that stores a threshold value of a criterion corresponding to each category of a behavior that requires the notification, wherein the notification section makes a notification when the threshold value of the criterion of the setting list for a category detected by the detection section is exceeded.

8. The information processing device according to claim 6, wherein the first event is recognized by analyzing a camera image of a camera that is the sensor by an image analysis engine, and the second event is a content of an action of a purchaser that is notified by the transaction processing device corresponding to an operation of the transaction processing device by the purchaser.

9. A storage medium that stores a program that causes a computer of an information processing device to have functions as: a detection section that detects a behavior of a purchaser that requires notification to a transaction processing device; and a notification section that, in a transaction, makes a notification when a behavior detected by the detection section exceeds a threshold value of a criterion for the behavior.

10. An information processing system, characterized by comprising: comprising: an output section that outputs information required to detect a behavior of a purchaser that requires notification to a transaction processing device; a detection section that detects a behavior of a purchaser that requires notification based on information output by the output section; and a notification section that, in a transaction, makes a notification when a behavior detected by the detection section exceeds a threshold value of a criterion for the behavior. ​

Images