Information processing device, information processing system, and computer-readable storage medium

Abstract

The present invention makes it possible to inexpensively and easily issue notification regarding missing payments. This information processing device comprises a detection unit, a tracking unit, an association unit, and a notification-issuing unit. The detection unit detects whether or not each of a plurality of transaction processing devices in which a customer performs an operation related to a settlement are in a state in which the operation related to the settlement has been completed, on the basis of image information making it possible to identify the state of each of the plurality of transaction processing devices. The tracking unit tracks movements made by the customer. The customer and the transaction processing device with which the customer performs an operation related to settlement are associated by the association unit associates on the basis of the position of the tracked customer. The notification unit issues notification of a missed payment when a transaction processing device associated with a customer who has moved to a predetermined prescribed position is not in a state in which the operation related to the settlement has been completed.

Description

Information processing device, information processing system, and computer-readable storage medium

[0001] An embodiment of the present invention relates to an information processing device, an information processing system, and a computer-readable storage medium storing a program for causing a computer to function as an information processing device.

[0002] In the past, semi-self-checkout systems were known in which a store clerk would register purchased items at a registration terminal, and the customer would then perform the payment process, i.e., the settlement process, at an accounting terminal. In recent years, full-self-checkout systems have also become popular, in which the customer not only performs the payment process but also the registration process.

[0003] In such self-checkout systems, there is a possibility that customers may register their products and then leave the store without completing payment, resulting in fraudulent payment.

[0004] Japanese Patent Application Publication No. 2018-136679

[0005] The problem to be solved by the embodiments of the present invention is to provide an information processing device and an information processing system that enable easy notification of missed payments.

[0006] FIG. 1 is a schematic diagram illustrating a self-checkout system to which the information processing system according to the first embodiment is applied, including a monitoring device to which the information processing device according to the first embodiment is applied. FIG. 2 is a schematic diagram illustrating an example of a checkout area of ​​a store. FIG. 3 is a perspective view illustrating an example of the external configuration of a self-checkout POS terminal as a transaction processing device included in the self-checkout system. FIG. 4 is a block diagram illustrating an example of the main circuit configuration of the self-checkout POS terminal. FIG. 5 is a perspective view illustrating an example of the external configuration of a self-checkout terminal as a transaction processing device included in the self-checkout system. FIG. 6 is a diagram illustrating status information of a transaction processing device represented by the color pattern and light emission pattern of a patrol lamp provided in the transaction processing device. FIG. 7 is a block diagram illustrating an example of the main circuit configuration of the monitoring device. FIG. 8 is a schematic diagram illustrating the main data structure stored in the tracking table of the monitoring device. FIG. 9 is a schematic diagram illustrating the main data structure stored in the linking table of the monitoring device. FIG. 10 is a sequence diagram illustrating fraud detection operations in a self-checkout system to which the information processing system according to the first embodiment is applied. FIG. 11 is a flowchart illustrating an example of the main steps of tracking processing executed by a processor in the monitoring device. FIG. 12 is a flowchart showing an example of the main steps of the payment omission detection process executed by the processor of the monitoring device. FIG. 13 is a diagram showing an example of a monitoring screen displayed on the touch panel of the attendant terminal. FIG. 14 is a diagram showing an example of a monitoring screen when an alert notification is received. FIG. 15 is a flowchart showing an example of the main steps of the payment omission detection process executed by the processor of the monitoring device to which the information processing device according to the second embodiment is applied. FIG. 16 is a diagram showing an example of an idle screen displayed on the touch panel of a self-service POS terminal as an example of a transaction processing device according to the third embodiment. FIG. 17 is a diagram showing an example of a transaction start screen displayed on the touch panel of a self-service POS terminal. FIG. 18 is a diagram showing an example of a registration screen displayed on the touch panel of a self-service POS terminal. FIG. 19 is a diagram showing an example of a payment selection screen displayed on the touch panel of a self-service POS terminal. FIG. 20 is a diagram showing an example of a transaction completion screen displayed on the touch panel of a self-service POS terminal.Fig. 21 is a schematic configuration diagram showing a self-checkout system to which an information processing system according to the fourth embodiment is applied, including a monitoring device to which an information processing device according to the fourth embodiment is applied. Fig. 22 is a schematic diagram showing an example of a checkout area of ​​a store in the fourth embodiment. Embodiment

[0007] In one embodiment, the information processing device includes a detection unit, a tracking unit, a linking unit, and a notification unit. The detection unit detects whether the settlement-related operations have been completed for each of the transaction processing devices based on image information that identifies the status of each of the transaction processing devices on which the customer performs settlement-related operations. The tracking unit tracks the movement of the customer. The linking unit links the customer to the transaction processing device on which the customer performs settlement-related operations based on the location of the tracked customer. The notification unit notifies the customer of an overdue payment if the status of the transaction processing device linked to the customer who has moved to a predetermined location is not in a state where the settlement-related operations have been completed.

[0008] Hereinafter, embodiments of an information processing device and an information processing system will be described with reference to the drawings.

[0009] [First Embodiment] Fig. 1 is a schematic diagram showing a self-checkout system to which an information processing system according to a first embodiment is applied. The self-checkout system includes a plurality of self-service transaction processing devices 1 and a store server 2 connected via a communication network NW such as a local area network (LAN). The information processing system according to the first embodiment serves as a notification system and includes an attendant terminal 3, a checkout area camera 4, and a monitoring device 5 to which the information processing device according to the first embodiment is applied. The monitoring device 5 is connected to the attendant terminal 3 and the checkout area camera 4. As shown by the dashed lines in Fig. 1, the attendant terminal 3 may also be connected to a plurality of transaction processing devices 1 and the store server 2 via the communication network NW.

[0010] A plurality of transaction processing devices 1, attendant terminals 3, and checkout area cameras 4 are installed in a checkout area CA of a store, as shown by the dashed line in Fig. 1. Fig. 2 is a schematic diagram showing an example of this checkout area CA. In the example of Fig. 2, eight transaction processing devices 1 and one attendant terminal 3 are installed in the checkout area CA. Although not limited to this, the following description of the configuration and operation will be based on the number and installation layout of transaction processing devices 1 in this checkout area CA.

[0011] Checkout area CA has an entrance ENT and an exit EXT for shoppers. A terminal number R1-R8 is assigned to entrance ENT to uniquely identify each transaction processing device 1. These terminal numbers R1-R8 can be assigned arbitrarily by the administrator of store server 2. A store clerk SP is stationed at attendant terminal 3.

[0012] The self-service transaction processing device 1 is a terminal at which a customer SC performs operations related to checkout. This self-service transaction processing device 1 includes a self-service POS (Point of Sales) terminal with product registration and payment functions, where the customer SC (the shopper) registers products and performs checkout based on the registered transaction data, and a self-service checkout terminal where the customer SC performs checkout based on product transaction data separately registered by a store clerk or the customer SC. The checkout terminal can also be referred to as a checkout device, payment terminal, settlement device, or checkout terminal. The POS terminal can also be referred to as a product registration checkout terminal, product registration checkout device, product registration payment terminal, product registration payment device, product registration checkout device, or product registration checkout terminal.

[0013] FIG. 3 is a perspective view showing an example of the external configuration of a self-checkout POS terminal 100 as a transaction processing device 1 included in a self-checkout system, and FIG. 4 is a block diagram showing an example of the main circuit configuration of the self-checkout POS terminal 100.

[0014] As shown in FIG. 3 , the self-service POS terminal 100 includes a main body 101 installed on the floor and a scale unit 102 installed beside the main body 101. A display pole 103 and a touch panel 104 are attached to the top of the main body 101. A basket stand 105 is provided in the center of the side of the main body 101 opposite the side on which the scale unit 102 is installed. The basket stand 105 is used by customers SC, who are shoppers coming from the sales floor, to place their shopping baskets containing purchased items. The customers SC work while standing in front of the main body 101 in FIG. 3 so that they can see the screen of the touch panel 104. Therefore, from the customer SC's perspective, the basket stand 105 is on the right side of the main body 101, and the scale unit 102 is on the left side. In the following description, the side on which the customer SC stands is referred to as the front of the main body 101, the side on which the scale unit 102 is installed is referred to as the left side of the main body 101, and the side on which the basket stand 105 is installed is referred to as the right side of the main body 101.

[0015] The scale unit 102 has a structure in which a scale pan 107 is provided on top of a housing 106, and a bag holder 108 is attached to the top of the scale pan 107. The top surface of the scale pan 107 serves as a placement surface 109. The bag holder 108 has a pair of holding arms 110, which hold a plastic bag or a shopping bag brought by the customer SC, known as a "my bag." The scale unit 102 measures the weight of an item placed in the plastic bag or my bag held by the holding arms 110 and placed on the placement surface 109. The placement surface 109 is an example of a product placement area where purchased items to be checked out are placed. The scale unit 102 is an example of a weighing device that measures the weight of an object placed in the product placement area.

[0016] The display pole 103 is provided at its tip with a patrol lamp 111 that displays status information indicating the state of the self-service POS terminal 100. The patrol lamp 111 includes a first light-emitting unit 112 that selectively emits light, for example, blue or red, and a second light-emitting unit 113 that emits light yellow. The light has two types of light emission patterns: continuous illumination, which is lighting, and flashing, which is lighting and turning off for a set period of time. The patrol lamp 111 can display status information by combining the color pattern and light emission pattern of the first light-emitting unit 112 with the light emission pattern of the second light-emitting unit 113.

[0017] The touch panel 104 is composed of a display for displaying various screens related to transactions, such as registration and accounting, to the user operating the self-checkout POS terminal 100, and a touch sensor for detecting touch inputs made by the user onto the screen. In the case of the self-checkout POS terminal 100, the user is usually a customer SC.

[0018] A reading window 115 for a scanner 114 (see FIG. 4), a card insertion slot 117 for a card reader 116 (see FIG. 4), and an issuing slot 119 for receipts printed by a printer 118 (see FIG. 4) are formed on the front of the main body 101. Furthermore, a coin insertion slot 121, a coin dispensing slot 122, a bill insertion slot 123, and a bill dispensing slot 124 of an automatic change dispenser 120 (see FIG. 4) are also formed on the front of the main body 101.

[0019] A communication cable 125 extends from the right side of the main body 101 to the outside, and a reader / writer 126 for electronic money media is connected to the tip of this communication cable 125. The reader / writer 126 is placed on a stand 127 provided on the upper right side of the main body 101.

[0020] As shown in Figure 4, the self-checkout POS terminal 100 includes the touch panel 104, patrol lamp 111, scanner 114, card reader 116, printer 118, automatic change dispenser 120, and reader / writer 126 described above, as well as a processor 128, main memory 129, auxiliary storage device 130, a communication interface 131, and a system transmission path 132. The system transmission path 132 includes an address bus, a data bus, control signal lines, and the like. The system transmission path 132 connects the processor 128 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 128, main memory 129, and auxiliary storage device 130 are connected via the system transmission path 132 to form the computer of the self-checkout POS terminal 100.

[0021] The processor 128 corresponds to the central part of the computer. The processor 128 controls each part to realize the various functions of the self-checkout POS terminal 100 in accordance with an operating system or control program. The processor 128 is, for example, a central processing unit (CPU). The processor 128 may be, for example, a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 128 may be a combination of these.

[0022] The main memory 129 corresponds to the main storage portion of the computer. The main memory 129 includes a nonvolatile memory area and a volatile memory area. The main memory 129 stores an operating system or control program in the nonvolatile memory area. The main memory 129 may store data required for the processor 128 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 129 uses the volatile memory area as a work area where data is rewritten by the processor 128 as appropriate. The nonvolatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory). For example, the main memory 129 stores a transaction file 133.

[0023] The transaction file 133 is a data file for storing transaction data related to one commercial transaction processed by the self-checkout POS terminal 100. The transaction file 133 stores data such as the transaction number, purchased item data, total number of items, total amount, discount amount, and settlement amount. The transaction number is a series of numbers that is issued each time a commercial transaction is processed by the self-checkout POS terminal 100. The purchased item data is record data created for each item sold in a commercial transaction identified by the transaction number. Here, the purchased item data consists of items such as the product code, product name, price, number of items, and amount. The number of items is the number of items purchased for the product identified by the product code. The amount is the amount for the number of items purchased. The transaction file 133 can store multiple purchased item data. The total number of items is the sum of the points for each purchased item data. The total amount is the sum of the amounts for each purchased item data. The discount amount is the amount discounted from the total amount. The settlement amount is the total amount minus the discount amount.

[0024] The auxiliary storage device 130 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 130 may be an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), a HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device 130 stores data used by the processor 128 when performing various processes, data created by the processes of the processor 128, etc. The auxiliary storage device 130 may also store the above-mentioned control program.

[0025] The communication interface 131 performs data communication in accordance with a preset communication protocol with an external device connected via the communication network NW. The external device is, for example, the store server 2, and may also include the attendant terminal 3.

[0026] The scanner 114 reads a code symbol from a product held over the reading window 115. Each product sold in a store is affixed with a code symbol that encodes a product ID or the like to identify the product. The code symbol is, for example, a barcode. The code symbol may also be, for example, a two-dimensional data code. The scanner 114 may be of a type that reads a code symbol by scanning with laser light, or of a type that reads a code symbol from an image captured by an imaging device.

[0027] The card reader 116 reads card data recorded on a card medium such as a credit card, a point card, etc. The card reader 116 pulls the card medium inserted into the card insertion slot 117 into the main body 101, reads the card data, and then ejects the card from the card insertion slot 117.

[0028] The printer 118 prints receipt data and other information indicating the details of the transaction onto receipt paper. The receipt paper on which the receipt data has been printed is ejected from the issuing port 119, cut by a cutter (not shown), and issued as a receipt or invoice.

[0029] The automatic change dispenser 120 includes a coin unit 134 and a bill unit 135. The coin unit 134 sorts coins inserted into the coin insertion slot 121 one by one, identifies the denomination, and stores them in a safe by denomination. The coin unit 134, for example, based on change data, removes coins of the corresponding denomination from the safe and dispenses them to the coin dispensing outlet 122. The bill unit 135 sorts banknotes inserted into the bill insertion slot 123 one by one, identifies the denomination, and stores them in the safe by denomination. The bill unit 135, for example, based on change data, removes banknotes of the corresponding denomination from the safe and dispenses them to the bill dispensing outlet 124.

[0030] The reader / writer 126 reads and rewrites electronic money recorded on an electronic money medium. The electronic money medium is, for example, a contactless IC card. The electronic money medium may also be an electronic device such as a smartphone or a tablet terminal.

[0031] The processor 128 implements processing units related to well-known product registration and payment. Each unit implemented in the processor 128 can also be referred to as a function. Each unit implemented in the processor 128 can also be referred to as being implemented in a control unit including the processor 128 and the main memory 129.

[0032] In this way, the self-service POS terminal 100 has a settlement function that allows both cash and cashless payments. Cashless payments include credit card payments, electronic money payments, points payments, code payments, etc. Customers SC place their purchases in a shopping basket or shopping cart in a sales area where products are displayed, and then move to the checkout area CA. Once in the checkout area CA, the customer SC operates the self-service POS terminal 100 to register the purchased products and perform accounting for the transaction data for the registered purchased products. In this way, the customer SC performs the payment for the transaction themselves, from product registration to payment.

[0033] 5 is a perspective view showing an example of the external configuration of a self-checkout terminal 150 serving as a transaction processing device 1 included in a self-checkout system. Components similar to those of the self-checkout POS terminal 100 are given the same reference numerals and will not be described again.

[0034] The self-checkout terminal 150 comprises a first housing 151 and a second housing 152 .

[0035] On the front of the first housing 151, there are arranged a coin inlet 121, a coin dispensing outlet 122, a bill inlet 123, and a bill dispensing outlet 124 of the automatic change dispenser 120, and a scanner reading window 115 of the scanner 114. Note that the scanner 114 is not for reading code symbols from merchandise like the self-service POS terminal 100, but is used to read code symbols for code payments.

[0036] The second housing 152 is placed on the top surface 153 of the first housing 151 and includes a printer 118. The printer 118 has a receipt issuing port 119 provided on the front surface 154 of the second housing 152.

[0037] A card reader 116 having a card insertion slot 117 is disposed on the top surface 153 of the first housing 151, to the left of the second housing 152. A touch panel 104 is attached to the top of the second housing 152.

[0038] The self-checkout terminal 150 also has a cylindrical display pole 103 erected on the rear side of the top surface 153 of the first housing 151. The display pole 103 has a patrol lamp 111 at its upper tip, which includes a first light-emitting element 112 and a second light-emitting element 113. The display pole 103 also has a display 155 below the patrol lamp 111, which displays images and text. The display 155 is composed of, for example, an LCD panel. The display 155 displays, for example, the details of any errors that have occurred in the self-checkout terminal 150.

[0039] A basket stand 105 for placing a shopping basket or the like is provided on the right side of the self-checkout terminal 150 as viewed from the front.

[0040] The attendant terminal 3 is a terminal operated by an attendant store clerk SP who monitors the checkout area CA. The attendant terminal 3 may be a personal computer connected to the monitoring device 5 by wire or wirelessly, or a tablet terminal that can be connected to the monitoring device 5 by wireless and that can be carried by the store clerk.

[0041] The attendant terminal 3 may also be connected to the communication network NW and have the function of monitoring multiple transaction processing devices 1. Furthermore, the attendant terminal 3 may have a configuration similar to that of the self-service POS terminal 100. An attendant terminal 3 with a monitoring function stores the status of each self-service POS terminal 100 in the main memory 129. For example, the attendant terminal 3 may store a list of purchased items registered in each self-service POS terminal 100, error occurrence status, and the like in the main memory 129. The attendant terminal 3 displays the status of each self-service POS terminal 100 stored in the main memory 129 on the display of the touch panel 104. In addition to cash and cashless payment functions, the attendant terminal 3 may also have functions for handling payments that require manual intervention, such as payments using non-machine-readable vouchers and gift certificates, returns, and refunds.

[0042] The store server 2 includes a product master database and a sales data database, and provides various services related to the self-checkout system to each self-checkout POS terminal 100 connected via the communication network NW. The product master database stores product data, such as product code, product name, unit price, and category name, for each product sold in the store, with each product treated as a separate record. The product code is a unique code assigned to each product to identify it individually. The sales data database stores sales data generated by each transaction, transmitted from each transaction processing device 1 upon completion of payment, i.e., upon completion of the transaction. The sales data database stores, for each transaction, transaction data indicating the transaction details of the products to be paid and payment data related to the payment of the transaction. The transaction data includes the product name (product code) of the product to be paid, its price, the total price of each product, etc. The payment data includes the customer's deposit amount and change amount, etc., if payment is made in cash, and information on the payment method, if payment is made cashlessly. The sales data may also include elements such as a membership code, a company code (company name) indicating the company that operates the store, a store code (store name), the store's telephone number, address, transaction date and time, transaction number (receipt number), terminal number, and the name of the person in charge. Note that the elements included in the sales data are not limited to these examples and may include other elements, or elements such as the telephone number and address may be removed.

[0043] In transaction processing device 1, for example, if it is self-checkout terminal 100, scanner 114 reads a code symbol printed or affixed to a product to obtain a product code, and then reads product data for the product identified by the product code from a product data file stored in transaction processing device 1. Self-checkout terminal 100 then registers the read product data in transaction data, thereby updating and storing sales data including transaction data and payment data. The product data file may be a copy of a product master database stored in store server 2. Furthermore, if transaction processing device 1 is self-checkout terminal 150, it receives and stores transaction data registered by a registration terminal (not shown). Alternatively, the scanner of self-checkout terminal 150 reads a paper medium on which a code indicating the transaction data registered by a registration terminal (not shown) is printed. Self-checkout terminal 150 then registers the transaction data and proceeds to the subsequent transaction process. In this case, the following may be used instead of a paper medium on which a code indicating the transaction data registered by a registration terminal (not shown). That is, a code indicating the transaction data is displayed on the screen of the consumer's mobile device, and the code is read by the scanner of self-checkout terminal 150 to register the transaction data and proceed to the subsequent checkout process. Once the checkout process is complete, that is, once one transaction is finished, transaction processing device 1 prints the sales data on paper to issue a receipt for that transaction, and also transmits the sales data for that transaction to store server 2. The sales data database of store server 2 stores the sales data transmitted from each transaction processing device 1 in this way.

[0044] As shown in FIG. 2 , the checkout area camera 4 is installed on the ceiling of the checkout area CA, captures images of the entire checkout area CA, and transmits the captured camera image to the monitoring device 5. The checkout area camera 4 may be, for example, a 360-degree camera or a fisheye camera. The checkout area camera 4 is installed so that its imaging range includes all of the multiple transaction processing devices 1 installed in the checkout area CA, specifically, the patrol lights 111 of all of the self-service POS terminals 100. Note that if a single checkout area camera 4 cannot capture images of all of the patrol lights 111, multiple checkout area cameras 4 may be installed. The checkout area camera 4 is an example of an acquisition device that acquires image information that can identify the status of each of the multiple transaction processing devices 1. The checkout area camera 4 may be an existing surveillance camera installed in the checkout area CA.

[0045] Monitoring device 5 detects the status of each transaction processing device 1 based on images of patrol lights 111 of each transaction processing device 1 included in the camera images captured by checkout area camera 4. FIG. 6 is a diagram illustrating the status information of transaction processing devices 1, i.e., self-checkout terminal 100 and self-checkout terminal 150, represented by the color and light patterns of patrol lights 111. In FIG. 6, solid hatching indicates a lit state, and dashed hatching indicates a flashing state. Patrol lights 111 indicate that they are in operation by flashing first light-emitting element 112 in blue, and indicate that they are idle by lighting first light-emitting element 112 in blue. Thus, monitoring device 5 can determine the status of each self-checkout POS terminal by recognizing the color and light pattern of patrol lights 111 of each self-checkout POS terminal 100 from the camera images captured by checkout area camera 4. Thus, patrol lights 111 can indicate whether transaction processing device 1 is idle or in operation by at least one of the light color and light state. Here, the "in operation" state means an unpaid state in which the customer SC has not yet made a payment. The "idle" state means a state in which the previous customer SC has made a payment and is waiting for use by the next customer SC, i.e., the previous customer SC has already paid. Therefore, patrol lamp 111 is an example of a display component that indicates whether a customer SC who is using or has used transaction processing device 1 is in an unpaid state or a paid state. While this embodiment uses an example of patrol lamp 111 having two light-emitting elements, first light-emitting element 112 and second light-emitting element 113, it may also be possible to use a single light-emitting element to indicate whether the customer SC is in an unpaid state or a paid state.

[0046] FIG. 7 is a block diagram showing an example of the main circuit configuration of the monitoring device 5. As shown in FIG. 7, the monitoring device 5 includes a processor 51, a main memory 52, an auxiliary storage device 53, a communication interface 54, an input / output device 55, and a system transmission path 56. The system transmission path 56 includes an address bus, a data bus, a control signal line, etc. The system transmission path 56 connects the processor 51 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 51, the main memory 52, and the auxiliary storage device 53 are connected via the system transmission path 56 to form a computer of the monitoring device 5. The computer is an example of an information processing device according to the first embodiment.

[0047] The processor 51 corresponds to the central part of the computer. The processor 51 controls each part to realize various functions of the monitoring device 5 in accordance with an operating system or an application program. The processor 51 is, for example, a CPU. The processor 51 may be a multi-core / multi-threaded processor, and can execute multiple processes in parallel. The processor 51 may be, for example, an MPU, SoC, DSP, GPU, ASIC, PLD, or FPGA. Alternatively, the processor 51 may be a combination of two or more of these.

[0048] The main memory 52 corresponds to the main storage portion of the computer. The main memory 52 includes a nonvolatile memory area and a volatile memory area. The main memory 52 stores an operating system or an application program in the nonvolatile memory area. The application program includes, for example, an information processing program for causing the computer to function as the information processing device according to the first embodiment. The nonvolatile memory area is an example of a computer-readable storage medium storing an information processing program for causing the computer to function as the information processing device according to the first embodiment. The main memory 52 may store data necessary for the processor 51 to execute processes for controlling each component in a nonvolatile or volatile memory area. The main memory 52 uses the volatile memory area as a work area where data can be rewritten by the processor 51 as needed. The nonvolatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

[0049] The auxiliary storage device 53 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM, a HDD, or an SSD can be the auxiliary storage device 53. The auxiliary storage device 53 stores data used by the processor 51 when performing various processes, data created by the processes in the processor 51, etc. The auxiliary storage device 53 may also store the control program. In this case, the auxiliary storage device 53 is an example of a computer-readable storage medium that stores an information processing program for causing the computer to function as the information processing device according to the first embodiment.

[0050] The communication interface 54 may be a general-purpose communication device that performs wired or wireless data communication with the attendant terminal 3. The communication interface 54 may also have a general video interface function for inputting camera images from the checkout area camera 4.

[0051] The input / output device 55 is a user interface that includes input devices such as a keyboard and a mouse, and a display device such as a liquid crystal display. Note that the monitoring device 5 does not necessarily have to include the input / output device 55. In this case, an information processing terminal (not shown) that can communicate via the communication interface 54 can be used as the user interface. The information processing terminal can include, for example, a personal computer, a smartphone, a tablet computer, etc. Furthermore, the communication interface 54 may be provided with a function for communicating with the store server 2 via the communication network NW, thereby enabling the user interface provided by the store server 2 to be used.

[0052] The monitoring device 5 includes, in the auxiliary storage device 53, a setting storage unit 531, a camera image temporary storage unit 532, a tracking table 533, and a linking table 534, for example.

[0053] The setting memory unit 531 stores the terminal number assigned to each transaction processing device 1. Furthermore, the setting memory unit 531 stores a specified position for detecting a customer SC who attempts to leave the store without paying. This specified position can be set in advance by the administrator of the monitoring device 5, for example, using the input / output device 55. For example, the specified position can be set to be outside a rectangular specified range RR, as shown by the dashed line in FIG. 2 , that is preset for the checkout area CA. The shape of the specified range RR can be set arbitrarily depending on the layout of the checkout area CA. Furthermore, the specified position may be set to a position on the entrance ENT or exit EXT side of a specified line arbitrarily set for the checkout area CA. In other words, the specified position can be a position beyond the specified line on the entrance ENT or exit EXT side. The specified line may be, for example, one side of the specified range RR on the entrance ENT or exit EXT side. Furthermore, setting storage unit 531 stores installation location information that is uniquely determined based on the arrangement of each transaction processing device 1 in checkout area CA, such as the position of patrol lamp 111 of each transaction processing device 1, and the position of the operation range where customer SC operating each transaction processing device 1 should be located. Furthermore, setting storage unit 531 can also store installation location information of attendant terminal 3.

[0054] The camera image temporary storage unit 532 temporarily stores camera images taken by the checkout area camera 4 for a certain period of time.

[0055] The tracking table 533 stores the location of each customer SC who entered the checkout area CA. FIG. 8 is a schematic diagram showing the main data structure stored in this tracking table 533. Note that the data structure shown in FIG. 8 is an example. The format is not particularly limited as long as the necessary information for the processor 51 can be obtained. As shown by the dashed-dotted lines in FIG. 2, the processor 51 sets rectangular ROIs (Regions of Interest) as regions of interest from which images of each customer SC who entered the checkout area CA are cut out for the camera images captured by the checkout area camera 4, and assigns ROI numbers to distinguish each ROI. Therefore, the ROI numbers can also be considered customer numbers. As shown in FIG. 8, the tracking table 533 stores, for example, the coordinates in the camera images captured by the checkout area camera 4 as the location of the customer SC for each ROI number of each customer SC who entered the checkout area CA. Although the coordinates are not limited to this, in the example of FIG. 8, the coordinates are the coordinates of two diagonal points of the rectangular ROI.

[0056] Correlation table 534 stores the correspondence between customer SCs and transaction processing devices 1. FIG. 9 is a schematic diagram showing the main data structure stored in this correlation table 534. Note that the data structure shown in FIG. 9 is one example. As long as the necessary information for processor 51 can be obtained, the format is not particularly limited. For example, as shown in FIG. 9, correlation table 534 stores the terminal number assigned to transaction processing device 1 for each ROI number that identifies a customer SC. Note that the same terminal number may be stored for multiple ROI numbers. This may occur, for example, when a married couple is shopping together.

[0057] Next, each unit implemented in the processor 51 of the monitoring device 5 will be described. As shown in Fig. 7 , the processor 51 implements, for example, a tracking unit 511, a linking unit 512, a terminal status detection unit 513, and a fraud determination unit 514. Each unit implemented in the processor 51 can also be referred to as each function. Each unit implemented in the processor 51 can also be referred to as being implemented in a control unit including the processor 51 and the main memory 52.

[0058] Tracking unit 511 receives camera images from checkout area cameras 4 via communication interface 54 and stores the received camera images in camera image temporary storage unit 532. The camera images include images of customers moving within checkout area CA as well as images of patrol lights 111 on each transaction processing device 1. Tracking unit 511 stores camera image temporary storage unit 532 with a duration that allows for determining whether patrol lights 111 are flashing or lit. In this way, tracking unit 511 can acquire image information that allows for identifying the status of each transaction processing device 1 where the customer SC performs settlement-related operations. Tracking unit 511 also tracks the movement of each customer SC in checkout area CA from the camera images for a certain period of time stored in camera image temporary storage unit 532 and stores the current location information of each customer SC in tracking table 533. Thus, tracking unit 511 is an example of a tracking unit that tracks the movement of a customer SC.

[0059] Linking unit 512 identifies customer SCs within the operation range of each transaction processing device 1 based on the location information of each customer SC stored in tracking table 533 and the installation location information of each transaction processing device 1 stored in setting memory unit 531. Linking unit 512 then registers the identified customer SCs operating each transaction processing device 1 in linking table 534. Linking unit 512 is an example of a linking unit that links a customer SC with a transaction processing device 1 on which the customer SC performs settlement-related operations based on the location of the customer SC being tracked.

[0060] Terminal status detection unit 513 detects the status of each transaction processing device 1 based on a certain period of camera footage stored in camera image temporary storage unit 532. Specifically, the camera footage includes images of patrol lights 111 possessed by each transaction processing device 1. In other words, terminal status detection unit 513 acquires, from camera image temporary storage unit 532, image information capable of identifying the status of each transaction processing device 1 on which the customer SC performs operations related to settlement. Terminal status detection unit 513 determines the status information of each transaction processing device 1 represented by the color pattern and light emission pattern of patrol lights 111 from the images of patrol lights 111 of each transaction processing device 1 captured in the acquired camera footage for the certain period of time. Based on the determined status information, terminal status detection unit 513 determines whether the transaction processing device 1 is in use or idle. As described above, being idle indicates that the previous customer SC has completed payment, i.e., that the operations related to settlement have been completed. In this way, the terminal status detection unit 513 is an example of a detection unit that detects whether or not the settlement-related operations have been completed for each of the multiple transaction processing devices 1 based on image information that can identify the status of each of the multiple transaction processing devices 1 on which the customer SC performs settlement-related operations.

[0061] Fraud determination unit 514 determines whether or not a payment omission fraud has occurred based on the location information of each customer SC stored in tracking table 533, the linking information between each customer SC and transaction processing device 1 stored in linking table 534, and the status of each transaction processing device 1 detected by terminal status detection unit 513. Specifically, fraud determination unit 514 determines that no fraud has occurred if the status of transaction processing device 1 linked to a customer SC located at a specified location, i.e., a customer SC that has moved outside specified range RR or crossed a specified line near entrance ENT or exit EXT, is idle, i.e., the operation related to settlement has been completed. In contrast, fraud determination unit 514 determines that a payment omission fraud has occurred if the status of transaction processing device 1 linked to a customer SC located at a specified location remains in operation, i.e., the operation related to settlement has not been completed. When fraud determination unit 514 determines that a payment omission fraud has occurred, it sends a payment omission notification to attendant terminal 3 via communication interface 54. Fraud determination unit 514 is an example of a notification unit that notifies a customer SC of a missed payment when transaction processing device 1 associated with the customer SC that has moved to a predetermined specified position is not in a state where the settlement-related operations have been completed. Missed payment can be rephrased as missed accounting, unaccounted, missed settlement, unsettled, missed settlement, unsettled, etc.

[0062] The monitoring device 5 can be realized, for example, by using a general-purpose computer device as hardware and writing a control program to the main memory 52 or auxiliary storage device 53. The control program may be stored in the main memory 52 or auxiliary storage device 53 when the monitoring device 5 is transferred, or may be transferred separately from the general-purpose computer device. In the latter case, the control program is recorded on a removable recording medium such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory, or transferred via a network.

[0063] The operation of the monitoring device 5 and the information processing system having such a configuration will be described below.

[0064] 10 is a sequence diagram illustrating a fraud detection operation in a self-checkout system to which the information processing system according to the first embodiment is applied. Note that FIG. 10 shows only one of a plurality of transaction processing devices 1 as a representative.

[0065] As shown in FIG. 10, the monitoring device 5 acquires camera images from the checkout area camera 4 (step S1).

[0066] The monitoring device 5 then tracks each customer SC within the checkout area CA using the camera images (step S2). The monitoring device 5 updates the current location of each tracked customer SC stored in the tracking table 533. When an untracked customer SC, i.e., a new customer SC, enters the checkout area CA, the monitoring device 5 assigns a new ROI number to the customer SC and registers its current location in the tracking table 533.

[0067] Furthermore, monitoring device 5 updates the linkage between customer SC and transaction processing device 1 based on the positional relationship between the current location of each customer SC and each transaction processing device 1 (step S3). That is, if a customer SC that is not linked enters the operation range of any transaction processing device 1, monitoring device 5 registers the linkage between that customer SC and transaction processing device 1 in linkage tracking table 533.

[0068] Furthermore, monitoring device 5 detects the status of each transaction processing device 1 based on the image of patrol lamp 111 of each transaction processing device 1 (step S4).

[0069] Then, monitoring device 5 determines whether or not there is any fraudulent payment omission (step S5). Specifically, for each transaction processing device 1 detected as being in operation, i.e., where the settlement operation has not been completed, monitoring device 5 checks the customer SC associated with that transaction processing device 1 using linking tracking table 533 and obtains the current location of that customer SC from tracking table 533. Then, whether or not there is any fraud is determined based on whether or not the obtained current location of the customer SC is a specified location, i.e., whether or not it is outside specified range RR or whether or not it exceeds a specified line. For example, if transaction processing devices 1 with terminal numbers R2, R3, R7, and R8 are in operation, the locations of the customer SC associated with those transaction processing devices 1 are confirmed. At this time, if the situation is as shown in FIG. 2, the customer SC associated with those transaction processing devices 1 are not in the specified location, i.e., they are within specified range RR or have not exceeded the specified line, and therefore it is determined that no fraud has occurred.

[0070] The operations of steps S1 through S5 are repeated thereafter. While FIG. 10 shows that the operations of steps S1 through S3 are followed by the operations of steps S4 and S5, the operations of steps S4 and S5 may be performed each time the operations of steps S1 through S3 are performed multiple times. In other words, simply because customer SC leaves the operational range of each transaction processing device 1 does not mean that he or she immediately leaves the specified range RR or crosses the specified line. Therefore, there is no need for a one-to-one relationship between the operations of steps S1 through S3 and the operations of steps S4 and S5.

[0071] Furthermore, for example, if transaction processing device 1 with terminal number R1 is in operation in addition to transaction processing devices 1 with terminal numbers R2, R3, R7, and R8 described above, in the situation shown in FIG. 2, customer SC associated with transaction processing device 1 with terminal number R1 is in a specified location, that is, outside specified range RR or beyond the specified line, and monitoring device 5 determines in step S5 that a payment omission fraud has occurred. In such a case, monitoring device 5 sends an alert notification to attendant terminal 3 (step S6). The alert notification includes information indicating transaction processing device 1 where a payment omission occurred, in this example, terminal number R1. The alert notification may also include an image of customer SC in the specified location, extracted from camera footage.

[0072] When an alert notification is sent from monitoring device 5, attendant terminal 3 displays the alert notification, for example, as a pop-up display (step S7). After confirming this alert notification, the store clerk SP performs a predetermined stop operation on attendant terminal 3 to stop the alert notification (step S8), and then takes necessary action, such as warning the corresponding customer SC. In response to this predetermined stop operation, attendant terminal 3 sends a terminal status update instruction to the corresponding transaction processing device 1 (step S9).

[0073] Transaction processing device 1, which has received this terminal status update instruction, updates its status to "idle" and changes patrol lamp 111 to indicate "idle" (step S10).

[0074] The operation of the monitoring device 5 for realizing such an operation will be described below. Fig. 11 is a flowchart showing an example of the main steps of the tracking process executed by the processor 51 of the monitoring device 5. In this specification, unless otherwise specified, the processing operation of the processor 51 transitions from ACTn (n is a natural number) to ACT(n+1). The procedure shown in Fig. 10 is an example. There are no particular limitations on the procedure as long as similar results can be obtained.

[0075] In ACT101, the tracking unit 511 implemented in the processor 51 acquires camera images transmitted from the checkout area camera 4 via the communication network NW through the communication interface 54. The tracking unit 511 stores the acquired camera images in the camera image temporary storage unit 532.

[0076] In ACT 102, the tracking unit 511 detects people in the checkout area CA from the temporarily stored camera video using known motion detection technology and person detection technology.

[0077] In ACT 103, the tracking unit 511 determines whether or not a new person has been detected. Specifically, the tracking unit 511 compares the location information of each detected person with the location information stored in the tracking table 533, and determines whether or not there is a person whose location information is not within a distance, i.e., within a number of pixels, determined based on the detection time interval. In addition, the range within the checkout area CA where this determination is made may be limited to the vicinity of the entrance ENT. If no new person has been detected, the tracking unit 511 determines NO in ACT 103 and proceeds to the processing of ACT 105. If a new person has been detected, the tracking unit 511 determines YES in ACT 103 and proceeds to the processing of ACT 104.

[0078] In ACT 104, the tracking unit 511 adds the newly detected person to the tracking targets. Specifically, the tracking unit 511 assigns an ROI number that is not stored in the tracking table 533 to the newly detected person, that is, the newly entering customer SC, and registers the ROI number in the tracking table 533.

[0079] In ACT 105, the tracking unit 511 updates the location information of each customer SC stored in the tracking table 533 based on the current location of each detected person.

[0080] In ACT 106, linking unit 512 implemented in processor 51 determines whether a new intruder has entered the operation range of each transaction processing device 1. Specifically, for each transaction processing device 1 whose terminal number is not stored in linking table 534, linking unit 512 determines whether a customer SC has entered the operation range stored in setting storage unit 531 based on the location information stored in tracking table 533. If there is no new intruder, linking unit 512 determines NO in ACT 106 and proceeds to the processing of ACT 101. If there is a new intruder, linking unit 512 determines YES in ACT 106 and proceeds to the processing of ACT 107.

[0081] In ACT 107, linking unit 512 registers a new link. Specifically, processor 51 stores in linking table 534 the terminal number of transaction processing device 1 determined to have a new intruder, in association with the ROI number of customer SC determined to be the new intruder. Thereafter, linking unit 512 proceeds to the processing of ACT 101.

[0082] Figure 12 is a flow chart showing an example of the main steps of the payment omission detection process executed by the processor 51 of the monitoring device 5. The processor 51 executes this process at regular intervals, for example, every few seconds, based on a control program stored in the main memory 52 or the auxiliary storage device 53. The steps shown in Figure 12 are an example. There are no particular limitations on the steps as long as similar results can be obtained.

[0083] In ACT 201, terminal status detection unit 513 implemented in processor 51 initializes the value of an internal counter n (not shown) configured in processor 51 to "1." The value of this internal counter n is used to specify the terminal number of transaction processing device 1.

[0084] As ACT202, the terminal status detection unit 513 acquires an image of the patrol lamp 111 of the transaction processing device 1 with terminal number n from a certain period of camera footage stored in the camera image temporary storage unit 532, and detects the terminal status of the transaction processing device 1 based on the status information of the transaction processing device 1 represented by the color pattern and light emission pattern of the patrol lamp 111.

[0085] In ACT 203, the terminal status detection unit 513 determines whether the detected terminal status is in operation. If the terminal status is not in operation, i.e., is idle, the terminal status detection unit 513 determines NO in ACT 203 and proceeds to the processing of ACT 209. If the terminal status is in operation, the terminal status detection unit 513 determines YES in ACT 203 and proceeds to the processing of ACT 204.

[0086] In ACT 204, fraud determination unit 514 implemented in processor 51 checks, using linking table 534, the ROI number, that is, the customer number, linked to transaction processing device 1 having terminal number n.

[0087] In ACT 205, the fraud determination unit 514 confirms the location of the customer SC of the confirmed ROI number. Specifically, the fraud determination unit 514 reads out location information corresponding to the confirmed ROI number from the tracking table 533.

[0088] In ACT 206, the fraud determination unit 514 determines whether the customer SC has moved to a specified position stored in the setting storage unit 531. Specifically, the fraud determination unit 514 determines whether the current position of the customer SC indicated by the confirmed position information is outside the specified range RR stored in the setting storage unit 531. It may also be configured to determine whether the current position of the customer SC indicated by the position information exceeds a specified line. If the customer SC has moved to the specified position, the fraud determination unit 514 determines YES in ACT 206 and proceeds to the processing of ACT 215. If the customer SC has not moved to the specified position, the fraud determination unit 514 determines NO in ACT 206 and proceeds to the processing of ACT 207.

[0089] In ACT 207, fraud determination unit 514 determines whether the value of internal counter n is "8," that is, whether the situation has been detected for all transaction processing devices 1 included in the self-checkout system. If the value of internal counter n is "8," fraud determination unit 514 determines YES in ACT 207 and ends this processing. If the value of internal counter n is not "8," fraud determination unit 514 determines NO in ACT 207 and proceeds to processing in ACT 208.

[0090] In ACT 208, the terminal status detection unit 513 increments the value of the internal counter n by 1. After that, the terminal status detection unit 513 proceeds to the process in ACT 202.

[0091] In ACT 209, terminal status detection unit 513 determines whether or not there is a link in association table 534 for transaction processing device 1 with terminal number n, whose terminal status has been determined to be idle. Specifically, terminal status detection unit 513 determines whether or not terminal number n is stored in association table 534. If there is no link, terminal status detection unit 513 determines NO in ACT 209 and proceeds to the processing of ACT 207. If there is a link, terminal status detection unit 513 determines YES in ACT 209 and proceeds to the processing of ACT 210.

[0092] In ACT 210, terminal status detection unit 513 checks, using linking table 534, the ROI number, that is, the customer number, linked to transaction processing device 1 having terminal number n.

[0093] In ACT 211, the terminal status detection unit 513 confirms the location of the customer SC of the confirmed ROI number. Specifically, the terminal status detection unit 513 reads out location information corresponding to the confirmed ROI number from the tracking table 533.

[0094] In ACT 212, the terminal status detection unit 513 determines whether the customer SC has moved to a specified position stored in the setting storage unit 531. Specifically, the terminal status detection unit 513 determines whether the current position of the customer SC indicated by the confirmed position information is outside the specified range RR stored in the setting storage unit 531. It may also be configured to determine whether the position of the customer SC exceeds a specified line. If the customer SC has not moved to the specified position, the terminal status detection unit 513 determines NO in ACT 212 and proceeds to the processing of ACT 207. If the customer SC has moved to the specified position, the terminal status detection unit 513 determines YES in ACT 212 and proceeds to the processing of ACT 213.

[0095] In ACT 213 , the terminal status detection unit 513 deletes the confirmed ROI number and the corresponding location information from the tracking table 533 .

[0096] In ACT 214, the terminal status detection unit 513 deletes the terminal number n stored in the association table 534. After that, the terminal status detection unit 513 proceeds to the process of ACT 207.

[0097] For example, suppose the status of transaction processing device 1 with terminal number R1 detected in the process of ACT 203 is idle, and terminal number R1 is linked to customer SC with ROI number ROI1, as shown in FIG. 9. This corresponds to a situation in which, as shown in FIG. 2, customer SC with ROI number ROI1 has completed a settlement operation at transaction processing device 1 with terminal number R1 and is heading toward the exit EXT of checkout area CA. In such a case, the process proceeds from ACT 203 to ACT 209, and from ACT 209, via ACT 210 and ACT 211, it is confirmed in ACT 212 that the customer SC is in a specified location. Then, the process of ACT 213 and ACT 214 deletes the record for ROI number ROI1 that stores the location information of the customer SC from tracking table 533, and also deletes the link between ROI number ROI1 and terminal number R1, i.e., the corresponding record, from linking table 534.

[0098] In ACT 215, the fraud determination unit 514 implemented in the processor 51 transmits an alert notification to the attendant terminal 3 via the communication interface 54. Thereafter, the fraud determination unit 514 proceeds to the processing of ACT 207 above.

[0099] For example, suppose the status of transaction processing device 1 with terminal number R1 detected in the processing of ACT203 is in operation, and terminal number R1 is linked to customer SC with ROI number ROI1 as shown in Figure 9. This corresponds to a situation in which customer SC with ROI number ROI1 has left the specified range RR and is heading toward the exit EXT of checkout area CA, but has not yet completed the operation related to settlement at transaction processing device 1 with terminal number R1, i.e., a payment has been missed, as shown in Figure 2. In such a case, the processing proceeds from ACT203 to ACT206 via ACT204 and ACT205, and when it is confirmed that customer SC has left the specified range RR, an alert notification is sent in ACT215.

[0100] 13 is a diagram showing an example of monitoring screen 31 displayed on the touch panel of attendant terminal 3 when attendant terminal 3 is connected to communication network NW and has the function of monitoring multiple transaction processing devices 1. Monitoring screen 31 includes terminal list area 32 displaying a list of multiple transaction processing devices 1 to be monitored using icons, and terminal status area 33 displaying the display content of touch panel 104 of each transaction processing device 1. Attendant terminal 3 receives status information transmitted from each transaction processing device 1 indicating the status of that transaction processing device 1 and the display information displayed on touch panel 104 of that transaction processing device 1. Based on the received status information, attendant terminal 3 changes the color, density, or shape of the icon displayed in terminal list area 32 to distinguishably display the current status of each transaction processing device 1. In the example of FIG. 13, transaction processing devices 1 with terminal numbers R1, R2, R3, R7, and R8 are displayed with icons indicating that they are in operation, and transaction processing devices 1 with terminal numbers R4, R5, and R6 are displayed with icons indicating that they are in idle status. Furthermore, attendant terminal 3 displays in terminal status area 33 a reduced image of the screen of each touch panel 104 based on the received display information displayed on touch panel 104 of each transaction processing device 1 .

[0101] FIG. 14 shows an example of monitoring screen 31 upon receiving an alert notification transmitted from monitoring device 5. As described above, the alert notification includes information indicating the transaction processing device 1 where the payment omission occurred and may include an image of the corresponding customer SC extracted from the camera image. Upon receiving such an alert notification, attendant terminal 3 superimposes a pop-up alert 34 based on the alert notification on the display content of each transaction processing device 1 displayed in terminal status area 33 of monitoring screen 31, as shown in FIG. 14. Pop-up alert 34 may include a customer image 35 along with a message indicating which transaction processing device 1 the payment omission occurred at. Furthermore, pop-up alert 34 includes an "OK" button 36 for the attendant store clerk SP to input that he or she has confirmed the occurrence of the payment omission. It goes without saying that the pop-up alert 34 shown in FIG. 14 is merely an example. Furthermore, upon receiving the alert notification, attendant terminal 3 changes the icon of the corresponding terminal number displayed in terminal list area 32 (in this example, the icon of the transaction processing device 1 with terminal number R1) to an icon that clearly indicates the occurrence of the payment omission, as shown in FIG. 14.

[0102] When "OK" button 36 in pop-up alert 34 is operated, attendant terminal 3 sends a terminal status update instruction to transaction processing device 1 that has been determined to have experienced a missed payment based on the alert notification. In other words, operating "OK" button 36 is an example of a predetermined stop operation that instructs the cessation of alert notifications. As described above, transaction processing device 1 that receives this terminal status update instruction, for example, transaction processing device 1 with terminal number R1 in the above example, updates its status to "idle" and changes patrol lamp 111 to indicate "idle."

[0103] If attendant terminal 3 is a general-purpose personal computer or tablet terminal that does not have the function of monitoring transaction processing device 1, the same display as pop-up alert 34 will be displayed on its display screen. In this case, "OK" button 36 is a button that instructs to end the display of pop-up alert 34. To actually update the status of transaction processing device 1 and change the display of patrol lamp 111, store clerk SP, who will act as an attendant, will go to the location where transaction processing device 1 is installed and perform the specified update operation.

[0104] Of course, attendant terminal 3 does not have the function of monitoring transaction processing device 1 , but may have the function of connecting to communication network NW and updating the status of transaction processing device 1 .

[0105] By updating the status of transaction processing device 1 in this way, in the next payment failure detection process executed by monitoring device 5 after a certain period of time has elapsed, the status of transaction processing device 1 with terminal number R1 detected in ACT 202 is idle, so the process proceeds from ACT 203 to ACT 209. Then, from ACT 209, via ACT 210 and ACT 211, the process in ACT 212 confirms that the customer SC is in the specified location. Then, by ACT 213 and ACT 214, the record for ROI number ROI1 that stores the location information of the customer SC is deleted from tracking table 533, and the link between ROI number ROI1 and terminal number R1 in linking table 534, i.e., the corresponding record, is deleted.

[0106] As described above, monitoring device 5, to which the information processing device according to the first embodiment is applied, uses tracking unit 511 to detect whether the settlement-related operations have been completed for each of multiple transaction processing devices 1, based on image information that identifies the status of each of multiple transaction processing devices 1 on which the customer SC performs settlement-related operations. Monitoring device 5 also uses tracking unit 511 to track the movement of the customer SC, and uses linking unit 512 to link the customer SC to the transaction processing device 1 on which the customer SC performs settlement-related operations, based on the location of the tracked customer SC. Monitoring device 5 then uses fraud determination unit 514 to notify of an incomplete payment if the status of transaction processing device 1 linked to a customer SC that has moved to a predetermined location is not one in which the settlement-related operations have been completed. In this way, the monitoring device 5 incorporating the information processing device according to the first embodiment uses image information to detect whether the transaction processing device 1 has completed the settlement process, eliminating the need to communicate with the transaction processing device 1. When the customer SC moves to a specified location, if the transaction processing device 1 associated with the customer SC has not completed the settlement process, the monitoring device 5 notifies the customer of a missed payment. This makes it easy to notify the customer of a missed payment. To combat fraudulent missed payments, gate systems have been introduced, such as those installed at the exit of a checkout area, where a receipt issued by a checkout terminal upon completion of payment can be held over the gate to open the gate. However, these gate systems are expensive and require users to hold a receipt over the gate, resulting in a poor user experience. Therefore, a notification system has also been proposed that communicates with a checkout terminal to obtain its status information, detects the customer's location, and notifies a store clerk or other personnel if the customer leaves the area where the checkout terminal is installed before completing the transaction process. However, in order to connect to a self-checkout system installed in a store and obtain status information from the accounting terminal, knowledge of the information exchanged with the self-checkout system is required.Therefore, if knowledge about the checkout system itself installed in the store is not available or cannot be obtained, it may be impossible to install the notification system. In contrast, monitoring device 5 to which the information processing device according to the first embodiment is applied does not need to communicate with transaction processing device 1, and therefore can be applied to a store even if knowledge about the checkout system itself installed in the store is not available or cannot be obtained.

[0107] Here, in the first embodiment, when a customer SC associated with transaction processing device 1 that has been detected as not having completed its settlement operations moves to a specified position, monitoring device 5 notifies the customer of an omitted payment via fraud determination unit 514. Thus, monitoring device 5 in the first embodiment performs omitted payment determination only for customers SC associated with transaction processing device 1, rather than for all customers SC entering checkout area CA, thereby reducing the amount of processing. Furthermore, omitted payment determination is not performed for customers SC and store clerk SP not associated with transaction processing device 1, thereby reducing the occurrence of erroneous notifications. Typically, store clerk SP, who is an attendant, pays particular attention to customers SC who enter checkout area CA and attempt to exit checkout area CA without operating transaction processing device 1, so the probability of a problem occurring is low even without monitoring by monitoring device 5.

[0108] Furthermore, monitoring device 5 in the first embodiment detects, by terminal status detection unit 513, whether transaction processing device 1 has completed operations for settlement, based on an image of patrol lamp 111 included in image information obtained by photographing patrol lamp 111, which is provided in transaction processing device 1 and indicates whether transaction processing device 1 has completed operations for settlement. Thus, according to monitoring device 5 in the first embodiment, by determining the status of transaction processing device 1 displayed by patrol lamp 111, it is possible to detect the status of transaction processing device 1 without communicating with transaction processing device 1.

[0109] In addition, the information processing system of the first embodiment applied to a self-checkout system includes an attendant terminal 3 used by a store clerk, a checkout area camera 4 that acquires image information that allows the status of each of the multiple transaction processing devices 1 on which the customer SC performs operations related to settlement to be identified, and a monitoring device 5 in the first embodiment that communicates with the attendant terminal 3 and the checkout area camera 4.The monitoring device 5 uses a terminal status detection unit 513 to detect whether operations related to settlement have been completed for each of the multiple transaction processing devices 1 based on the image information acquired by the checkout area camera 4, and if the status of the transaction processing device 1 linked to the customer SC that has moved to a predetermined specified position is not one in which operations related to settlement have been completed, the fraud determination unit 514 notifies the attendant terminal 3 of an incomplete payment. In this way, according to the information processing system of the first embodiment, image information is used to detect whether the transaction processing device 1 has completed operations related to settlement, eliminating the need to communicate with the transaction processing device 1.When a customer SC moves to a specified position, if the transaction processing device 1 linked to that customer SC has not completed operations related to settlement, a notification of missed payment is sent, making it possible to easily notify of missed payment.

[0110] Furthermore, in the information processing system according to the first embodiment, the monitoring device 5 tracks the movement of the customer SC using the tracking unit 511 based on the camera images captured by the checkout area camera 4. This eliminates the need to prepare a camera separate from the checkout area camera 4 for tracking the movement of the customer SC, making it possible to build an information processing system at low cost.

[0111] Second Embodiment Next, a second embodiment will be described. Note that the same reference numerals as in the first embodiment are used to designate the same configurations and operations as in the first embodiment, and descriptions thereof will be omitted.

[0112] 15 is a flowchart showing an example of the main steps of the payment omission detection process executed by the processor 51 of the monitoring device 5 to which the information processing device according to the second embodiment is applied. The processor 51 executes this process at regular intervals, for example, every few seconds, based on a control program stored in the main memory 52 or the auxiliary storage device 53. The steps shown in FIG. 15 are merely an example. There are no particular limitations on the steps as long as similar results can be obtained.

[0113] In ACT 251, the terminal status detection unit 513 implemented in the processor 51 initializes the value of an internal counter n to “1.” In this embodiment, the value of this internal counter n is used to specify an ROI number.

[0114] In ACT 252, the terminal status detection unit 513 acquires the location information of the customer SC of the ROI number n, that is, ROIn. Specifically, the terminal status detection unit 513 reads the location information of the customer SC of ROIn from the tracking table 533.

[0115] In ACT 253, the terminal status detection unit 513 determines whether the customer SC has moved to a specified position stored in the setting storage unit 531. Specifically, the terminal status detection unit 513 determines whether the current position of the customer SC indicated by the acquired position information is outside the specified range RR stored in the setting storage unit 531. It may also be configured to determine whether the current position of the customer SC indicated by the position information exceeds a specified line. If the customer SC has moved to the specified position, the terminal status detection unit 513 determines YES in ACT 253 and proceeds to the processing of ACT 256. If the customer SC has not moved to the specified position, the terminal status detection unit 513 determines NO in ACT 253 and proceeds to the processing of ACT 254.

[0116] In ACT 254, terminal status detection unit 513 determines whether the value of internal counter n is "8," that is, whether the status has been detected for all transaction processing devices 1 included in the self-checkout system. If the value of internal counter n is "8," terminal status detection unit 513 determines YES in ACT 254 and ends this processing. If the value of internal counter n is not "8," terminal status detection unit 513 determines NO in ACT 254 and proceeds to the processing of ACT 255.

[0117] In ACT 255, the terminal status detection unit 513 increments the value of the internal counter n by 1. After that, the terminal status detection unit 513 proceeds to the process in ACT 252.

[0118] In ACT 256, fraud determination unit 514 implemented in processor 51 checks, using linking table 534, the terminal number of transaction processing device 1 linked to customer SC of ROI number n.

[0119] In ACT 257, the fraud determination unit 514 acquires an image of the patrol lamp 111 of the transaction processing device 1 corresponding to the confirmed terminal number from a certain period of camera footage stored in the temporary camera image storage unit 532, and detects the terminal status of the transaction processing device 1 based on the status information of the transaction processing device 1 represented by the color pattern and light emission pattern of the patrol lamp 111.

[0120] In ACT 258, the fraud determination unit 514 determines whether the detected terminal status is in operation. If it is in operation, the fraud determination unit 514 determines YES in ACT 258 and proceeds to the processing of ACT 261. If it is not in operation, i.e., if it is idle, the fraud determination unit 514 determines NO in ACT 258 and proceeds to the processing of ACT 259.

[0121] In ACT 259, the fraud determination unit 514 deletes the ROI number n, that is, ROIn, and the corresponding location information from the tracking table 533. That is, the fraud determination unit 514 deletes the record of ROIn.

[0122] In ACT260, the fraud determination unit 514 deletes the ROI number n, i.e., ROIn, and the corresponding terminal number stored in the association table 534. That is, the fraud determination unit 514 deletes the record of ROIn. Thereafter, the terminal status detection unit 513 proceeds to the process of ACT254.

[0123] In ACT261, the fraud determination unit 514 transmits an alert notification to the attendant terminal 3 via the communication interface 54. Thereafter, the fraud determination unit 514 proceeds to the process of ACT254.

[0124] As described above, monitoring device 5 incorporating the information processing device according to the second embodiment uses terminal status detection unit 513 to detect the status of transaction processing device 1 associated with a tracked customer SC when the customer SC moves to a specified location. When transaction processing device 1 associated with the customer SC who has moved to the specified location has not completed the settlement-related operations, monitoring device 5 utilizes fraud determination unit 514 to notify the customer of an missed payment. Thus, monitoring device 5 incorporating the information processing device according to the second embodiment performs missed payment determination only for transaction processing device 1 associated with the customer SC who has moved to the specified location, rather than for all transaction processing devices 1 installed in the checkout area CA, thereby reducing the amount of processing. Furthermore, missed payment determination is not performed for customers SC or store clerk SP who are not associated with transaction processing device 1, thereby reducing the occurrence of false notifications. Typically, store clerk SP, who is an attendant, pays particular attention to customers SC who enter the checkout area CA and attempt to exit the checkout area CA without operating transaction processing device 1. Therefore, the probability of a problem occurring is low even without monitoring by monitoring device 5.

[0125] [Third embodiment] Monitoring device 5, to which the information processing device according to the first and second embodiments is applied, determines the current status of each transaction processing device 1 based on the display state of patrol lamp 111. This embodiment is an example in which the object of determination used to determine the current status of each transaction processing device 1 is something other than patrol lamp 111. Specifically, in this embodiment, the display screen of touch panel 104 of each transaction processing device 1 is the object of determination. The following description will be given taking the case where transaction processing device 1 is self-checkout terminal 100 as an example, but it goes without saying that transaction processing device 1 may also be self-checkout terminal 150.

[0126] 16 is a diagram showing an example of an idle screen 1041 displayed on touch panel 104 of self-service POS terminal 100, which is an example of transaction processing device 1 in the third embodiment. When self-service POS terminal 100 is available, a specified idle screen 1041 indicating this is displayed on touch panel 104. For example, as shown in FIG. 16, idle screen 1041 includes a display that shows the terminal number of self-service POS terminal 100 to the customer SC.

[0127] 17 is a diagram showing an example of a transaction start screen 1042 displayed on the touch panel 104 of the self-checkout POS terminal 100. For example, when the customer SC touches the touch panel 104, the idle screen 1041 in FIG.

[0128] 18 is a diagram showing an example of a registration screen 1043 displayed on the touch panel 104 of the self-checkout POS terminal 100. The registration screen 1043 displays a list of registered products, their prices, and the like.

[0129] 19 is a diagram showing an example of a payment selection screen 1044 displayed on the touch panel 104 of the self-checkout POS terminal 100. When the "Proceed to Checkout" button 10431 included in the registration screen 1043 shown in FIG. 18 is touched, the screen switches to this payment selection screen 1044.

[0130] 20 is a diagram showing an example of transaction completion screen 1045 displayed on touch panel 104 of self-checkout POS terminal 100. When the settlement at transaction processing device 1 is completed, transaction completion screen 1045 is displayed. After transaction completion screen 1045 is displayed for a certain period of time, the screen returns to idle screen 1041 shown in FIG.

[0131] Therefore, in this embodiment, an image of the display screen of the touch panel 104 of each transaction processing device 1 is extracted from the camera image captured by the checkout area camera 4, and the current status of the transaction processing device 1 is determined based on the screen that is displayed.

[0132] Specifically, terminal status detection unit 513 of monitoring device 5 determines that transaction processing device 1 is in an idle state if touch panel 104 displays idle screen 1041. If the display on touch panel 104 switches from idle screen 1041 to transaction start screen 1042, terminal status detection unit 513 determines that transaction processing device 1 has entered an operating state. If touch panel 104 displays registration screen 1043, payment selection screen 1044, a settlement screen (not shown), or the like, terminal status detection unit 513 determines that transaction processing device 1 is in an operating state. Furthermore, if touch panel 104 displays transaction end screen 1045, terminal status detection unit 513 may determine that self-checkout POS terminal 100 has entered an idle state immediately without waiting for the display to switch to idle screen 1041, or after a certain amount of time has passed.

[0133] In this way, touch panel 104 is an example of a display member provided in transaction processing device 1 that displays whether transaction processing device 1 has completed operations related to settlement. It can also be said that touch panel 104 is an example of a display member that displays, by using at least one of letters and symbols, whether transaction processing device 1 is in an idle state or an operating state.

[0134] Specifically, in ACT202 and ACT257, the monitoring device 5 extracts the display screen portion of the touch panel 104, rather than the patrol lamp 111 portion, from the camera image, and detects the status of the transaction processing device 1 based on the display status of that display screen.

[0135] In the first and second embodiments, a certain amount of video was required to determine whether the patrol lamp 111 was flashing or lit, but in this embodiment, a one-shot image may be sufficient.

[0136] As described above, monitoring device 5 to which the information processing device of the third embodiment is applied detects, by terminal status detection unit 513, whether transaction processing device 1 has completed operations for settlement, based on an image of the display screen of touch panel 104 included in image information obtained by capturing an image of the display screen of touch panel 104, which displays whether transaction processing device 1 has completed operations for settlement. Therefore, according to monitoring device 5 to which the information processing device of the third embodiment is applied, the status of transaction processing device 1 can be determined by capturing an image of touch panel 104 displaying the status of transaction processing device 1.

[0137] Touch panel 104 indicates the status of transaction processing device 1 by at least one of characters and symbols such as buttons. Therefore, according to monitoring device 5 in the third embodiment, the status of transaction processing device 1 can be easily determined simply by determining the characters and / or symbols displayed on the screen of touch panel 104.

[0138] [Fourth embodiment] In the third embodiment, the display screen of the touch panel 104 of each transaction processing device 1 is the object of judgment, but in this embodiment, the display screen of the touch panel provided in the attendant terminal 3 is the object of judgment.

[0139] Fig. 21 is a schematic diagram showing a self-checkout system to which an information processing system according to the fourth embodiment is applied, including a monitoring device 5 to which an information processing device according to the fourth embodiment is applied. Fig. 22 is a schematic diagram showing an example of a checkout area CA of a store in the fourth embodiment.

[0140] In this embodiment, attendant terminal 3 is connected to communication network NW and has the function of monitoring multiple transaction processing devices 1. As shown in Figures 21 and 22, the information processing system according to the fourth embodiment includes, in addition to checkout area camera 4, attendant screen camera 6 installed so that its imaging range 61 corresponds to the display screen of the touch panel of attendant terminal 3. Attendant screen camera 6 transmits captured images to monitoring device 5.

[0141] 13 is displayed on the touch panel display screen of attendant terminal 3, which has the function of monitoring transaction processing devices 1. As described above, attendant terminal 3 changes the color, density, or shape of the icon displayed in terminal list area 32 of monitoring screen 31 based on the status information received from each transaction processing device 1, thereby displaying the current status of each transaction processing device 1 in an identifiable manner.

[0142] Therefore, in this embodiment, attendant screen camera 6 captures an image of monitoring screen 31 displayed on touch panel 104 of attendant terminal 3. Then, terminal status detection unit 513 of monitoring device 5 extracts an image showing the status of transaction processing device 1 to be detected in terminal list area 32 from the image captured by attendant screen camera 6, and determines the current status of transaction processing device 1 to be detected based on the type of image displayed. By fixing the positional relationship between attendant screen camera 6 and monitoring screen 31 of attendant terminal 3, it is possible to know which pixel in the captured image corresponds to which icon of transaction processing device 1. If the positional relationship is not fixed, monitoring device 5 can determine which pixel in the captured image corresponds to which icon of transaction processing device 1 by determining terminal list area 32 using a known image recognition process. Of course, the current status of the transaction processing device 1 may be determined using the image in the terminal status area 33 instead of the image in the terminal list area 32, or the accuracy of the determination may be improved by using both the image in the terminal list area 32 and the image in the terminal status area 33.

[0143] In this way, attendant terminal 3 is an example of an attendant terminal that communicates with each transaction processing device 1 and displays the status of each transaction processing device 1, and the touch panel of the attendant terminal that displays monitoring screen 31 is an example of a display screen that displays the status of multiple transaction processing devices 1. Attendant screen camera 6 is an example of an acquisition device that acquires image information that can identify the status of each of multiple transaction processing devices 1.

[0144] Specifically, in ACT202 and ACT257, the monitoring device 5 extracts an image portion, such as an icon related to the transaction processing device 1 to be detected, from the image captured by the attendant screen camera 6 on the monitoring screen 31 displayed on the touch panel of the attendant terminal 3, and detects the status of the transaction processing device 1 based on the display content of that image portion.

[0145] As described above, monitoring device 5 to which the information processing device according to the fourth embodiment is applied detects, by terminal status detection unit 513, whether transaction processing device 1 has completed operations related to settlement, based on an image of the status display for multiple transaction processing devices 1 included in image information obtained by capturing monitoring screen 31 displaying the status of multiple transaction processing devices 1 at attendant terminal 3, which communicates with each transaction processing device 1 and displays the status of each transaction processing device 1. Therefore, according to monitoring device 5 to which the information processing device according to the fourth embodiment is applied, the status of transaction processing device 1 can be easily determined by capturing an image of monitoring screen 31 of attendant terminal 3 displaying the status of transaction processing device 1.

[0146] Other Embodiments Although the embodiments of the information processing device have been described above, the embodiments are not limited to these.

[0147] For example, in the first to third embodiments, images of the patrol lamp 111 and touch panel 104 of each transaction processing device 1 are extracted from the camera image of the checkout area camera 4, but each transaction processing device 1 may also be equipped with a camera that captures the patrol lamp 111 and touch panel 104.

[0148] Furthermore, the conditions under which each transaction processing device 1, based on the camera image from the checkout area camera 4, detects that the settlement-related operations have not been completed may not be limited to the conditions described in the embodiment, but may be set arbitrarily by the administrator of the monitoring device 5 as an information processing device.

[0149] Furthermore, the information processing device is not limited to the monitoring device 5, and its functions may be provided in a computer constituting the attendant terminal 3 or the store server 2. Furthermore, by connecting an internet connection device to the communication network NW, a server computer provided on the cloud may be used as the information processing device.

[0150] The program that causes a computer to function as an information processing device according to the embodiment may be transferred in a state stored in an electronic device, or may be transferred in a state not stored in an electronic device. In the latter case, the program may be transferred via a network, or may be transferred in a state stored in a storage medium. The storage medium is a non-transitory tangible medium. The storage medium is a computer-readable medium. The storage medium may be in any form, such as a CD-ROM or a memory card, as long as it is capable of storing the program and is computer-readable.

[0151] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims.

Claims

1. An information processing device comprising: a detection unit that detects whether or not the settlement-related operations have been completed for each of a plurality of transaction processing devices based on image information that can identify the status of each of the plurality of transaction processing devices on which a customer performs settlement-related operations; a tracking unit that tracks the movement of the customer; a linking unit that links the customer to the transaction processing device on which the customer performs settlement-related operations based on the location of the customer being tracked; and a notification unit that notifies of an incomplete payment if the status of the transaction processing device linked to the customer who has moved to a predetermined specified position is not such that the settlement-related operations have been completed.

2. The information processing device according to claim 1, wherein the notification unit notifies the customer of the missed payment when the customer associated with the transaction processing device for which the settlement-related operation has been detected as not being completed moves to the specified location.

3. An information processing device as described in claim 1, wherein the detection unit detects the status of the transaction processing device linked to the customer when the customer being tracked moves to the specified location, and the notification unit notifies of the missed payment when the transaction processing device linked to the customer who has moved to the specified location has not completed the operations related to the settlement.

4. An information processing device described in any one of claims 1 to 3, wherein the detection unit detects whether the transaction processing device is in a state where operations related to the settlement have been completed based on an image of a display element included in the image information obtained by photographing a display element provided on the transaction processing device that indicates whether operations related to the settlement have been completed on the transaction processing device.

5. An information processing device as described in any one of claims 1 to 3, wherein the detection unit detects whether the transaction processing device is in a state where the settlement-related operations have been completed based on an image of the status display for the multiple transaction processing devices contained in the image information captured from a display screen displaying the status of the multiple transaction processing devices at an attendant terminal that communicates with each of the transaction processing devices and displays the status of each of the transaction processing devices.

6. An information processing system comprising: an attendant terminal used by a store clerk; an acquisition device that acquires image information that can identify the status of each of multiple transaction processing devices on which a customer performs operations related to settlement; and an information processing device that communicates with the attendant terminal and the acquisition device, wherein the information processing device comprises: a detection unit that detects whether the operations related to settlement have been completed for each of the multiple transaction processing devices based on the image information acquired by the acquisition device; a tracking unit that tracks the movement of the customer; a linking unit that links the customer to the transaction processing device on which the customer performs operations related to settlement based on the position of the customer being tracked; and a notification unit that notifies the attendant terminal of an incomplete payment if the status of the transaction processing device linked to the customer who has moved to a predetermined specified position is not one in which the operations related to settlement have been completed.

7. A computer-readable storage medium storing a program for enabling a computer of an information processing device to perform the following functions: a function for detecting whether or not the settlement-related operations have been completed for each of a plurality of transaction processing devices based on image information that can identify the status of each of the transaction processing devices on which the customer performs settlement-related operations; a function for tracking the movement of the customer; a function for linking the customer to the transaction processing device on which the customer performs settlement-related operations based on the location of the customer being tracked; and a function for notifying the customer of an incomplete payment if the status of the transaction processing device linked to the customer who has moved to a predetermined specified location is not one in which the settlement-related operations have been completed.

Images