Information processing apparatus and information processing system

Abstract

To inexpensively and easily notify a user of payment omission.SOLUTION: The information processing device includes a detection unit, a tracking unit, a linking unit, and a notification unit. The detection unit detects whether or not the operation related to the settlement is completed for each of the plurality of transaction processing apparatuses based on image information for identifying a state of each of the plurality of transaction processing apparatuses in which the customer performs the operation related to the settlement. The tracking unit tracks the movement of the customer. The associator associates the customer with the transaction processing apparatus in which the customer performs the operation relating to the settlement based on the tracked position of the customer. The notification unit notifies a payment omission in a case where a state of the transaction processing apparatus associated with the customer who moves to a predetermined specified position is not a state in which an operation related to settlement is completed.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing system. [Background technology]

[0002] In the past, semi-self-checkout systems were known in which a store clerk would register purchased items at a registration terminal, and the customer would then perform the payment process, i.e., the settlement process, at an accounting terminal. In recent years, full-self-checkout systems have also become popular, in which the customer not only performs the payment process but also the registration process.

[0003] In such self-checkout systems, there is a possibility that customers may register their products and then leave the store without completing payment, resulting in fraudulent payment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-136679 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the embodiments of the present invention is to provide an information processing device and an information processing system that enable easy notification of missed payments. [Means for solving the problem]

[0006] In one embodiment, the information processing device includes a detection unit, a tracking unit, a linking unit, and a notification unit. The detection unit detects whether the settlement-related operations have been completed for each of the transaction processing devices based on image information that identifies the status of each of the transaction processing devices on which the customer performs settlement-related operations. The tracking unit tracks the movement of the customer. The linking unit links the customer to the transaction processing device on which the customer performs settlement-related operations based on the location of the tracked customer. The notification unit notifies the customer of an overdue payment if the status of the transaction processing device linked to the customer who has moved to a predetermined location is not in a state where the settlement-related operations have been completed. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic configuration diagram showing a self-checkout system to which an information processing system according to the first embodiment is applied, including a monitoring device to which an information processing device according to the first embodiment is applied. [Figure 2] FIG. 2 is a schematic diagram showing an example of a checkout area of ​​a store. [Figure 3] FIG. 3 is a perspective view showing an example of the external configuration of a self-checkout POS terminal as a transaction processing device included in the self-checkout system. [Figure 4] FIG. 4 is a block diagram showing an example of a main circuit configuration of a self-service POS terminal. [Figure 5] FIG. 5 is a perspective view showing an example of the external configuration of a self-checkout terminal as a transaction processing device included in the self-checkout system. [Figure 6] FIG. 6 is a diagram illustrating the status information of a transaction processing device represented by the color pattern and light emission pattern of a patrol lamp provided on the transaction processing device. [Figure 7] FIG. 7 is a block diagram showing an example of a main circuit configuration of a monitoring device. [Figure 8] FIG. 8 is a schematic diagram showing the main data structures stored in the tracking table of the monitoring device. [Figure 9]FIG. 9 is a schematic diagram showing the main data structure stored in the association table of the monitoring device. [Figure 10] FIG. 10 is a sequence diagram for explaining a fraud detection operation in a self-checkout system to which the information processing system according to the first embodiment is applied. [Figure 11] FIG. 11 is a flowchart showing an example of the main steps of the tracking process executed by the processor of the monitoring device. [Figure 12] FIG. 12 is a flowchart showing an example of the main steps of a payment omission detection process executed by the processor of the monitoring device. [Figure 13] FIG. 13 is a diagram showing an example of a monitoring screen displayed on the touch panel of the attendant terminal. [Figure 14] FIG. 14 is a diagram showing an example of a monitoring screen when an alert notification is received. [Figure 15] FIG. 15 is a flowchart showing an example of the main steps of a payment omission detection process executed by a processor of a monitoring device to which an information processing device according to the second embodiment is applied. [Figure 16] FIG. 16 is a diagram showing an example of an idle screen displayed on a touch panel of a self-service POS terminal as an example of a transaction processing apparatus according to the third embodiment. [Figure 17] FIG. 17 is a diagram showing an example of a transaction start screen displayed on the touch panel of the self-service POS terminal. [Figure 18] FIG. 18 is a diagram showing an example of a registration screen displayed on the touch panel of the self-service POS terminal. [Figure 19] FIG. 19 is a diagram showing an example of a payment selection screen displayed on the touch panel of the self-service POS terminal. [Figure 20] FIG. 20 is a diagram showing an example of a transaction completion screen displayed on the touch panel of the self-service POS terminal. [Figure 21] FIG. 21 is a schematic configuration diagram showing a self-checkout system to which an information processing system according to the fourth embodiment is applied, including a monitoring device to which an information processing device according to the fourth embodiment is applied. [Figure 22] FIG. 22 is a schematic diagram showing an example of a checkout area of ​​a store in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of an information processing device and an information processing system will be described with reference to the drawings.

[0009] [First embodiment] FIG. 1 is a schematic diagram showing a self-checkout system to which an information processing system according to a first embodiment is applied. The self-checkout system includes a plurality of self-service transaction processing devices 1 and a store server 2 connected via a communication network NW such as a local area network (LAN). The information processing system according to the first embodiment, which serves as an alarm system, includes an attendant terminal 3, a checkout area camera 4, and a monitoring device 5 to which the information processing device according to the first embodiment is applied. The monitoring device 5 is connected to the attendant terminal 3 and the checkout area camera 4. As shown by the dashed lines in FIG. 1, the attendant terminal 3 may also be connected to a plurality of transaction processing devices 1 and the store server 2 via the communication network NW.

[0010] Multiple transaction processing devices 1, attendant terminals 3, and checkout area cameras 4 are installed in a checkout area CA of a store, as shown by the dashed line in Fig. 1. Fig. 2 is a schematic diagram showing an example of this checkout area CA. In the example of Fig. 2, eight transaction processing devices 1 and one attendant terminal 3 are installed in the checkout area CA. Although not limited to this, the following description of the configuration and operation will be based on the number and installation layout of transaction processing devices 1 in this checkout area CA.

[0011] Checkout area CA has an entrance ENT and an exit EXT for customers. At entrance ENT, a terminal number R1-R8 is assigned to uniquely identify each transaction processing device 1. These terminal numbers R1-R8 can be assigned arbitrarily by the administrator of store server 2. A store clerk SP is stationed at attendant terminal 3.

[0012] The self-service transaction processing device 1 is a terminal where the customer SC performs operations related to the transaction process. This self-service transaction processing device 1 has product registration and payment functions and includes a self-service POS (Point of Sales) terminal where the customer SC (the shopper) registers the product and processes the transaction based on the registered transaction data, and a self-service checkout terminal where the customer SC processes the transaction based on product transaction data registered separately by a store clerk or the customer SC. The checkout terminal can also be referred to as a checkout device, payment terminal, settlement device, or checkout terminal. The POS terminal can also be referred to as a product registration checkout terminal, product registration checkout device, product registration payment terminal, product registration payment device, product registration checkout device, or product registration checkout terminal.

[0013] FIG. 3 is a perspective view showing an example of the external configuration of self-checkout POS terminal 100 as transaction processing device 1 included in the self-checkout system, and FIG. 4 is a block diagram showing an example of the main circuit configuration of self-checkout POS terminal 100.

[0014] As shown in FIG. 3, the self-service POS terminal 100 includes a main body 101 installed on the floor and a scale unit 102 installed beside the main body 101. A display pole 103 and a touch panel 104 are attached to the top of the main body 101. A basket stand 105 is provided in the center of the side of the main body 101 opposite the side on which the scale unit 102 is installed. The basket stand 105 is used by a customer SC, who is a shopper coming from the sales floor, to place a shopping basket containing purchased items. The customer SC works while standing in front of the main body 101 in FIG. 3 so that the screen of the touch panel 104 can be seen. Therefore, as viewed from the customer SC, the basket stand 105 is on the right side of the main body 101, and the scale unit 102 is on the left side. In the following description, the side on which the customer SC stands is referred to as the front of the main body 101, the side on which the scale unit 102 is installed is referred to as the left side of the main body 101, and the side on which the basket stand 105 is installed is referred to as the right side of the main body 101.

[0015] The scale unit 102 has a structure in which a scale pan 107 is provided on top of a housing 106, and a bag holder 108 is attached to the top of the scale pan 107. The top surface of the scale pan 107 serves as a placement surface 109. The bag holder 108 has a pair of holding arms 110, which hold a plastic bag or a shopping bag brought by the customer SC, known as a "my bag." The scale unit 102 measures the weight of an item placed in the plastic bag or my bag held by the holding arms 110 and placed on the placement surface 109. The placement surface 109 is an example of a product placement area where purchased items to be checked out are placed. The scale unit 102 is an example of a weighing device that measures the weight of an object placed in the product placement area.

[0016] The display pole 103 is provided at its tip with a patrol lamp 111 that displays status information indicating the state of the self-service POS terminal 100. The patrol lamp 111 includes a first light-emitting unit 112 that selectively emits light, for example, blue or red, and a second light-emitting unit 113 that emits yellow. The light has two types of light emission patterns: continuous illumination, which is lighting, and flashing, which is lighting and turning off for a certain period of time. The patrol lamp 111 can display status information by combining the color pattern and light emission pattern of the first light-emitting unit 112 with the light emission pattern of the second light-emitting unit 113.

[0017] The touch panel 104 is composed of a display for displaying various screens related to transactions such as registration and accounting to the user operating the self-service POS terminal 100, and a touch sensor for detecting touch inputs to the screen by the user. In the case of the self-service POS terminal 100, the user is usually a customer SC.

[0018] A reading window 115 for a scanner 114 (see FIG. 4), a card insertion slot 117 for a card reader 116 (see FIG. 4), and an issuing slot 119 for receipts printed by a printer 118 (see FIG. 4) are formed on the front of the main body 101. Furthermore, a coin insertion slot 121, a coin dispensing slot 122, a bill insertion slot 123, and a bill dispensing slot 124 of an automatic change dispenser 120 (see FIG. 4) are also formed on the front of the main body 101.

[0019] Furthermore, a communication cable 125 extends from the right side of the main body 101 to the outside, and a reader / writer 126 for electronic money media is connected to the tip of this communication cable 125. The reader / writer 126 is placed on a stand 127 provided on the upper right side of the main body 101.

[0020] As shown in FIG. 4, the self-service POS terminal 100 includes the touch panel 104, patrol lamp 111, scanner 114, card reader 116, printer 118, automatic change dispenser 120, and reader / writer 126 described above, as well as a processor 128, main memory 129, auxiliary storage device 130, a communication interface 131, and a system transmission path 132. The system transmission path 132 includes an address bus, a data bus, and control signal lines. The system transmission path 132 connects the processor 128 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 128, main memory 129, and auxiliary storage device 130 are connected via the system transmission path 132 to form a computer for the self-service POS terminal 100.

[0021] The processor 128 corresponds to the central part of the computer. The processor 128 controls each part to realize various functions of the self-checkout POS terminal 100 in accordance with an operating system or a control program. The processor 128 is, for example, a central processing unit (CPU). The processor 128 may be, for example, a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 128 may be a combination of these.

[0022] The main memory 129 corresponds to the main storage portion of the computer. The main memory 129 includes a nonvolatile memory area and a volatile memory area. The main memory 129 stores an operating system or a control program in the nonvolatile memory area. The main memory 129 may store data required for the processor 128 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 129 uses the volatile memory area as a work area where data is rewritten by the processor 128 as appropriate. An example of the nonvolatile memory area is ROM (Read Only Memory). An example of the volatile memory area is RAM (Random Access Memory). For example, the main memory 129 stores a transaction file 133.

[0023] The transaction file 133 is a data file for storing transaction data related to one transaction processed by the self-service POS terminal 100. The transaction file 133 stores data such as a transaction number, purchased item data, total number of items, total amount, discount amount, and settlement amount. The transaction number is a series of numbers that is issued each time a transaction is processed by the self-service POS terminal 100. The purchased item data is record data created for each item sold in a transaction identified by the transaction number. Here, the purchased item data consists of items such as the item code, item name, price, number of items, and amount. The number of items is the number of items purchased for the item identified by the item code. The amount is the amount for the number of items purchased. The transaction file 133 can store multiple purchased item data. The total number of items is the sum of the number of items for each purchased item data. The total amount is the sum of the amounts for each purchased item data. The discount amount is the amount discounted from the total amount. The settlement amount is the amount obtained by subtracting the discount amount from the total amount.

[0024] The auxiliary storage device 130 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 130 may be an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device 130 stores data used by the processor 128 when performing various processes, data created by the processes in the processor 128, etc. The auxiliary storage device 130 may also store the above-mentioned control program.

[0025] The communication interface 131 performs data communication in accordance with a preset communication protocol with an external device connected via the communication network NW. The external device is, for example, a store server 2, and may also include an attendant terminal 3.

[0026] The scanner 114 reads a code symbol from a product held over the reading window 115. Each product sold in a store is affixed with a code symbol that encodes a product ID or the like to identify the product. The code symbol is, for example, a barcode. The code symbol may also be, for example, a two-dimensional data code. The scanner 114 may be of a type that reads a code symbol by scanning with laser light, or of a type that reads a code symbol from an image captured by an imaging device.

[0027] Card reader 116 reads card data recorded on a card medium such as a credit card, a point card, etc. Card reader 116 pulls the card medium inserted into card insertion slot 117 into main body 101, reads the card data, and then ejects the card from card insertion slot 117.

[0028] The printer 118 prints receipt data and other information indicating the details of the transaction onto receipt paper. The receipt paper with the printed receipt data is ejected from the issuing port 119, cut by a cutter (not shown), and issued as a receipt or invoice.

[0029] The automatic change dispenser 120 includes a coin unit 134 and a banknote unit 135. The coin unit 134 sorts coins inserted into the coin insertion slot 121 one by one, identifies the denomination, and stores them in a safe by denomination. The coin unit 134, for example, based on change data, removes coins of the corresponding denomination from the safe and dispenses them to the coin dispensing outlet 122. The banknote unit 135 sorts banknotes inserted into the banknote insertion slot 123 one by one, identifies the denomination, and stores them in the safe by denomination. The banknote unit 135, for example, based on change data, removes banknotes of the corresponding denomination from the safe and dispenses them to the banknote dispensing outlet 124.

[0030] The reader / writer 126 reads and rewrites electronic money recorded on an electronic money medium. The electronic money medium is, for example, a contactless IC card. The electronic money medium may also be an electronic device such as a smartphone or a tablet terminal.

[0031] Processor 128 implements processing units related to well-known product registration and payment. Each unit implemented in processor 128 can also be referred to as a function. Each unit implemented in processor 128 can also be referred to as being implemented in a control unit including processor 128 and main memory 129.

[0032] In this way, the self-service POS terminal 100 has a settlement function that allows both cash and cashless payments. Cashless payments include credit card payments, electronic money payments, point payments, code payments, etc. A customer SC who places purchased items in a shopping basket or a shopping cart in a sales floor where products are displayed and moves to the checkout area CA operates the self-service POS terminal 100 to register the purchased items and to perform accounting processing for the transaction data of the registered purchased items. In this way, the customer SC performs payment for the transaction by himself, from product registration to payment.

[0033] 5 is a perspective view showing an example of the external configuration of self-checkout terminal 150 as transaction processing device 1 included in the self-checkout system. Components similar to those of self-POS terminal 100 are given the same reference numerals and will not be described again.

[0034] The self-checkout terminal 150 comprises a first housing 151 and a second housing 152.

[0035] On the front of the first housing 151, there are arranged a coin insertion slot 121, a coin dispensing outlet 122, a bill insertion slot 123, and a bill dispensing outlet 124 of the automatic change dispenser 120, and a scanner reading window 115 of the scanner 114. Note that the scanner 114 is not for reading code symbols from merchandise like the self-service POS terminal 100, but is used to read code symbols for code payments.

[0036] Second housing 152 is placed on top surface 153 of first housing 151, and includes printer 118. Printer 118 has a receipt issuing opening 119 provided on front surface 154 of second housing 152.

[0037] A card reader 116 having a card insertion slot 117 is disposed on the top surface 153 of first housing 151, on the left side of second housing 152. A touch panel 104 is attached to the top of second housing 152.

[0038] The self-checkout terminal 150 also has a cylindrical display pole 103 erected on the rear side of the top surface 153 of the first housing 151. The display pole 103 has a patrol lamp 111 at its upper tip, which includes a first light-emitting element 112 and a second light-emitting element 113. The display pole 103 also has a display 155 below the patrol lamp 111, which displays images and text. The display 155 is composed of, for example, an LCD panel. The display 155 displays, for example, the details of any errors that have occurred in the self-checkout terminal 150.

[0039] A basket stand 105 for placing a shopping basket or the like is provided on the right side of the self-checkout terminal 150 as viewed from the front.

[0040] The attendant terminal 3 is a terminal operated by an attendant store clerk SP who is a supervisor who monitors the checkout area CA. The attendant terminal 3 may be a personal computer connected to the monitoring device 5 by wire or wirelessly, or a tablet terminal that can be connected to the monitoring device 5 by wireless and that can be carried by the store clerk.

[0041] Furthermore, attendant terminal 3 may be connected to communication network NW and have a function of monitoring multiple transaction processing devices 1. Furthermore, attendant terminal 3 may have a configuration similar to self-service POS terminal 100. In attendant terminal 3 with a monitoring function, main memory 129 stores the status of each self-service POS terminal 100. For example, attendant terminal 3 can store a list of purchased items registered in each self-service POS terminal 100, error occurrence status, etc. in main memory 129. Attendant terminal 3 displays the status of each self-service POS terminal 100 stored in main memory 129 on the display of touch panel 104. In addition to cash and cashless payment functions, attendant terminal 3 may also have a function for handling payments requiring manual intervention, such as payments using non-machine-readable vouchers and gift certificates, returns, and refunds.

[0042] The store server 2 includes a product master database and a sales data database, and provides various services related to the self-checkout system to each self-checkout POS terminal 100 connected via the communication network NW. The product master database stores product data, such as product code, product name, unit price, and category name, for each product sold in the store, with each product treated as a separate record. The product code is a unique code assigned to each product to identify it individually. The sales data database stores sales data generated by each transaction, transmitted from each transaction processing device 1 upon completion of payment, i.e., upon completion of the transaction. The sales data database stores, with each transaction treated as a separate record, transaction data indicating the transaction details of the products to be paid and payment data related to the payment of the transaction. The transaction data includes the product name (product code) of the product to be paid, its price, the total price of each product, etc. The payment data includes the amount paid by the customer, the amount of change, etc., if payment is made in cash, and information on the payment method if payment is made cashlessly. The sales data may also include elements such as a membership code, a company code (company name) indicating the company that operates the store, a store code (store name), the store's telephone number, address, transaction date and time, transaction number (receipt number), terminal number, and the name of the person in charge. Note that the elements included in the sales data are not limited to these examples and may include other elements, or elements such as the telephone number and address may be removed.

[0043] For example, in transaction processing device 1, if it is self-checkout POS terminal 100, scanner 114 reads a code symbol printed or affixed to a product to obtain a product code. For the product identified by the product code, transaction processing device 1 reads product data from a product data file stored in transaction processing device 1 and registers the product data in the transaction data, thereby updating and storing sales data including transaction data and payment data. The product data file may be a copy of a product master database stored in store server 2. Furthermore, if transaction processing device 1 is self-checkout terminal 150, it receives and stores transaction data registered by a registration terminal (not shown). Alternatively, the scanner of self-checkout terminal 150 scans a paper medium on which a code indicating the transaction data registered by a registration terminal (not shown) is printed. Self-checkout terminal 150 then registers the transaction data and proceeds to the subsequent transaction process. In this case, instead of using a paper medium on which a code indicating the transaction data registered by a registration terminal (not shown) is printed, a code indicating the transaction data may be displayed on the screen of a mobile device held by the consumer, and the code may be scanned by the scanner of self-checkout terminal 150 to register the transaction data and proceed to the subsequent transaction process. When the transaction process is complete, that is, when one transaction is finished, transaction processing device 1 prints the sales data on paper to issue a receipt for that transaction, and also transmits the sales data for that transaction to store server 2. The sales data database of store server 2 stores the sales data transmitted from each transaction processing device 1 in this manner.

[0044] As shown in FIG. 2, checkout area camera 4 is installed on the ceiling of checkout area CA or the like, captures an image of the entire checkout area CA, and transmits the captured camera image to monitoring device 5. Checkout area camera 4 is, for example, a 360-degree camera or a fisheye camera capable of capturing images, and is installed so that all of the multiple transaction processing devices 1 installed in checkout area CA, specifically, the patrol lights 111 of all self-service POS terminals 100, are within its capturing range. Note that if one checkout area camera 4 cannot capture an image of all of the patrol lights 111, multiple checkout area cameras 4 may be used. Checkout area camera 4 is an example of an acquisition device that acquires image information that can identify the status of each of the multiple transaction processing devices 1. Checkout area camera 4 may be an existing surveillance camera installed in checkout area CA.

[0045] Monitoring device 5 detects the status of each transaction processing device 1 based on an image of patrol lamp 111 of each transaction processing device 1 included in the camera image captured by checkout area camera 4. FIG. 6 is a diagram illustrating the status information of transaction processing device 1, i.e., self-service POS terminal 100 and self-checkout terminal 150, represented by the color pattern and light emission pattern of patrol lamp 111. Note that in FIG. 6, solid hatching indicates a lit state, and dashed hatching indicates a flashing state. Patrol lamp 111 indicates that it is in operation by flashing first light-emitting unit 112 in blue, and indicates that it is idle by lighting first light-emitting unit 112 in blue. Therefore, monitoring device 5 can determine the status of each self-service POS terminal by recognizing the color pattern and light emission pattern of patrol lamp 111 of each self-service POS terminal 100 from the camera image captured by checkout area camera 4. In this way, patrol lamp 111 can indicate whether transaction processing device 1 is idle or in operation by at least one of its light emission color and light emission state. Here, the operating state means an unpaid state in which the customer SC has not yet made a payment, and the idle state means a state in which the previous customer SC has made a payment and is waiting for use by the next customer SC, i.e., the previous customer SC has already paid. Therefore, patrol lamp 111 is an example of a display component that indicates whether a customer SC who is using or has used transaction processing device 1 is in an unpaid state or a paid state. While this embodiment uses an example of patrol lamp 111 having two light-emitting elements, first light-emitting element 112 and second light-emitting element 113, it may also be possible to use a single light-emitting element to indicate whether the customer SC is in an unpaid state or a paid state.

[0046] FIG. 7 is a block diagram showing an example of the main circuit configuration of the monitoring device 5. As shown in FIG. As shown in Fig. 7, the monitoring device 5 includes a processor 51, a main memory 52, an auxiliary storage device 53, a communication interface 54, an input / output device 55, and a system transmission path 56. The system transmission path 56 includes an address bus, a data bus, a control signal line, etc. The system transmission path 56 connects the processor 51 to each of the other components directly or via a signal input / output circuit, and transmits data signals exchanged between them. The processor 51, the main memory 52, and the auxiliary storage device 53 are connected by the system transmission path 56 to form a computer of the monitoring device 5. The computer is an example of an information processing device according to the first embodiment.

[0047] The processor 51 corresponds to the central part of the computer. The processor 51 controls each part to realize various functions of the monitoring device 5 in accordance with an operating system or an application program. The processor 51 is, for example, a CPU. The processor 51 may be a multi-core / multi-threaded processor, and can execute multiple processes in parallel. The processor 51 may be, for example, an MPU, an SoC, a DSP, a GPU, an ASIC, a PLD, or an FPGA. Alternatively, the processor 51 may be a combination of two or more of these.

[0048] The main memory 52 corresponds to the main storage portion of the computer. The main memory 52 includes a nonvolatile memory area and a volatile memory area. The main memory 52 stores an operating system or an application program in the nonvolatile memory area. The application program includes, for example, an information processing program for causing the computer to function as the information processing device according to the first embodiment. The main memory 52 may store data required for the processor 51 to execute processes for controlling each component in a nonvolatile or volatile memory area. The main memory 52 uses the volatile memory area as a work area where data is rewritten by the processor 51 as needed. The nonvolatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM. The auxiliary storage device 53 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM, a HDD, or an SSD may be used as the auxiliary storage device 53. The auxiliary storage device 53 stores data used by the processor 51 in performing various processes, data created by the processor 51's processes, etc. The auxiliary storage device 53 may also store the control program.

[0049] The communication interface 54 may be a general-purpose communication device that performs wired or wireless data communication with the attendant terminal 3. The communication interface 54 may also have a general video interface function for inputting camera images from the checkout area camera 4.

[0050] The input / output device 55 is a user interface including input devices such as a keyboard and a mouse, and a display device such as a liquid crystal display. The monitoring device 5 does not necessarily have to include the input / output device 55. In this case, an information processing terminal (not shown) that can communicate via the communication interface 54 can be used as the user interface. The information processing terminal can include, for example, a personal computer, a smartphone, a tablet computer, etc. Furthermore, the communication interface 54 may be provided with a function to communicate with the store server 2 via the communication network NW, thereby enabling the user interface provided by the store server 2 to be used.

[0051] The monitoring device 5 includes, in the auxiliary storage device 53, a setting storage unit 531, a camera image temporary storage unit 532, a tracking table 533, and a linking table 534, for example.

[0052] The setting memory unit 531 stores the terminal number assigned to each transaction processing device 1. Furthermore, the setting memory unit 531 stores a predetermined position for detecting a customer SC who attempts to leave the store without paying. This predetermined position can be set in advance by the administrator of the monitoring device 5, for example, using the input / output device 55. For example, the predetermined position can be set to be outside a rectangular predetermined range RR, as shown by the dashed line in FIG. 2 , which is predetermined for the checkout area CA. The shape of the predetermined range RR can be set arbitrarily depending on the layout of the checkout area CA. Furthermore, the predetermined position may be set to a position on the entrance ENT or exit EXT side of a predetermined line arbitrarily set for the checkout area CA. In other words, the predetermined position can be a position beyond the predetermined line on the entrance ENT or exit EXT side. The predetermined line may be, for example, one side of the predetermined range RR on the entrance ENT or exit EXT side. Furthermore, setting storage unit 531 stores installation location information that is uniquely determined based on the arrangement of each transaction processing device 1 in checkout area CA, such as the position of patrol lamp 111 of each transaction processing device 1, and the position of the operation range where customer SC operating each transaction processing device 1 should be located. Furthermore, setting storage unit 531 can also store installation location information of attendant terminal 3.

[0053] The camera image temporary storage unit 532 temporarily stores camera images taken by the checkout area camera 4 for a certain period of time.

[0054] The tracking table 533 stores the location of each customer SC who entered the checkout area CA. FIG. 8 is a schematic diagram showing the main data structure stored in this tracking table 533. Note that the data structure shown in FIG. 8 is an example. The format is not particularly limited as long as the necessary information for the processor 51 can be obtained. As shown by the dashed-dotted lines in FIG. 2, the processor 51 sets rectangular ROIs (Regions of Interest) as regions of interest from which images of each customer SC who entered the checkout area CA are cut out for the camera images captured by the checkout area camera 4, and assigns ROI numbers to distinguish each ROI. Therefore, the ROI numbers can also be considered customer numbers. As shown in FIG. 8, the tracking table 533 stores, for example, the coordinates in the camera images captured by the checkout area camera 4 as the location of the customer SC for each ROI number of each customer SC who entered the checkout area CA. Although the coordinates are not limited to this, in the example of FIG. 8, the coordinates are the coordinates of two diagonal points of the rectangular ROI.

[0055] Correlation table 534 stores the correspondence between customer SC and transaction processing device 1. FIG. 9 is a schematic diagram showing the main data structure stored in this correlation table 534. Note that the data structure shown in FIG. 9 is one example. As long as the necessary information for processor 51 can be obtained, the format is not particularly limited. For example, as shown in FIG. 9, correlation table 534 stores the terminal number assigned to transaction processing device 1 for each ROI number that identifies the customer SC. Note that there are cases where the same terminal number is stored for multiple ROI numbers. This is the case, for example, when a married couple is shopping together.

[0056] Next, each unit implemented in the processor 51 of the monitoring device 5 will be described. As shown in Fig. 7, the processor 51 implements, for example, a tracking unit 511, a linking unit 512, a terminal status detection unit 513, and a fraud determination unit 514. Each unit implemented in the processor 51 can also be referred to as each function. Each unit implemented in the processor 51 can also be referred to as being implemented in a control unit including the processor 51 and the main memory 52.

[0057] Tracking unit 511 receives camera images from checkout area camera 4 via communication interface 54 and stores the received camera images in camera image temporary storage unit 532. The camera images include images of customers moving within checkout area CA as well as images of patrol lights 111 attached to each transaction processing device 1. Tracking unit 511 stores camera images for a period of time that enables determination of whether patrol lights 111 are flashing or lit in camera image temporary storage unit 532. In this manner, tracking unit 511 can acquire image information that enables identification of the status of each transaction processing device 1 where customer SC performs settlement-related operations. Tracking unit 511 also tracks the movement of each customer SC in checkout area CA from the camera images for a certain period of time stored in camera image temporary storage unit 532 and stores the current location information of each customer SC in tracking table 533. Thus, tracking unit 511 is an example of a tracking unit that tracks the movement of customers SC.

[0058] Linking unit 512 identifies customer SCs within the operation range of each transaction processing device 1 based on the location information of each customer SC stored in tracking table 533 and the installation location information of each transaction processing device 1 stored in setting memory unit 531. Then, linking unit 512 registers the customer SCs operating each identified transaction processing device 1 in linking table 534. Linking unit 512 is an example of a linking unit that links a customer SC with a transaction processing device 1 on which the customer SC performs settlement-related operations based on the location of the customer SC being tracked.

[0059] Terminal status detection unit 513 detects the status of each transaction processing device 1 based on a certain period of camera footage stored in camera image temporary storage unit 532. Specifically, the camera footage includes images of patrol lights 111 possessed by each transaction processing device 1. In other words, terminal status detection unit 513 acquires from camera image temporary storage unit 532 image information capable of identifying the status of each transaction processing device 1 on which the customer SC performs settlement-related operations. Terminal status detection unit 513 determines the status information of each transaction processing device 1 represented by the color pattern and light emission pattern of patrol lights 111 from the images of patrol lights 111 of each transaction processing device 1 captured in the acquired camera footage for the certain period of time. Based on the determined status information, terminal status detection unit 513 determines whether the transaction processing device 1 is in use or idle. As described above, idle indicates that the previous customer SC has completed payment, i.e., that the settlement-related operations have been completed. In this way, the terminal status detection unit 513 is an example of a detection unit that detects whether or not the settlement-related operations have been completed for each of the multiple transaction processing devices 1 based on image information that can identify the status of each of the multiple transaction processing devices 1 on which the customer SC performs settlement-related operations.

[0060] Fraud determination unit 514 determines whether or not a payment omission fraud has occurred based on the location information of each customer SC stored in tracking table 533, the linking information between each customer SC and transaction processing device 1 stored in linking table 534, and the status of each transaction processing device 1 detected by terminal status detection unit 513. Specifically, fraud determination unit 514 determines that no fraud has occurred if the status of transaction processing device 1 linked to a customer SC located at a specified location, i.e., a customer SC who has moved outside specified range RR or crossed a specified line near entrance ENT or exit EXT, is idle, i.e., the operation related to settlement has been completed. In contrast, fraud determination unit 514 determines that a payment omission fraud has occurred if the status of transaction processing device 1 linked to a customer SC located at a specified location is in operation, i.e., the operation related to settlement has not been completed. When fraud determination unit 514 determines that a payment omission fraud has occurred, fraud determination unit 514 sends a payment omission notification to attendant terminal 3 via communication interface 54. Fraud determination unit 514 is an example of a notification unit that notifies of an incomplete payment when transaction processing device 1 associated with customer SC that has moved to a predetermined specified position is not in a state where the settlement-related operations have been completed. In other words, an incomplete payment can be referred to as an incomplete transaction, an unsettled transaction, an incomplete settlement, an unsettled payment, an incomplete settlement, an unsettled payment, etc.

[0061] The monitoring device 5 can be realized, for example, by using a general-purpose computer device as hardware and writing a control program to the main memory 52 or auxiliary storage device 53. The control program may be stored in the main memory 52 or auxiliary storage device 53 when the monitoring device 5 is transferred, or may be transferred separately from the general-purpose computer device. In the latter case, the control program is transferred by recording it on a removable recording medium such as a magnetic disk, magneto-optical disk, optical disk, or semiconductor memory, or via a network.

[0062] The operation of the monitoring device 5 and information processing system having such a configuration will be described below.

[0063] 10 is a sequence diagram illustrating a fraud detection operation in a self-checkout system to which the information processing system according to the first embodiment is applied. Note that FIG. 10 shows only one of a plurality of transaction processing devices 1 as a representative.

[0064] As shown in FIG. 10, the monitoring device 5 acquires camera images from the checkout area camera 4 (step S1).

[0065] The monitoring device 5 then tracks each customer SC in the checkout area CA using the camera images (step S2). The monitoring device 5 updates the current location of each customer SC stored in the tracking table 533 using the current location of each tracked customer SC. Furthermore, when an untracked customer SC, i.e., a new customer SC, enters the checkout area CA, the monitoring device 5 assigns a new ROI number to the customer SC and registers its current location in the tracking table 533.

[0066] Furthermore, monitoring device 5 updates the linkage between customer SC and transaction processing device 1 based on the positional relationship between the current location of each customer SC and each transaction processing device 1 (step S3). That is, if a customer SC that is not linked enters the operation range of any transaction processing device 1, monitoring device 5 registers the linkage between that customer SC and transaction processing device 1 in linkage tracking table 533.

[0067] Furthermore, monitoring device 5 detects the status of each transaction processing device 1 based on an image of patrol lamp 111 of each transaction processing device 1 (step S4).

[0068] Monitoring device 5 then determines whether or not there is any fraudulent payment omission (step S5). Specifically, for each transaction processing device 1 detected as being in operation, i.e., where the settlement operation has not yet been completed, monitoring device 5 checks the customer SC associated with that transaction processing device 1 using linking tracking table 533 and obtains the current location of that customer SC from tracking table 533. The presence or absence of fraud is determined based on whether the obtained current location of customer SC is a specified location, i.e., whether it is outside specified range RR or exceeds a specified line. For example, if transaction processing devices 1 with terminal numbers R2, R3, R7, and R8 are in operation, the locations of the customers SC associated with those transaction processing devices 1 are confirmed. At this time, if the situation is as shown in FIG. 2, the customers SC associated with those transaction processing devices 1 are not in the specified location, i.e., they are within specified range RR or have not exceeded the specified line, and therefore it is determined that no fraud has occurred.

[0069] The operations of steps S1 through S5 are repeated thereafter. While FIG. 10 shows that the operations of steps S1 through S3 are followed by the operations of steps S4 and S5, the operations of steps S4 and S5 may be performed each time the operations of steps S1 through S3 are performed multiple times. In other words, simply because customer SC leaves the operational range of each transaction processing device 1 does not mean that he or she immediately leaves the specified range RR or crosses the specified line. Therefore, there is no need for a one-to-one relationship between the operations of steps S1 through S3 and the operations of steps S4 and S5.

[0070] Furthermore, for example, if transaction processing device 1 with terminal number R1 is in operation in addition to transaction processing devices 1 with terminal numbers R2, R3, R7, and R8 described above, in the situation shown in FIG. 2, customer SC associated with transaction processing device 1 with terminal number R1 is in a specified location, that is, outside specified range RR or beyond the specified line, and monitoring device 5 determines in step S5 that a payment omission fraud has occurred. In such a case, monitoring device 5 sends an alert notification to attendant terminal 3 (step S6). The alert notification includes information indicating the transaction processing device 1 where the payment omission occurred, in this example, terminal number R1. The alert notification may also include an image of customer SC in the specified location, extracted from camera footage.

[0071] When an alert notification is sent from monitoring device 5, attendant terminal 3 displays the alert notification, for example, as a pop-up (step S7). After confirming this alert notification, store clerk SP performs a predetermined stop operation on attendant terminal 3 to instruct the stop of the alert notification (step S8), and then takes necessary measures, such as giving a warning to the corresponding customer SC. In response to this predetermined stop operation, attendant terminal 3 sends a terminal status update instruction to the corresponding transaction processing device 1 (step S9).

[0072] Transaction processing device 1 that has received this terminal status update instruction updates its status to "idle" and changes patrol lamp 111 to indicate "idle" (step S10).

[0073] The operation of the monitoring device 5 for realizing such an operation will be described below. Fig. 11 is a flowchart showing an example of the main steps of the tracking process executed by the processor 51 of the monitoring device 5. In this specification, unless otherwise specified, the processing operation of the processor 51 transitions from ACTn (n is a natural number) to ACT(n+1). The procedure shown in Fig. 10 is an example. There are no particular limitations on the procedure as long as similar results can be obtained.

[0074] As ACT101, the tracking unit 511 implemented in the processor 51 acquires camera images transmitted from the checkout area camera 4 via the communication network NW through the communication interface 54. The tracking unit 511 stores the acquired camera images in the camera image temporary storage unit 532.

[0075] In ACT102, the tracking unit 511 detects people in the checkout area CA from the temporarily stored camera image using known motion detection technology and human detection technology.

[0076] In ACT103, the tracking unit 511 determines whether or not a new person has been detected. Specifically, the tracking unit 511 compares the position information of each detected person with the position information stored in the tracking table 533, and determines whether or not there is a person whose position information is not within a distance, i.e., within the number of pixels, determined based on the detection time interval. The range within the checkout area CA where this determination is made may also be limited to the vicinity of the entrance ENT. If no new person has been detected, the tracking unit 511 determines NO in ACT103 and proceeds to the processing of ACT105. If a new person has been detected, the tracking unit 511 determines YES in ACT103 and proceeds to the processing of ACT104.

[0077] In ACT 104, the tracking unit 511 adds the newly detected person to the tracking targets. Specifically, the tracking unit 511 assigns an ROI number that is not stored in the tracking table 533 to the newly detected person, that is, the newly entering customer SC, and registers the ROI number in the tracking table 533.

[0078] In ACT105, the tracking unit 511 updates the location information of each customer SC stored in the tracking table 533 based on the current location of each detected person.

[0079] In ACT106, linking unit 512 implemented in processor 51 determines whether or not a new intruder has entered the operation range of each transaction processing device 1. Specifically, for each transaction processing device 1 whose terminal number is not stored in linking table 534, linking unit 512 determines whether or not a customer SC has entered the operation range stored in setting storage unit 531, based on the location information stored in tracking table 533. If there is no new intruder, linking unit 512 determines NO in ACT106 and proceeds to processing in ACT101. If there is a new intruder, linking unit 512 determines YES in ACT106 and proceeds to processing in ACT107.

[0080] In ACT 107, linking unit 512 registers a new link. Specifically, processor 51 stores in linking table 534 the terminal number of transaction processing device 1 determined to have a new intruder, in association with the ROI number of customer SC determined to be the new intruder. Thereafter, linking unit 512 proceeds to the processing of ACT 101.

[0081] Figure 12 is a flow chart showing an example of the main steps of the payment omission detection process executed by the processor 51 of the monitoring device 5. The processor 51 executes this process at regular intervals, for example, every few seconds, based on a control program stored in the main memory 52 or the auxiliary storage device 53. The steps shown in Figure 12 are an example. There are no particular limitations on the steps as long as similar results can be obtained.

[0082] As ACT201, terminal status detection unit 513 implemented in processor 51 initializes the value of an internal counter n (not shown) configured in processor 51 to "1." The value of this internal counter n is used to specify the terminal number of transaction processing device 1.

[0083] As ACT202, the terminal status detection unit 513 acquires an image of the patrol lamp 111 of the transaction processing device 1 with terminal number n from a certain period of camera footage stored in the camera image temporary storage unit 532, and detects the terminal status of the transaction processing device 1 based on the status information of the transaction processing device 1 represented by the color pattern and light emission pattern of the patrol lamp 111.

[0084] In ACT 203, the terminal status detection unit 513 determines whether the detected terminal status is in operation. If the terminal status is not in operation, i.e., is idle, the terminal status detection unit 513 determines NO in ACT 203 and proceeds to the processing of ACT 209. If the terminal status is in operation, the terminal status detection unit 513 determines YES in ACT 203 and proceeds to the processing of ACT 204.

[0085] As ACT204, fraud determination unit 514 implemented in processor 51 checks, using linking table 534, the ROI number, ie, the customer number, linked to transaction processing device 1 having terminal number n.

[0086] In ACT 205, the fraud determination unit 514 confirms the location of the customer SC of the confirmed ROI number. Specifically, the fraud determination unit 514 reads out location information corresponding to the confirmed ROI number from the tracking table 533.

[0087] In ACT206, the fraud determination unit 514 determines whether the customer SC has moved to a specified position stored in the setting storage unit 531. Specifically, the fraud determination unit 514 determines whether the current position of the customer SC indicated by the confirmed position information is outside a specified range RR stored in the setting storage unit 531. It may also be configured to determine whether the current position of the customer SC indicated by the position information exceeds a specified line. If the customer SC has moved to the specified position, the fraud determination unit 514 determines YES in ACT206 and proceeds to the processing of ACT215. If the customer SC has not moved to the specified position, the fraud determination unit 514 determines NO in ACT206 and proceeds to the processing of ACT207.

[0088] In ACT207, fraud determination unit 514 determines whether the value of internal counter n is "8", that is, whether the situation has been detected for all transaction processing devices 1 included in the self-checkout system. If the value of internal counter n is "8", fraud determination unit 514 determines YES in ACT207 and ends this processing. If the value of internal counter n is not "8", fraud determination unit 514 determines NO in ACT207 and proceeds to processing in ACT208.

[0089] In ACT208, the terminal status detection unit 513 increments the value of the internal counter n by 1. After that, the terminal status detection unit 513 proceeds to the processing of ACT202.

[0090] In ACT209, terminal status detection unit 513 determines whether or not there is an association in association table 534 for transaction processing device 1 with terminal number n, whose terminal status has been determined to be idle. Specifically, terminal status detection unit 513 determines whether or not terminal number n is stored in association table 534. If there is no association, terminal status detection unit 513 determines NO in ACT209 and proceeds to the processing of ACT207. If there is an association, terminal status detection unit 513 determines YES in ACT209 and proceeds to the processing of ACT210.

[0091] In ACT 210, terminal status detection unit 513 checks, using linking table 534, the ROI number, ie, the customer number, linked to transaction processing device 1 having terminal number n.

[0092] In ACT 211, the terminal status detection unit 513 confirms the location of the customer SC of the confirmed ROI number. Specifically, the terminal status detection unit 513 reads out the location information corresponding to the confirmed ROI number from the tracking table 533.

[0093] In ACT212, the terminal status detection unit 513 determines whether the customer SC has moved to a specified position stored in the setting storage unit 531. Specifically, the terminal status detection unit 513 determines whether the current position of the customer SC indicated by the confirmed position information is outside the specified range RR stored in the setting storage unit 531. It may also be configured to determine whether the position of the customer SC exceeds a specified line. If the customer SC has not moved to the specified position, the terminal status detection unit 513 determines NO in ACT212 and proceeds to the processing of ACT207. If the customer SC has moved to the specified position, the terminal status detection unit 513 determines YES in ACT212 and proceeds to the processing of ACT213.

[0094] In ACT 213 , the terminal status detection unit 513 deletes the confirmed ROI number and the corresponding location information from the tracking table 533 .

[0095] In ACT214, the terminal status detection unit 513 deletes the terminal number n stored in the association table 534. After that, the terminal status detection unit 513 proceeds to the processing of ACT207.

[0096] For example, suppose the status of transaction processing device 1 with terminal number R1 detected in the processing of ACT203 is idle, and terminal number R1 is linked to customer SC with ROI number ROI1, as shown in FIG. 9. This corresponds to a situation in which customer SC with ROI number ROI1 has completed a settlement operation at transaction processing device 1 with terminal number R1 and is heading toward the exit EXT of checkout area CA, as shown in FIG. 2. In such a case, the processing proceeds from ACT203 to ACT209, and from ACT209, via ACT210 and ACT211, it is confirmed in ACT212 that customer SC is in a specified location. Then, the processing of ACT213 and ACT214 deletes the record for ROI number ROI1 that stores the location information of customer SC from tracking table 533, and also deletes the link between ROI number ROI1 and terminal number R1, i.e., the corresponding record, from linking table 534.

[0097] In ACT215, the fraud determination unit 514 implemented in the processor 51 transmits an alert notification to the attendant terminal 3 via the communication interface 54. Thereafter, the fraud determination unit 514 proceeds to the processing of ACT207.

[0098] For example, suppose the status of transaction processing device 1 with terminal number R1 detected in the processing of ACT203 is in operation, and terminal number R1 is linked to customer SC with ROI number ROI1, as shown in Figure 9. This corresponds to a situation in which customer SC with ROI number ROI1 has left the specified range RR and is heading toward the exit EXT of checkout area CA, but has not yet completed the operation related to settlement at transaction processing device 1 with terminal number R1, i.e., a payment has been missed, as shown in Figure 2. In such a case, the processing proceeds from ACT203 to ACT206 via ACT204 and ACT205, and when it is confirmed that customer SC has left the specified range RR, an alert notification is sent in the processing of ACT215.

[0099] FIG. 13 shows an example of monitoring screen 31 displayed on the touch panel of attendant terminal 3 when attendant terminal 3 is connected to communication network NW and has the function of monitoring multiple transaction processing devices 1. Monitoring screen 31 includes terminal list area 32 displaying a list of multiple transaction processing devices 1 to be monitored as icons, and terminal status area 33 displaying the display content of touch panel 104 of each transaction processing device 1. Attendant terminal 3 receives status information transmitted from each transaction processing device 1 indicating the status of that transaction processing device 1 and the display information displayed on touch panel 104 of that transaction processing device 1. Based on the received status information, attendant terminal 3 changes the color, density, or shape of the icon displayed in terminal list area 32 to distinguishably display the current status of each transaction processing device 1. In the example of FIG. 13, transaction processing devices 1 with terminal numbers R1, R2, R3, R7, and R8 are displayed with icons indicating that they are in operation, and transaction processing devices 1 with terminal numbers R4, R5, and R6 are displayed with icons indicating that they are in idle status. Furthermore, attendant terminal 3 displays in terminal status area 33 a reduced image of the screen of each touch panel 104 based on the received display information displayed on touch panel 104 of each transaction processing device 1 .

[0100] FIG. 14 shows an example of monitoring screen 31 upon receiving an alert notification transmitted from monitoring device 5. As described above, the alert notification includes information indicating the transaction processing device 1 where the payment omission occurred and may include an image of the corresponding customer SC extracted from the camera video. Upon receiving such an alert notification, attendant terminal 3 superimposes a pop-up alert 34 based on the alert notification on the display content of each transaction processing device 1 displayed in terminal status area 33 of monitoring screen 31, as shown in FIG. 14. Pop-up alert 34 may include a customer image 35 along with a message indicating which transaction processing device 1 the payment omission occurred at. Furthermore, pop-up alert 34 includes an "OK" button 36 that allows attendant store clerk SP to input that he or she has confirmed the occurrence of the payment omission. It goes without saying that the pop-up alert 34 shown in FIG. 14 is merely an example. Furthermore, upon receiving the alert notification, attendant terminal 3 changes the icon of the corresponding terminal number displayed in terminal list area 32—in this example, the icon of the transaction processing device 1 with terminal number R1—to an icon that clearly indicates the occurrence of a payment omission, as shown in FIG. 14.

[0101] When "OK" button 36 in pop-up alert 34 is operated, attendant terminal 3 sends a terminal status update instruction to transaction processing device 1 that has been determined to have experienced a missed payment based on the alert notification. In other words, operating "OK" button 36 is an example of a predetermined stop operation that instructs the cessation of alert notifications. As described above, transaction processing device 1 that receives this terminal status update instruction, for example, transaction processing device 1 with terminal number R1 in the above example, updates its status to idle and changes patrol lamp 111 to indicate idle.

[0102] If attendant terminal 3 is a general-purpose personal computer or tablet terminal that does not have the function of monitoring transaction processing device 1, the same display as pop-up alert 34 will be displayed on its display screen. In this case, "OK" button 36 is a button that instructs the display of pop-up alert 34 to end. To actually update the status of transaction processing device 1 and change the display of patrol lamp 111, attendant store clerk SP will go to the location where transaction processing device 1 is installed and perform the specified update operation.

[0103] Of course, attendant terminal 3 does not have the function of monitoring transaction processing device 1, but may have the function of connecting to communication network NW and updating the status of transaction processing device 1.

[0104] By updating the status of transaction processing device 1 in this way, in the next payment failure detection process executed by monitoring device 5 after a certain period of time has elapsed, the status of transaction processing device 1 with terminal number R1 detected in ACT202 will be idle, so monitoring device 5 will proceed from processing in ACT203 to processing in ACT209. Then, from ACT209, via processing in ACT210 and ACT211, processing in ACT212 confirms that the customer SC is in the specified location. Then, processing in ACT213 and ACT214 deletes the record for ROI number ROI1 that stores the location information of the customer SC from tracking table 533, and also deletes the link between ROI number ROI1 and terminal number R1 from linking table 534, i.e., the corresponding record.

[0105] As described above, monitoring device 5, to which the information processing device according to the first embodiment is applied, uses tracking unit 511 to detect whether the settlement-related operations have been completed for each of multiple transaction processing devices 1, based on image information that identifies the status of each of multiple transaction processing devices 1 on which customer SC performs settlement-related operations. Monitoring device 5 also uses tracking unit 511 to track the movement of customer SC, and uses linking unit 512 to link customer SC to transaction processing devices 1 on which customer SC performs settlement-related operations, based on the location of the tracked customer SC. Monitoring device 5 then uses fraud determination unit 514 to notify of an incomplete payment if the status of transaction processing device 1 linked to customer SC that has moved to a predetermined location is not one in which the settlement-related operations have been completed. In this way, according to the monitoring device 5 to which the information processing device of the first embodiment is applied, image information is used to detect whether the transaction processing device 1 has completed operations related to settlement, thereby eliminating the need to communicate with the transaction processing device 1, and when a customer SC moves to a specified position, if the transaction processing device 1 linked to the customer SC has not completed operations related to settlement, a notification of missed payment is sent, making it possible to easily notify of missed payment. To combat payment omissions and fraud, gate systems have emerged that install gates at the exits of checkout areas and open them by holding a receipt issued by a checkout terminal over the gate upon payment completion. However, such gate systems are expensive and require users to hold a receipt over the gate, resulting in a poor user experience. To address this issue, a notification system has been proposed that communicates with checkout terminals to obtain their status information, detects the customer's location, and notifies store staff if the customer exits the area where the checkout terminal is installed before completing the transaction. However, connecting to a self-checkout system installed in a store and obtaining status information from the checkout terminal requires knowledge of the information exchanged with the self-checkout system. Therefore, if knowledge of the checkout system installed in the store is unavailable or unavailable, the notification system cannot be implemented. In contrast, the monitoring device 5 incorporating the information processing device according to the first embodiment does not need to communicate with the transaction processing device 1, making it applicable to stores even if knowledge of the checkout system installed in the store is unavailable or unavailable.

[0106] Here, in the first embodiment, when a customer SC associated with a transaction processing device 1 that has been detected as not having completed settlement operations moves to a specified position, the monitoring device 5 notifies the customer of an incomplete payment via the fraud determination unit 514. Therefore, according to monitoring device 5 in the first embodiment, the amount of processing can be reduced by determining whether a payment has been omitted only for customers SC linked to transaction processing device 1, rather than for all customers SC who enter checkout area CA. Also, because a payment omission determination is not performed for customers SC or store clerk SP who are not linked to transaction processing device 1, the occurrence of erroneous notifications can be reduced. Normally, store clerk SP, who is an attendant, pays particular attention to customers SC who enter checkout area CA and try to leave without operating transaction processing device 1, so the probability of a problem occurring is low even without monitoring by monitoring device 5.

[0107] In addition, in the first embodiment, the monitoring device 5 uses the terminal status detection unit 513 to detect whether the transaction processing device 1 is in a state where operations related to settlement have been completed, based on an image of the patrol lamp 111 included in image information obtained by photographing the patrol lamp 111, which is provided on the transaction processing device 1 and indicates whether operations related to settlement have been completed on the transaction processing device 1. Therefore, according to the monitoring device 5 in the first embodiment, by determining the status of the transaction processing device 1 displayed by the patrol lamp 111, it is possible to detect the status of the transaction processing device 1 without communicating with the transaction processing device 1.

[0108] In addition, the information processing system of the first embodiment applied to a self-checkout system includes an attendant terminal 3 used by a store clerk, a checkout area camera 4 that acquires image information that allows identification of the status of each of the multiple transaction processing devices 1 on which the customer SC performs operations related to settlement, and a monitoring device 5 in the first embodiment that communicates with the attendant terminal 3 and the checkout area camera 4.The monitoring device 5 uses a terminal status detection unit 513 to detect whether or not the operations related to settlement have been completed for each of the multiple transaction processing devices 1 based on the image information acquired by the checkout area camera 4, and if the status of the transaction processing device 1 linked to the customer SC that has moved to a predetermined specified position is not in a state where the operations related to settlement have been completed, the fraud determination unit 514 notifies the attendant terminal 3 of an incomplete payment. In this way, according to the information processing system of the first embodiment, image information is used to detect whether the transaction processing device 1 has completed operations related to settlement, eliminating the need to communicate with the transaction processing device 1.When a customer SC moves to a specified position, if the transaction processing device 1 linked to that customer SC has not completed operations related to settlement, a notification of missed payment is sent, making it possible to easily notify the customer of missed payment.

[0109] Furthermore, in the information processing system according to the first embodiment, the monitoring device 5 tracks the movement of the customer SC using the tracking unit 511 based on the camera images captured by the checkout area camera 4. Therefore, there is no need to prepare a camera separate from the checkout area camera 4 for tracking the movement of the customer SC, and an information processing system can be constructed at low cost.

[0110] [Second embodiment] Next, a second embodiment will be described. Note that the same reference numerals as in the first embodiment are used to denote the same configurations and operations as in the first embodiment, and descriptions thereof will be omitted.

[0111] Figure 15 is a flowchart showing an example of the main steps of the payment omission detection process executed by the processor 51 of the monitoring device 5 to which the information processing device according to the second embodiment is applied. The processor 51 executes this process at regular intervals, for example, every few seconds, based on a control program stored in the main memory 52 or the auxiliary storage device 53. The steps shown in Figure 15 are an example. There are no particular limitations on the steps as long as similar results can be obtained.

[0112] As ACT251, the terminal status detection unit 513 implemented in the processor 51 initializes the value of an internal counter n to “1.” In this embodiment, the value of this internal counter n is used to specify the ROI number.

[0113] In ACT 252, the terminal status detection unit 513 acquires the location information of the customer SC whose ROI number is n, that is, ROIn. Specifically, the terminal status detection unit 513 reads the location information of the customer SC of ROIn from the tracking table 533.

[0114] In ACT253, the terminal status detection unit 513 determines whether the customer SC has moved to a specified position stored in the setting storage unit 531. Specifically, the terminal status detection unit 513 determines whether the current position of the customer SC indicated by the acquired position information is outside the specified range RR stored in the setting storage unit 531. It may also be configured to determine whether the current position of the customer SC indicated by the position information exceeds a specified line. If the customer SC has moved to the specified position, the terminal status detection unit 513 determines YES in ACT253 and proceeds to the processing of ACT256. If the customer SC has not moved to the specified position, the terminal status detection unit 513 determines NO in ACT253 and proceeds to the processing of ACT254.

[0115] In ACT254, terminal status detection unit 513 determines whether the value of internal counter n is "8", that is, whether the status has been detected for all transaction processing devices 1 included in the self-checkout system. If the value of internal counter n is "8", terminal status detection unit 513 determines YES in ACT254 and ends this processing. If the value of internal counter n is not "8", terminal status detection unit 513 determines NO in ACT254 and proceeds to processing in ACT255.

[0116] In ACT255, the terminal status detection unit 513 increments the value of the internal counter n by 1. After that, the terminal status detection unit 513 proceeds to the processing of ACT252.

[0117] As ACT256, fraud determination unit 514 implemented in processor 51 checks, using linking table 534, the terminal number of transaction processing device 1 linked to customer SC of ROI number n.

[0118] As ACT257, the fraud determination unit 514 acquires an image of the patrol lamp 111 of the transaction processing device 1 corresponding to the confirmed terminal number from a certain period of camera footage stored in the temporary camera image storage unit 532, and detects the terminal status of the transaction processing device 1 based on the status information of the transaction processing device 1 represented by the color pattern and light emission pattern of the patrol lamp 111.

[0119] In ACT258, the fraud determination unit 514 determines whether the detected terminal status is in operation. If it is in operation, the fraud determination unit 514 determines YES in ACT258 and proceeds to the processing of ACT261. If it is not in operation, that is, if it is idle, the fraud determination unit 514 determines NO in ACT258 and proceeds to the processing of ACT259.

[0120] In ACT259, the fraud determination unit 514 deletes the ROI number n, that is, ROIn, and the corresponding position information from the tracking table 533. That is, the fraud determination unit 514 deletes the record of ROIn.

[0121] In ACT260, the fraud determination unit 514 deletes the ROI number n, i.e., ROIn, and the corresponding terminal number stored in the association table 534. That is, the fraud determination unit 514 deletes the record of ROIn. After that, the terminal status detection unit 513 proceeds to the processing of ACT254 above.

[0122] In ACT261, the fraud determination unit 514 transmits an alert notification to the attendant terminal 3 via the communication interface 54. Thereafter, the fraud determination unit 514 proceeds to the processing of ACT254.

[0123] As described above, the monitoring device 5 to which the information processing device of the second embodiment is applied detects the status of the transaction processing device 1 linked to the customer SC when the customer SC being tracked moves to a specified location using the terminal status detection unit 513, and notifies the customer of a missed payment using the fraud determination unit 514 when the transaction processing device 1 linked to the customer SC that has moved to this specified location has not completed operations related to settlement. In this way, monitoring device 5, to which the information processing device according to the second embodiment is applied, performs a payment omission determination only for transaction processing devices 1 associated with customers SC who have moved to specified positions, rather than for all transaction processing devices 1 installed in checkout area CA, thereby reducing the amount of processing. Furthermore, since payment omission determination is not performed for customers SC or store clerk SP who are not associated with transaction processing devices 1, the occurrence of erroneous notifications can be reduced. Normally, store clerk SP, who is an attendant, pays particular attention to customers SC who enter checkout area CA and try to leave without operating transaction processing devices 1, so the probability of a problem occurring is low even without monitoring by monitoring device 5.

[0124] [Third embodiment] Monitoring device 5, to which the information processing device according to the first and second embodiments is applied, determines the current status of each transaction processing device 1 based on the display state of patrol lamp 111. This embodiment is an example in which the object of determination used to determine the current status of each transaction processing device 1 is something other than patrol lamp 111. Specifically, this embodiment determines the display screen of touch panel 104 of each transaction processing device 1 as the object of determination. The following description will be given taking the case where transaction processing device 1 is self-service POS terminal 100 as an example, but it goes without saying that transaction processing device 1 may also be self-service checkout terminal 150.

[0125] 16 is a diagram showing an example of idle screen 1041 displayed on touch panel 104 of self-service POS terminal 100 as an example of transaction processing device 1 in the third embodiment. When self-service POS terminal 100 is available, a specified idle screen 1041 indicating this is displayed on touch panel 104. For example, as shown in FIG. 16, idle screen 1041 includes a display that shows customer SC the terminal number of self-service POS terminal 100.

[0126] 17 is a diagram showing an example of a transaction start screen 1042 displayed on the touch panel 104 of the self-service POS terminal 100. For example, when the customer SC touches the touch panel 104, the idle screen 1041 in FIG. 16 is switched to this transaction start screen 1042.

[0127] 18 is a diagram showing an example of a registration screen 1043 displayed on the touch panel 104 of the self-checkout POS terminal 100. The registration screen 1043 displays a list of registered products, prices, etc.

[0128] 19 is a diagram showing an example of a payment selection screen 1044 displayed on the touch panel 104 of the self-service POS terminal 100. When the "Proceed to checkout" button 10431 included in the registration screen 1043 shown in FIG. 18 is touched, the screen switches to this payment selection screen 1044.

[0129] 20 is a diagram showing an example of transaction completion screen 1045 displayed on touch panel 104 of self-service POS terminal 100. When settlement at transaction processing device 1 is completed, transaction completion screen 1045 is displayed. After transaction completion screen 1045 is displayed for a certain period of time, the screen returns to idle screen 1041 shown in FIG.

[0130] Therefore, in this embodiment, an image of the display screen of the touch panel 104 of each transaction processing device 1 is extracted from the camera image captured by the checkout area camera 4, and the current status of the transaction processing device 1 is determined based on the screen displayed.

[0131] Specifically, terminal status detection unit 513 of monitoring device 5 determines that transaction processing device 1 is in the idle state if touch panel 104 displays idle screen 1041. If the display on touch panel 104 switches from idle screen 1041 to transaction start screen 1042, terminal status detection unit 513 determines that transaction processing device 1 has entered the operating state. If touch panel 104 displays registration screen 1043, payment selection screen 1044, a settlement screen (not shown), or the like, terminal status detection unit 513 determines that transaction processing device 1 is in the operating state. Furthermore, if the display on touch panel 104 changes to transaction end screen 1045, terminal status detection unit 513 may determine that self-service POS terminal 100 has entered the idle state immediately without waiting for the screen to switch to idle screen 1041, or after a certain amount of time has passed.

[0132] Thus, touch panel 104 is an example of a display member provided in transaction processing device 1 that displays whether transaction processing device 1 has completed operations related to settlement. It can also be said that touch panel 104 is an example of a display member that displays, by at least one of letters and symbols, whether transaction processing device 1 is in an idle state or an operating state.

[0133] Specifically, in ACT202 and ACT257, the monitoring device 5 extracts the display screen portion of the touch panel 104 from the camera image, rather than the portion of the patrol lamp 111, and detects the status of the transaction processing device 1 based on the display status of that display screen.

[0134] In the first and second embodiments, a certain amount of video was required to determine whether the patrol lamp 111 was flashing or lit, but in this embodiment, a one-shot image may be sufficient.

[0135] As described above, the monitoring device 5 to which the information processing device of the third embodiment is applied detects, by the terminal status detection unit 513, whether the transaction processing device 1 is in a state where operations related to settlement have been completed, based on an image of the display screen of the touch panel 104 included in the image information obtained by capturing the display screen of the touch panel 104, which is provided in the transaction processing device 1 and indicates whether operations related to settlement have been completed. Therefore, according to monitoring device 5 to which the information processing device in the third embodiment is applied, the state of transaction processing device 1 can be determined by photographing touch panel 104 that displays the state of transaction processing device 1.

[0136] Touch panel 104 indicates the state of transaction processing device 1 by at least one of characters and symbols such as buttons. Therefore, according to monitoring device 5 in the third embodiment, the state of transaction processing device 1 can be easily determined simply by determining the characters and / or symbols displayed on the screen of touch panel 104.

[0137] [Fourth embodiment] In the third embodiment, the display screen of touch panel 104 of each transaction processing device 1 is the object of judgment, but in the present embodiment, the display screen of the touch panel provided in attendant terminal 3 is the object of judgment.

[0138] Fig. 21 is a schematic configuration diagram showing a self-checkout system to which an information processing system according to the fourth embodiment is applied, including a monitoring device 5 to which an information processing device according to the fourth embodiment is applied. Fig. 22 is a schematic diagram showing an example of a checkout area CA of a store in the fourth embodiment.

[0139] In the present embodiment, attendant terminal 3 is connected to communication network NW and has the function of monitoring multiple transaction processing devices 1. As shown in Figures 21 and 22, the information processing system according to the fourth embodiment includes, in addition to checkout area camera 4, attendant screen camera 6 installed so that its imaging range 61 corresponds to the display screen of the touch panel of attendant terminal 3. Attendant screen camera 6 transmits captured images to monitoring device 5.

[0140] 13 is displayed on the display screen of the touch panel of attendant terminal 3, which has the function of monitoring transaction processing devices 1. As described above, attendant terminal 3 changes the color, density, or shape of the icon displayed in terminal list area 32 of monitoring screen 31 based on the status information received from each transaction processing device 1, thereby displaying the current status of each transaction processing device 1 in an identifiable manner.

[0141] Therefore, in this embodiment, attendant screen camera 6 captures an image of monitoring screen 31 displayed on touch panel 104 of attendant terminal 3. Then, terminal status detection unit 513 of monitoring device 5 extracts an image showing the status of transaction processing device 1 to be detected in terminal list area 32 from the image captured by attendant screen camera 6, and determines the current status of transaction processing device 1 to be detected based on the type of image displayed. By fixing the positional relationship between attendant screen camera 6 and monitoring screen 31 of attendant terminal 3, it is possible to know which pixel in the captured image corresponds to which icon of transaction processing device 1. If the positional relationship is not fixed, monitoring device 5 can detect which pixel in the captured image corresponds to which icon of transaction processing device 1 by determining terminal list area 32 using a known image recognition process. Of course, the current status of the transaction processing device 1 may be determined using an image in the terminal status area 33 instead of an image in the terminal list area 32, or the accuracy of the determination may be improved by using both an image in the terminal list area 32 and an image in the terminal status area 33.

[0142] In this way, attendant terminal 3 is an example of an attendant terminal that communicates with each transaction processing device 1 and displays the status of each transaction processing device 1, and the touch panel of the attendant terminal that displays monitoring screen 31 is an example of a display screen that displays the status of multiple transaction processing devices 1. Attendant screen camera 6 is an example of an acquisition device that acquires image information that can identify the status of each of multiple transaction processing devices 1.

[0143] Specifically, in ACT202 and ACT257, the monitoring device 5 extracts an image portion, such as an icon related to the transaction processing device 1 to be detected, from the image captured by the attendant screen camera 6 on the monitoring screen 31 displayed on the touch panel of the attendant terminal 3, and detects the status of the transaction processing device 1 based on the display content of that image portion.

[0144] As described above, the monitoring device 5 to which the information processing device of the fourth embodiment is applied detects, by the terminal status detection unit 513, whether the transaction processing device 1 is in a state in which operations related to settlement have been completed, based on an image of the status display for multiple transaction processing devices 1 contained in image information captured from the monitoring screen 31 displaying the status of multiple transaction processing devices 1 on the attendant terminal 3, which communicates with each transaction processing device 1 and displays the status of each transaction processing device 1. Therefore, according to the monitoring device 5 to which the information processing device relating to the information processing device of the fourth embodiment is applied, the status of the transaction processing device 1 can be easily determined by photographing the monitoring screen 31 of the attendant terminal 3 which displays the status of the transaction processing device 1.

[0145] [Other embodiments] Although the embodiment of the information processing device has been described above, the embodiment is not limited to this.

[0146] For example, in the first to third embodiments, images of the patrol lamp 111 and touch panel 104 of each transaction processing device 1 are extracted from the camera image of the checkout area camera 4, but each transaction processing device 1 may also be equipped with a camera that captures images of the patrol lamp 111 and touch panel 104.

[0147] Furthermore, the conditions under which each transaction processing device 1, based on the camera image from the checkout area camera 4, detects that the settlement-related operations have not been completed may not be limited to the conditions described in the embodiment, but may be set arbitrarily by the administrator of the monitoring device 5 as an information processing device.

[0148] Furthermore, the information processing device is not limited to the monitoring device 5, and its functions may be provided in a computer constituting the attendant terminal 3 or the store server 2. Furthermore, by connecting a device for connecting to the Internet to the communication network NW, a server computer provided on the cloud may be used as the information processing device.

[0149] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0150] 1...transaction processing device, 2...store server, 3...attendant terminal, 4...checkout area camera, 5...monitoring device, 6...attendant screen camera, 31...monitoring screen, 32...terminal list area, 33...terminal status area, 34...pop-up alert, 35...customer image, 36..."OK" button, 51...processor, 511...tracking unit, 512...linking unit, 513...terminal status detection unit, 514...fraud determination unit, 52...main memory, 53...auxiliary storage device, 531...settings storage unit, 532...camera image temporary storage unit, 533...tracking table, 534...linking table, 54...communication interface, 55...input / output device, 56...system transmission path, 100...self-service POS terminal, 104...touch panel, 1041...idle screen, 1042...transaction start screen, 1043...registration screen, 10431..."Proceed to checkout" button, 1044...selection screen, 1045...transaction completion screen, 111...patrol lamp, 112...first light-emitting unit, 113...second light-emitting unit, CA...checkout area, ENT...entrance, EXT...exit, NW...communication network, RR...specified range, SC...customer, SP...store clerk.

Claims

1. a detection unit that detects whether the settlement-related operations have been completed for each of a plurality of transaction processing devices based on image information that identifies the status of each of the transaction processing devices on which a customer performs settlement-related operations; a tracking unit that tracks the movement of the customer; a linking unit that links the customer with the transaction processing device on which the customer performs the settlement operation based on the tracked location of the customer; a notification unit that notifies the customer of an incomplete payment when the transaction processing device associated with the customer that has moved to a predetermined specified position is not in a state where the operation related to the settlement has been completed; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the notification unit notifies the customer of the missed payment when the customer associated with the transaction processing device for which the settlement-related operation has been detected as not being completed moves to the specified location.

3. the detection unit detects the state of the transaction processing device associated with the customer when the customer being tracked moves to the specified position; The information processing device according to claim 1 , wherein the notification unit notifies the customer of the missed payment when the transaction processing device associated with the customer who has moved to the specified position has not completed operations related to the settlement.

4. An information processing device as described in any one of claims 1 to 3, wherein the detection unit detects whether the transaction processing device is in a state where operations related to the settlement have been completed based on an image of a display element included in the image information obtained by photographing the display element, which is provided on the transaction processing device and indicates whether operations related to the settlement have been completed on the transaction processing device.

5. An information processing device as described in any one of claims 1 to 3, wherein the detection unit detects whether the transaction processing device is in a state where operations related to the settlement have been completed based on an image of the status display for the multiple transaction processing devices contained in the image information captured from a display screen displaying the status of the multiple transaction processing devices at an attendant terminal that communicates with each of the transaction processing devices and displays the status of each of the transaction processing devices.

6. Attendant terminals used by store staff, an acquisition device that acquires image information that allows identification of the status of each of a plurality of transaction processing devices on which a customer performs operations related to settlement; an information processing device that communicates with the attendant terminal and the acquisition device; Equipped with The information processing device includes: a detection unit that detects whether the operation related to the settlement has been completed for each of the plurality of transaction processing devices based on the image information acquired by the acquisition device; a tracking unit that tracks the movement of the customer; a linking unit that links the customer with the transaction processing device on which the customer performs the settlement operation based on the tracked location of the customer; a notification unit that notifies the attendant terminal of an incomplete payment when the transaction processing device associated with the customer that has moved to a predetermined specified position is not in a state in which the operation related to the settlement has been completed; An information processing system comprising:

Images