Information processing system and program

Abstract

To properly check an operation status.SOLUTION: An information processing device that advances processing based on customer operations includes: imaging means for capturing images of customers; and display means capable of accepting touch operations and displaying information corresponding to the progress of processing. The display means displays both the captured image from the imaging means and a display transition image showing the transition of the displayed content on the same screen, synchronizing the timing of capture with the timing of display.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing device, and a program. [Background technology]

[0002] As a function for checking (inspecting) whether or not an unauthorized operation has been performed, a cash register is known that displays a video of the operation status captured by a camera (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-120635 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is room for improvement in the method of checking the operation status using the conventional technology as disclosed in Patent Document 1.

[0005] The present invention has been made in view of the above circumstances, and has an object to provide a technique for conveniently checking an operation status. [Means for solving the problem]

[0006] One aspect for solving the above-mentioned problem is an information processing device that progresses processing based on customer operations, and is equipped with an imaging means for imaging the customer and a display means that is capable of accepting touch operations and displays information according to the progress of processing, and is characterized in that the display means displays, on the same screen, an image captured by the imaging means and a display transition image showing the transition of the display content by the display means, with the time of imaging coinciding with the time of display. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is an example of the appearance of a registration and settlement device. [Figure 2] 1 is a configuration example of a registration and settlement device. [Figure 3] 10 is a display example of a registration settlement device. [Figure 4] 10 is a display example of a registration settlement device. [Figure 5] 10 is a display example of a registration settlement device. [Figure 6] 10 is a display example of a registration settlement device. [Figure 7] 10 is a display example of a registration settlement device. DETAILED DESCRIPTION OF THE INVENTION

[0008] (Sales System) The registration and settlement device 10 of this embodiment is one of the devices constituting a sales system 1 (not shown). The sales system 1 can be introduced into various business types (supermarkets, convenience stores, drug stores, home improvement stores, electronics retailers, etc.). In addition to the registration and settlement device 10, the sales system 1 also includes a transaction status management device 40 (not shown), a store controller 50 (not shown), and a headquarters server 60 (not shown). The registration and settlement device 10, the transaction status management device 40, and the store controller 50 are installed in a store. These are connected to each other via a LAN (which may be wired or wireless) so that they can communicate with each other. The headquarters server 60 is installed outside the store (for example, at a headquarters or data center). The devices installed in the store (the registration and settlement device 10, the transaction status management device 40, and the store controller 50) can communicate with a device installed outside the store (the headquarters server 60) (direct communication or communication via the store controller 50). The number of devices installed is not particularly limited. The headquarters server 60 may be a cloud server. The headquarters server 60 may be composed of two or more servers.

[0009] (Registration settlement device 10) The registration and settlement device 10 is a so-called fully self-service cash register that registers products based on customer operations and settles payments based on the customer operations. In other words, the registration and settlement device 10 executes a registration process for registering products and a settlement process for settling payments for registered products, but both processes are basically executed based on the customer's own operations. The registration and settlement device 10 can also execute an operation confirmation and verification process (described below) based on the operations of a store clerk.

[0010] (Transaction status management device 40) The transaction status management device 40 is a device that manages devices installed in the store (monitoring and controlling processing status, operating status, etc.). The transaction status management device 40 may also manage operators of devices installed in the store (monitoring operations performed by customers while using the device, and checking operations performed by customers after use). The transaction status management device 40 includes at least a CPU, a memory unit, a display unit (which may be configured as a touch panel), an operation unit (keyboard, mouse, etc.), and a communication unit. The transaction status management device 40 is used by store staff.

[0011] (Store Controller 50) The store controller 50 is a server that mainly handles communication of information between devices installed inside the store and devices installed outside the store. The store controller 50 includes at least a CPU, a memory unit, and a communication unit.

[0012] Note that devices other than those described above may also be installed within the store. For example, a registration and settlement device 210 (not shown) that executes registration processing and settlement processing based on the operation of a store clerk, a registration device 220 (not shown) that executes registration processing based on the operation of a store clerk, and a settlement device 230 (not shown) that acquires registration information from the registration device 220 and executes settlement processing based on the operation of a customer may be installed. Also, a settlement device 250 (not shown) that acquires registration information from a mobile registration terminal 240 (not shown) that executes registration processing based on the operation of a customer in the product sales area and executes settlement processing based on the operation of the customer may be installed. Also, a charging device 260 (not shown) that charges electronic money may be installed within the store.

[0013] FIG. 1 shows an example of the appearance of registration and settlement device 10. Registration and settlement device 10 comprises a main body 11, an upstream counter 12, and a downstream counter 13. Upstream counter 12 is a counter located upstream of main body 11 on the customer flow path. For example, products that have not yet been registered or shopping baskets containing products that have not yet been registered are placed on upstream counter 12. Downstream counter 13 is a counter located downstream of main body 11 on the customer flow path. For example, products that have been registered or shopping bags (reusable bags, paid plastic bags, etc.) containing products that have been registered are placed on downstream counter 13. Note that upstream counter 12 (as well as downstream counter 13) may be separate entities that are attached to registration and settlement device 10 (main body 11).

[0014] A cash payment unit (change dispenser) 106 is provided in the lower part of the main body 11. A printing unit 108, a scanner unit 104, a display unit 103, a non-cash payment unit (payment terminal) 105, an imaging unit (camera) 110, and a support pole unit 111 are provided in the upper part of the main body 11. A light emitting unit 112, a monitoring display unit 113, and a detection unit (sensor unit) 120 are provided on the support pole unit 111 in this order from the top.

[0015] FIG. 2 shows an example of the configuration of the registration and settlement device 10. Although some of the components have already been mentioned in FIG. 1, the registration and settlement device 10 includes a CPU 101, a memory unit 102, a display unit 103, a scanner unit 104, a non-cash payment unit 105, a cash payment unit 106, a communication unit 107, a printing unit 108, an audio output unit 109, an imaging unit 110, a light-emitting unit 112, a monitoring and display unit 113, and a detection unit 120. These components can communicate with each other via a bus. Although not shown, the registration and settlement device 10 also includes a timing unit (timing function). For example, the registration and settlement device 10 can use the timing unit to manage the current time, the elapsed time from a certain point in time, and the remaining time until a certain time. The registration and settlement device 10 may also include an audio input unit (microphone).

[0016] The CPU 101 is a central processing unit that controls the operation of the registration and settlement device 10 by reading and executing programs stored in the storage unit 102. For example, the CPU 101 controls the entire transaction from product registration to settlement. In other words, the CPU 101 executes (progresses) the registration process and settlement process.

[0017] (Generation of transaction information, electronic journal information, and operation log information) The CPU 101 generates various types of information. For example, the CPU 101 generates information (hereinafter referred to as "transaction information") indicating transaction details, including product information (product registration information) for products registered in each transaction and payment information (information including payment type). The CPU 101 also generates transaction history information (hereinafter referred to as "electronic journal information") that records events (hereinafter referred to as "events"; described below) that occurred in each transaction, and operation history information (hereinafter referred to as "operation log information") that records the history of operations (described below) in each transaction. The CPU 101 generates transaction information, electronic journal information, and operation log information without any explicit instruction (such as an operation from an operator).

[0018] (Events recorded in electronic journal information) Events stored in the electronic journal information include, for example, screen transitions (display / deletion), display / deletion of small screens, scan results (e.g., barcode reading OK, barcode reading NG, non-file (PLU failed)), product registration (product registration by scanning a barcode, product registration using a product preset button), product cancellation, increase / decrease in purchase quantity, selection of payment method (selected payment method), display / deletion of various messages (e.g., errors, alerts, guidance), occurrence / resolution of errors, continuation of a predetermined period of no operation with no operation performed on the display unit 103 or the scanner unit 104 (scanning operation), notification from the non-cash payment unit 105 (described below), notification from the cash payment unit 106 (described below), calling a store clerk, etc.

[0019] Notifications from the non-cash payment unit 105 include, for example, notifications that various waiting states (e.g., waiting for a credit card or the like to be read, waiting for a PIN (Personal Identification Number) to be entered) have continued for more than a predetermined time, notifications that an unauthorized credit card or the like (a lost card report has been submitted) has been read, notifications that a PIN has been entered, notifications that the entered PIN is incorrect, notifications that credit (authorization) has been denied, notifications that payment has been completed, notifications that a credit card or the like has been left behind, and in the case of electronic money, notifications that there is insufficient balance.

[0020] Notifications from the cash settlement unit 106 include, for example, notifications that various waiting states (e.g., waiting for cash to be inserted, waiting for change to be removed) have continued for more than a predetermined time, notifications regarding inserted coins (deposits) (that they have been inserted, the number of coins of each denomination), notifications regarding dispensed coins (change) (that they have been dispensed, the number of coins of each denomination), etc.

[0021] That is, for example, when the registration and settlement device 10 (CPU 101) displays screen A, it stores in the electronic journal information as an event that screen A has been displayed; when small screen B is displayed, it stores in the electronic journal information as an event that small screen C has been displayed; when product C is registered using the product preset button, it stores in the electronic journal information as an event that product C has been registered using the product preset button; when product D is canceled, it stores in the electronic journal information as an event that product D has been canceled; and when a store clerk is called, it stores in the electronic journal information as an event that a store clerk has been called. Also, when a PIN is entered, the registration and settlement device 10 stores in the electronic journal information as an event that the PIN has been entered; and when 5,000 yen (one 5,000 yen bill) is inserted as a deposit, it stores in the electronic journal information as an event that the deposit was 5,000 yen (one 5,000 yen bill).

[0022] Furthermore, when the total amount (transaction amount) of registered products (registered products) reaches (or exceeds) a predetermined amount (for example, 10,000 yen), the registration settlement device 10 may store in the electronic journal information as an event the fact that the total amount of the registered products has reached (or exceeded) the predetermined amount. When the total amount of registered products using the product preset buttons reaches (or exceeds) the predetermined amount, the registration settlement device 10 may store in the electronic journal information as an event the fact that the total amount of registered products has reached (or exceeded) the predetermined amount. When the number of operations of the product preset buttons reaches (or exceeds) a predetermined number, the registration settlement device 10 may store in the electronic journal information as an event the fact that the number of operations of the product preset buttons has reached (or exceeded) the predetermined number.

[0023] When the number of canceled products (the number of products canceled using the cancel button) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the number of canceled products has reached (or exceeded) the predetermined number.When the number of decreased purchase quantities (the number of products decreased using the minus button) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the decreased purchase quantities have reached (or exceeded) the predetermined number.When the total price of canceled or decreased products (products canceled using the cancel button or products decreased using the minus button) reaches (or exceeds) a predetermined amount, the registered settlement device 10 may store in the electronic journal information as an event that the total price of the canceled or decreased products has reached (or exceeded) the predetermined amount.

[0024] When an act of taking away unregistered goods (described later) occurs, the registered settlement device 10 may store in the electronic journal information as an event that an act of taking away unregistered goods has occurred. When the number of occurrences (detection number) of acts of taking away unregistered goods (described later) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the number of occurrences of acts of taking away unregistered goods (described later) has reached (or exceeded) the predetermined number.

[0025] The registered settlement device 10 generates electronic journal information to indicate the time (timing) at which an event occurs within a transaction. For example, the registered settlement device 10 may store in the electronic journal information the current time at the start of a transaction and the current time at which each event occurs. In other words, by referencing the electronic journal information, it may be possible to determine that a transaction started at XX:XX, event a occurred at XX:XX, and event b occurred at XX:XX, △ seconds. The registered settlement device 10 may also store in the electronic journal information the current time at the start of a transaction and the elapsed time from the start of a transaction at the time each event occurs. In other words, by referencing the electronic journal information, it may be possible to determine that a transaction started at XX:XX, event a occurred XX seconds after the start of the transaction, and event b occurred △△ seconds later.

[0026] (Operations recorded as history in the operation log information) The operations stored as history in the operation log information are operations that can change the display content of the display unit 103. In other words, the operations stored as history in the operation log information are operations that can change the display content of the display unit 103. The registration and settlement device 10 (CPU 101) stores operations that can change the display content of the display unit 103 in the operation log information. Specifically, the registration and settlement device 10 stores operations on the display unit 103 in the operation log information. In other words, because an operation on the display unit 103 can change the display information within the same screen or transition to another screen (because an operation on the display unit 103 can change the display content), the registration and settlement device 10 stores operations on the display unit 103 in the operation log information. For example, the registration and settlement device 10 stores the type of button touched, the touch position (coordinates), etc. as operations on the display unit 103 in the operation log information. In addition to cases where the display content of the display unit 103 changes due to a touch, even when the display content of the display unit 103 does not change due to a touch (for example, when a button does not respond when touched (when a button that is in an inoperable state is touched), or when a location other than a button is touched), the type of button that was touched (determined from the touch position), the touch position, etc. may be stored in the operation log information.

[0027] The display content of display unit 103 can change depending on operations on display unit 103, but the display content of display unit 103 can also change depending on other operations. For example, the display content of display unit 103 can also change depending on operations on scanner unit 104 (scanning operation), operations on non-cash payment unit 105 (operation of buttons on non-cash payment unit 105, etc.), and operations on cash payment unit 106 (insertion of banknotes or coins, removal of change, etc.). Therefore, in addition to operations on display unit 103, registration and settlement apparatus 10 also stores operations on scanner unit 104, operations on non-cash payment unit 105, and operations on cash payment unit 106 in the operation log information.

[0028] The registration and settlement device 10 (CPU 101) generates operation log information so that it is possible to determine at what point in a transaction an operation occurred. For example, the registration and settlement device 10 may store in the operation log information the current time at the start of a transaction and the current time at the time of each operation. That is, by referring to the operation log information, it may be possible to determine that a transaction started at XX:XX minute, XX seconds, operation c occurred at XX:XX minute, and operation d occurred at XX:XX minute, △ seconds. The registration and settlement device 10 may also store in the operation log information the current time at the start of a transaction and the elapsed time from the start of a transaction at the time of each operation. That is, by referring to the operation log information, it may be possible to determine that a transaction started at XX:XX minute, XX seconds after the start of the transaction, operation c occurred, and △△ seconds after the start of the transaction.

[0029] The registration and settlement device 10 may store something other than an operation in the operation log information. For example, if the display content of the display unit 103 changes due to something other than an operation (if there is something other than an operation that changes the display content of the display unit 103), the registration and settlement device 10 may store something other than an operation that changes the display content of the display unit 103 as operation log information. In a configuration in which a message (alert, etc.) is displayed on the display unit 103 when a no-operation state, in which no operation is performed on the display unit 103 or no operation (scanning operation) is performed on the scanner unit 104, continues for a predetermined time, the registration and settlement device 10 may store in the operation log information an event that indicates that a no-operation state has continued for a predetermined time. In other words, the registration and settlement device 10 may store in the operation log information a portion of the events stored in the electronic journal information. The registration and settlement device 10 may store in the operation log information an operation other than an operation that changes the display content of the display unit 103 (an operation that does not correspond to an event stored in the electronic journal information). Operations other than those that change the display content of the display unit 103 may be stored in at least one of operation log information, electronic journal information, or third history information that is different from both operation log information and electronic journal information.

[0030] (Generation of display transition images) CPU 101 reproduces the display content on display unit 103 (more specifically, the transition of the display content, i.e., the history of changes in the display content). Specifically, CPU 101 generates an image showing the transition of the display content on display unit 103 (hereinafter referred to as a display transition image or a display history image). The display content on display unit 103 changes in response to an operation by an operator (such as a change in display information within the same screen or a transition to another screen), so CPU 101 generates (assembles) a display transition image based on screen information for each screen and operation log information (i.e., the operations actually performed). The display transition image is a moving image because it is an image that expresses the transition of the display content and is generated based on operation log information that includes a temporal component.

[0031] In addition, when the display content of display unit 103 changes due to something other than an operation (for example, when a configuration is such that a message is displayed on display unit 103 when a no-operation state continues for a predetermined time), and when something other than an operation that changes the display content of display unit 103 (that a no-operation state has continued for a predetermined time) is stored as operation log information, CPU 101 generates a display transition image by referring to the operation log information.

[0032] On the other hand, if the display content of the display unit 103 changes due to something other than an operation and operations other than those that change the display content of the display unit 103 are not stored as operation log information, the CPU 101 generates a display transition image by referring to information that stores operations other than those that change the display content of the display unit 103 in addition to the operation log information (e.g., electronic journal information, third history information).

[0033] In addition, the CPU 101 may generate a display transition image when there is an explicit instruction (a specified operation by a store clerk, for example, an operation by a store clerk on the screens of Figures 5(B), 7(A), and 7(B)), or it may generate a display transition image even without an explicit instruction, as with transaction information, electronic journal information, and operation log information.

[0034] The memory unit 102 is a collective term for one or more various memory components (for example, memory or storage, such as RAM, ROM, hard disk, SSD (Solid State Drive), etc.). The memory unit 102 stores various information. For example, the memory unit 102 stores programs executed by the CPU 101, information referenced by the CPU 101 (for example, product master), information generated by the CPU 101 (for example, transaction information, electronic journal information, operation log information, display transition image), captured images captured by the imaging unit 110, etc.

[0035] The product master is a file that stores product names, sales prices, etc. in association with product identification information (for example, JAN codes or product codes previously set for each product preset button).

[0036] CPU 101 stores transaction information in memory unit 102 so that it is possible to identify which transaction it belongs to. For example, CPU 101 may generate transaction information including transaction identification information (transaction number) that is assigned (numbered) for each transaction and store the generated transaction information in memory unit 102, or may generate transaction information that does not include transaction identification information and store the generated transaction information in memory unit 102 in association with the transaction identification information. Similarly, CPU 101 stores electronic journal information and operation log information in memory unit 102 so that it is possible to identify which transaction it belongs to. The transaction number is assigned (numbered) at the start of the transaction (after operating registration start button BT01, described below).

[0037] The CPU 101 stores the display transition image in the storage unit 102 so that it is possible to identify which transaction it corresponds to. Specifically, a display transition image including the transaction identification number of the operation log information that was the basis for generating the display transition image may be generated and the generated display transition image may be stored in the storage unit 102, or a display transition image not including the transaction identification information may be generated and the generated display transition image may be stored in the storage unit 102 in association with the transaction identification information.

[0038] The CPU 101 stores the captured image captured by the imaging unit 110 in the storage unit 102 so that it is possible to identify which transaction it is for. For example, the CPU 101 stores the captured image captured by the imaging unit 110 in the storage unit 102 in association with transaction identification information.

[0039] That is, the transaction information, electronic journal information, operation log information, display transition image, and captured image are all stored in the storage unit 102 so that it is clear which transaction each of them pertains to.

[0040] The display unit 103 is a touch display that displays various information (for example, a screen for registration processing, a screen for settlement processing) to the operator (customer) and also accepts operations by the operator. Note that the operation history (touch positions) of the operator on the display unit 103 is stored in the storage unit 102 as operation log information, as described above.

[0041] The scanner unit 104 optically reads various pieces of information. For example, the scanner unit 104 scans a code attached to a product to read the product code.

[0042] The non-cash settlement unit 105 is a settlement mechanism for non-cash settlement (e.g., credit settlement, electronic money settlement, etc.). The non-cash settlement unit 105 may be a settlement terminal separate from the registration and settlement apparatus 10. The payment terminal 105 includes, for example, a reading unit, a communication unit (a communication unit that communicates with the registration and settlement apparatus 10, and a communication unit that communicates with an external server such as a credit server that executes credit settlement), an operation unit, and a display unit. The cash settlement unit 106 is a settlement mechanism for cash. The cash settlement unit 106 has an insertion slot for banknotes and coins and an ejection slot for banknotes and coins. The cash settlement unit 106 detects currency inserted into the insertion slot, identifies the number of coins inserted for each denomination, calculates the inserted amount, calculates the change amount, which is the difference between the inserted amount and the purchase amount, and ejects the change from the ejection slot. The insertion of banknotes and coins and the removal of change may be detected by a sensor.

[0043] The communication unit 107 transmits and receives information to and from other devices (e.g., the transaction status management device 40, the store controller 50, etc.). For example, the communication unit 107 receives a product master from the store controller 50 (or the head office server 60). The communication unit 107 also transmits transaction information to the store controller 50 (or the head office server 60).

[0044] The printing unit 108 prints and issues various media (receipts, coupons). A sensor may be used to detect the removal of a medium. The audio output unit 109 outputs audio (for example, a confirmation sound, a warning sound, audio guidance, etc.).

[0045] The imaging unit 110 captures an image of the operator (customer). Specifically, the imaging unit 110 captures an image of the vicinity of the front of the scanner unit 104. That is, the imaging unit 110 can capture images of the customer's scanning operation, the operation of inserting cash into the cash settlement unit 106 (or the operation of removing cash from the cash settlement unit 106), etc. The image captured by the imaging unit 110 may be a still image or a moving image. For example, under the control of the CPU 101, the imaging unit 110 captures a moving image from the start of the transaction (when the registration start button BT01 is operated) to the end of the transaction (when a receipt is issued or when removal of the receipt is detected). Note that the images (still images and moving images) captured by the imaging unit 110 are stored in the memory unit 102, as described above.

[0046] The registration and settlement apparatus 10 may include an imaging unit 1110 (not shown) that images the customer's face in addition to the imaging unit 110 that images the area in front of the scanner unit 104 (such as the customer's scanning operation or cash insertion operation). The registration and settlement apparatus 10 may use the imaging unit 1110 to capture a still image of the customer's face at the start of the transaction (when the registration start button BT01 is pressed) as a representative face image. The registration and settlement apparatus 10 may also include an imaging unit 2110 (not shown) that images the upstream side (the top surface of the upstream counter 12) and an imaging unit 3110 (not shown) that images the downstream side (the top surface of the downstream counter 13). The registration and settlement apparatus 10 may include multiple imaging units as described above, but the PIN input area of ​​the non-cash payment unit 105 is not imaged (or the image is processed and stored). Like the imaging unit 110, the imaging unit 1110 (the same applies to the imaging units 2110 and 3110) is capable of capturing both moving images and still images.

[0047] The light-emitting unit 112 notifies various pieces of information by emitting light. For example, the light-emitting unit 112 notifies that an operator has performed an operation to call a store clerk. The light-emitting unit 112 may also notify operations related to an operation to cancel a product, an operation to reduce the purchase quantity, an operation of a product preset button, etc. For example, when the number of product cancellations (similar to the number of reductions in the purchase quantity and the number of operations of the product preset button) exceeds a predetermined threshold, the light-emitting unit 112 may emit light in a manner that indicates that the number of product cancellations has exceeded the predetermined threshold.

[0048] The monitoring display unit 113 is, for example, a liquid crystal display, and displays various information to those around it (the operator, other customers, and store staff). For example, the monitoring display unit 113 may display the number of the device while it is on standby (waiting for a customer), and may display an image (moving image or still image) captured by the imaging unit 110 during processing. The number of the device is displayed on both sides and the back of the monitoring display unit 113.

[0049] Detection unit 120 detects the presence of a person using a human presence sensor (e.g., a sensor using infrared rays). Detection unit 120 may be equipped with multiple human presence sensors with different detection positions. For example, detection unit 120 may be composed of a first sensor that detects a person in a direction looking down near the front of (front and nearby) upstream counter 12 (i.e., detects the presence of a person near the front of upstream counter 12), a second sensor that detects a person in a direction looking down near the front of main unit 11 (i.e., detects the presence of a person near the front of main unit 11), and a third sensor that detects a person in a direction looking down near the front of downstream counter 13 (i.e., detects the presence of a person near the front of downstream counter 13).

[0050] (Operation confirmation verification process) As described above, the registration and settlement device 10 can execute an operation confirmation and verification process based on the operation of a store clerk. The operation confirmation and verification process is a process for confirming whether a customer's operation (operation in the registration process or the settlement process) is correct (or incorrect), or for verifying whether the customer's operation is correct (or incorrect). The registration and settlement device 10 proceeds with the process (registration process, settlement process) based on the customer's operation, i.e., without the intervention of a store clerk, so there is a higher possibility of an incorrect operation (intentional incorrect operation, incorrect operation due to mistake) compared to when the store clerk is involved. As a countermeasure against incorrect operations, the store (store clerk) executes an operation confirmation and verification process as appropriate to verify whether the customer's operation is correct. Hereinafter, the function realized by (provided by) the operation confirmation and verification process may be referred to as an operation confirmation and verification function (or operation confirmation function, or operation verification function).

[0051] Specifically, as an operation confirmation verification process (operation confirmation verification function), the registration and settlement apparatus 10 displays on the display unit 103 captured images (moving images) captured by the imaging unit 110 and display transition images (moving images) generated based on operation log information. For example, the registration and settlement apparatus 10 displays the captured images and the display transition images side by side and in sync on the same screen (e.g., small screen SG40 in FIG. 5(C) etc.) based on a predetermined operation by the store clerk on a predetermined screen dedicated to the store clerk (e.g., small screen SG30 in FIG. 5(B), small screen SG31 in FIG. 7(A), small screen SG32 in FIG. 7(B)). Therefore, the store clerk (or the store clerk and the customer when the store clerk operates in the presence of the customer) can simultaneously check the captured images and the display transition images displayed in sync with the captured images.

[0052] Figures 3 and 4 are examples of displays on the registration and settlement device 10 (examples of displays on the display unit 103). Specifically, Figures 3 and 4 show examples of displays from the start of product registration to the selection of a payment method. Figure 3(A) is a transaction start screen (standby screen). Before the transaction begins (while waiting for a customer), the registration and settlement device 10 displays the transaction start screen as shown in Figure 3(A). A registration start button BT01 for starting a transaction is provided (displayed) on the transaction start screen. In addition, a store clerk call button is provided at the top of the transaction start screen (as are other screens).

[0053] 3(A), since the registration start button BT01 is provided relatively high on the transaction start screen (close to the image capture unit 110), the customer's line of sight is relatively upward when the customer operates the registration start button BT01. Therefore, when a representative face image is captured, a suitable representative face image (with the line of sight relatively upward) can be obtained.

[0054] When the registration start button BT01 on the transaction start screen is pressed, the registration and settlement device 10 displays the initial registration screen (the registration screen before the first product is registered) as shown in FIG. 3(B). The initial registration screen (and the registration screen as well) has a total amount display area HR10 that displays total information (total number of items purchased, total amount), and a product information display area HR11 that displays information about the registered products. The initial registration screen (and the registration screen as well) also has a checkout button BT12 that is operated to proceed to checkout (payment) (i.e., to complete registration). Note that the checkout button BT12 becomes operable after the first product is registered, and therefore, on the initial registration screen, which is the registration screen before the first product is registered, the checkout button BT12 is displayed in an inoperable mode (grayed out, indicated by a dashed line in the figure)

[0055] The initial registration screen (and the registration screen as well) also has various buttons (vegetable button, fruit button, fresh fish button, meat button) that are operated when registering products that do not have barcodes. For example, when the vegetable button is operated, the registration and settlement device 10 displays product preset buttons for registering individual vegetables (cucumber, carrot, etc.) (see FIG. 4(A)). The same applies to the fruit button, fresh fish button, and meat button.

[0056] When the customer scans the barcode of a can of beer from the scene shown in Fig. 3(B) (initial registration screen), the registration and settlement device 10 displays a registration screen such as that shown in Fig. 3(C). Product information display area HR11 displays information about the product (can of beer) (product name, quantity, price). Additionally, total amount display area HR10 displays the total amount for the product.

[0057] The display field for the product (canned beer) in the product information display area HR11 is provided with a plus button (the button marked "+" in the figure) that is operated to increase the purchase quantity of the product (canned beer), a minus button (the button marked "-" in the figure) that is operated to decrease the purchase quantity of the product, and a cancel button (the button marked "cancel" in the figure) that is operated to cancel the purchase of the product. Note that the minus button is operable when the purchase quantity is 2 or more, and therefore in the scene of Figure 3(C), the minus button is displayed in a manner that indicates that it cannot be operated.

[0058] When the vegetable button is operated from the scene (registration screen) of Fig. 3(C), the registration and settlement device 10 displays a small screen SG13 on top of the registration screen, as shown in Fig. 4(A). The small screen SG13 is provided with product preset buttons (cucumber button, carrot button, etc.) for registering individual vegetables. Although not shown, the small screen SG13 displayed on top of the registration screen when the fruit button is operated from the scene (registration screen) of Fig. 3(C) is provided with product preset buttons (apple button, mandarin orange button, etc.) for registering individual fruits. The same applies when the fresh fish button or the meat button is operated.

[0059] The small screen SG13 is also provided with a Next Product button that is operated to display the next product (another product preset button) on the small screen SG13, and a Close button that is operated to close (erase) the small screen SG13. The registration and settlement device 10 also closes the small screen SG13 when any of the product preset buttons is operated in addition to the Close button.

[0060] When the cucumber product preset button is operated from the scene in Figure 4(A) (registration screen with small screen SG13 displayed in the foreground), the registration and settlement device 10 closes the small screen SG13 as shown in Figure 4(B), adds information about the product (cucumber) to the product information display area HR11 of the registration screen, and updates the total information in the total amount display area HR10.

[0061] When the checkout button BT12 is pressed from the scene (registration screen) shown in Fig. 4(B), the registration settlement device 10 displays the payment method selection screen (also called the payment type selection screen or the settlement method selection screen) shown in Fig. 4(C). The payment method selection screen has buttons (cash button, credit button, electronic money button) corresponding to each payment method (cash payment, credit payment, electronic money payment).

[0062] Although not shown, the registered settlement device 10 displays (sequentially in response to the customer's operations, etc.) one or more settlement screens (one or more cash settlement screens, one or more credit payment screens, one or more electronic money payment screens) according to the payment method selected on the payment method selection screen (cash payment when the cash button is pressed, credit payment when the credit button is pressed, or electronic money payment when the electronic money button is pressed), and proceeds with the settlement process.The registered settlement device 10 prints and issues a receipt as the final operation in the settlement process (or as the final operation of the transaction after the settlement process is completed).

[0063] The registration and settlement device 10 is capable of reading point cards. There are no particular limitations on the timing of reading the point card before registration begins. For example, the registration and settlement device 10 may read the point card before registration begins (operating the registration start button BT01 to registering one product), may read the point card during product registration (registering one product to operating the checkout button BT12), or may read the point card when selecting a payment method (operating the checkout button BT12 to operating one button on the payment method selection screen).

[0064] Fig. 5 is a display example (display example on the display unit 103) of the registration and settlement apparatus 10. Specifically, Fig. 5 shows a display example up to the point where the captured image and the display transition image are displayed.

[0065] For example, when the clerk scans the transaction start screen (FIG. 3(A)) to read the clerk code of the clerk, the registration settlement device 10 displays the small screen SG20 in front of the transaction start screen, as shown in FIG. 5(A). The small screen SG20 is a maintenance screen for the clerk. Various buttons are provided on the small screen SG20. For example, the small screen SG20 has a record confirmation button BT21 that is operated to check the record (history).

[0066] A store clerk may monitor a customer who is registering or settling products using a registration and settlement device 10 (either directly with the naked eye in the checkout area or on a monitor that displays images from a surveillance camera), and if the store clerk notices anything concerning with the customer's operation, the store clerk may move to the registration and settlement device 10 (where the customer is) and, after the transaction is completed (or during the transaction), have the registration and settlement device 10 read the clerk code and display the small screen SG20. Also, if the store clerk finds a registration and settlement device 10 that is illuminating the light-emitting unit 112 (i.e., a registration and settlement device 10 that is calling a store clerk, or a registration and settlement device 10 where the number of product cancellations, the number of purchase quantity reductions, or the number of product preset button operations exceeds a predetermined threshold), the store clerk may move to the registration and settlement device 10 and, after the transaction is completed (or during the transaction), have the registration and settlement device 10 read the clerk code and display the small screen SG20.

[0067] When the record confirmation button BT21 is operated from the scene in Figure 5(A) (the transaction start screen with small screen SG20 displayed in the foreground), the registered settlement device 10 displays small screen SG30 in front of the transaction start screen in place of small screen SG20, as shown in Figure 5(B). Small screen SG30 is a transaction confirmation screen (electronic journal search and viewing screen) for the clerk. Small screen SG30 has a transaction list display area HR31.

[0068] The transaction list display area HR31 shown in FIG. 5(B) displays basic transaction information (described below) for each transaction in a selectable manner. That is, the registered settlement device 10 accepts a touch (selection) of the basic transaction information (display area for the basic transaction information) of a transaction as a selection operation for that transaction. In other words, when a store clerk selects a transaction, the clerk touches (selects) the basic transaction information (display area for the basic transaction information) of that transaction.

[0069] (Basic transaction information) The basic transaction information is basic information about a transaction. The basic transaction information includes, for example, the transaction date and time (e.g., the transaction completion date and time), the receipt number (transaction number), the journal number, and event information within the transaction. The event information within the transaction is information about events that occurred in the transaction, i.e., events recorded in the electronic journal information of the transaction, such as information about an event that the transaction amount (total amount) reached 10,000 yen (i.e., information indicating that the transaction amount was 10,000 yen or more), information about an event that a store clerk was called (i.e., information indicating that the transaction involved a store clerk call), information about an event that a product was registered using a product preset button (i.e., information indicating that the transaction involved an operation of the product preset button), information about an event that a product was canceled (i.e., information indicating that the transaction involved a cancellation operation (operation of the cancel button)), and information about an event that a purchase quantity was reduced (i.e., information indicating that the transaction involved a reduction operation of the purchase quantity (operation of the minus button)).

[0070] For example, "Man" in Figure 5(B) indicates that the transaction amount is 10,000 yen or more. "Call" indicates that the transaction involved a call to a store clerk. "P" indicates that the transaction involved an operation of the product preset button. "Take" indicates that the transaction involved a cancellation operation. "Decrease" indicates that the transaction involved an operation to decrease the purchase quantity.

[0071] In addition, various buttons are provided on the small screen SG30. For example, the small screen SG30 is provided with a display button BT32 that is operated to display an electronic journal file, an image playback button BT33 that is operated to play (display) a captured image or a display transition image, and an end button BT34 that is operated to erase the small screen SG30 (to display the small screen SG20 instead of the small screen SG30).

[0072] It is also possible to make the end button BT34 of the small screen SG30 inoperable while the small screen SG40 (described later) is displayed. Also, the end button BT34 of the small screen SG30 can be operated while the small screen SG40 is displayed, and when the end button BT34 is operated while the small screen SG40 is displayed, the small screen SG40 is closed (erased) and the small screen SG20 is displayed instead of the small screen SG30. Alternatively, when the end button BT34 is operated while the small screen SG40 is displayed, a confirmation small screen on which a confirmation button and a cancel button are arranged may be further displayed, and when the confirmation button is operated, the confirmation small screen and the small screen SG40 are closed and the small screen SG20 is displayed instead of the small screen SG30, and when the cancel button is operated, the confirmation small screen may simply be closed (that is, the operation of the end button BT34 may be canceled).

[0073] When the display button BT32 is operated following the selection of a transaction in the transaction list display area HR31 (i.e., when a transaction is selected), the registered settlement device 10 displays the electronic journal information of the transaction in the transaction list display area HR31 instead of the basic transaction information.

[0074] When the registration settlement device 10 reads the clerk code during a transaction (for example, while the registration screen is displayed), it displays a small screen SG20 (Figure 5(A)) in front of the registration screen. However, if the record confirmation button BT21 is operated on the small screen SG20, the registration settlement device 10 may display electronic journal information for the transaction (electronic journal information up to the middle of the transaction) in the transaction list display area HR31 of the small screen SG30 displayed in place of the small screen SG20. In other words, when the record confirmation button BT21 is operated on the small screen SG20 displayed during a transaction, the registration settlement device 10 may display the electronic journal information for the transaction in progress in the transaction list display area HR31 from the beginning (i.e., omitting the transaction selection operation or the operation of the display button BT32).

[0075] Furthermore, when the image playback button BT33 is operated following the selection of a transaction in the transaction list display area HR31, the registration settlement device 10 displays the captured image and display transition image of that transaction. Note that the registration settlement device 10 may also display the captured image and display transition image of that transaction when the image playback button BT33 is operated while the electronic journal information of a transaction (including electronic journal information of a transaction in progress) is displayed in the transaction list display area HR31.

[0076] When a transaction (for example, a transaction shown in Figures 3 and 4) is selected in the transaction list display area HR31 from the scene in Figure 5(B) (the transaction start screen with small screen SG30 displayed in the foreground), followed by operation of the image playback button BT33, the registered settlement device 10 displays small screen SG40 in the foreground of the transaction start screen, as shown in Figure 5(C).

[0077] When the image playback button BT33 on the small screen SG30 is operated, the registration and settlement device 10 plays back the moving image on the small screen SG40 from the start (beginning) of the transaction. When the small screen SG40 is displayed from the small screen SG31 (FIG. 7(A)) or the small screen SG32 (FIG. 7(B)), the moving image is also played back from the start of the transaction. In other words, the moving image on the small screen SG40 in FIG. 5(C) does not show the playback content immediately after the image playback button BT33 on the small screen SG30 is operated (i.e., when the display of the small screen SG40 starts), but shows the point at which playback has progressed from the start of display (or after the playback point is specified after the display starts).

[0078] When the image playback button BT33 on the small screen SG30 is operated, the registration settlement device 10 may refer to the basic transaction information (such as "P") of the transaction and play (start playback) from a specific point (time point) on the small screen SG40, as shown in Fig. 5(C). Similarly, when displaying the small screen SG40 from the small screen SG31 (Fig. 7(A)) or the small screen SG32 (Fig. 7(B)), playback may also be performed from a specific point.

[0079] Fig. 6 shows a display example (display example of the small screen SG40) of the registration settlement device 10. Specifically, Fig. 6(A) is an enlarged view of the small screen SG40 shown in Fig. 5(A). The small screen SG40 is an image confirmation screen on which an image is played back and confirmed.

[0080] As shown in Fig. 6(A), the registration and settlement device 10 displays (plays) a captured image (symbol A in the same figure) and a display transition image (symbol B in the same figure) side by side on the small screen SG40. The captured image in Fig. 6(A) is an image captured by the imaging unit 110 when two items are registered, and the display transition image in Fig. 6(A) is a display transition image when two items are registered. In other words, the registration and settlement device 10 displays a captured image and a display transition image side by side at the same timing (the timing when two items are registered in Fig. 6(A)).

[0081] Image GA1 in the display transition image is an image showing the customer's operation position (touch position). When the touch position (coordinates) is included in the operation log information, the registration and settlement device 10 generates a display transition image in which the touch position is visualized using image GA1 (image GA1 is displayed at the touch position). Note that the registration and settlement device 10 may display image GA1 so that the center of image GA1 is the touch position (i.e., image GA1 may be displayed with the touch position at the center). Note that image GA1 in the figure is simply a circular image, but image GA1 may also be an image that evokes a touch (for example, a fingerprint-like image or a fingertip-like image).

[0082] 6(A) confirms that the customer actually registered a cucumber (¥50) when he should have registered a carrot (¥70). It also confirms that the customer operated (touched) the upper right portion of the cucumber product preset button. Note that the display transition image in FIG. 6(A) shows (is being played back) the moment the customer picked up a carrot and operated the cucumber product preset button (upper right portion) on the small screen SG13. Therefore, the registration information for the cucumber is not reflected (displayed) in the display transition image (the product information display area HR11 of the registration screen displayed as the display transition image). However, in the display transition image (not shown) immediately afterwards, the small screen SG13 is erased and the registration information for the cucumber is reflected.

[0083] The small screen SG40 also has a playback bar (symbol C in the figure) that indicates the playback location (playback time) of the video (captured image, display transition image). When a location on the playback bar is operated (touched), the registration and settlement device 10 plays the video from that location (i.e., changes the playback location). In other words, the store clerk can check the playback location of the video on the playback bar and operate the desired location on the playback bar to change (adjust) the playback location.

[0084] The lower part of the playback bar displays the location (time) of an event that occurred in the transaction (start, registering 1st item, registering 2nd item (P), cash selection, etc.). The "S" in "Register 1st Item S" indicates that the registration was made by scanning a barcode. In other words, "Register 1st Item S" indicates that the first item was registered by scanning a barcode (location). The "P" in "Register 2nd Item P" indicates that the registration was made by pressing a product preset button. In other words, "Register 2nd Item P" indicates that the second item was registered by pressing a product preset button (location). The information below the playback bar is useful when checking or changing the playback location. The events shown in FIG. 6(A) are just an example, and other events (e.g., a cancellation operation, a purchase quantity reduction operation, etc.) may also be displayed. The location of the event may be displayed above the playback bar instead of or in addition to the bottom of the playback bar.

[0085] Instead of or in addition to displaying the location of an event in association with the playback bar (displaying the location of the event at the bottom or top of the playback bar), buttons corresponding to each event (event-related buttons) may be provided. In the example shown in FIG. 6(A), the event-related buttons may be arranged on the small screen SG40 as a first-item registration S button, a second-item registration P button, a cash selection button, and a cash (300 yen) insertion button. When a button is operated, the registration and settlement device 10 starts playing the video from the corresponding location. For example, when the second-item registration P button is operated, the registration and settlement device 10 starts playing the video from the location where the second item is registered using the product preset button.

[0086] The small screen SG40 is also provided with a play (forward) button, a stop button, a fast-forward button, a rewind button, and a fast-rewind button. The play button is a button that is operated when playing back and displaying a moving image at a normal speed (the speed at which real time passes). The stop button is a button that is operated when freezing and displaying a moving image (displaying it like a still image). The fast-forward button is a button that is operated when playing back and displaying a moving image at a speed faster than normal. The rewind button is a button that is operated when playing back and displaying a moving image in reverse at a speed faster than normal. Note that when the image play button BT33 on the small screen SG30 is operated (i.e., when the display of the small screen SG40 starts), the registration settlement device 10 may play back and display the moving image from the beginning (or a specific point) at the normal speed, or may stop and display the beginning (or a specific point) of the moving image.

[0087] The small screen SG40 is provided with a simultaneous screen (simultaneous image) button BT41, a monitoring screen (captured image) button BT42, an operation screen (display transition image) button BT43, and a close button BT44. The close button BT44 is a button that is operated when closing (erasing) the small screen SG40.

[0088] The simultaneous screen button BT41, monitoring screen button BT42, and operation screen button BT43 are buttons that accept an operation to switch the image display and notify the current display state. The simultaneous screen button BT41 is a button that is operated when a captured image and a display transition image are to be displayed simultaneously. The monitoring screen button BT42 is a button that is operated when a captured image is to be displayed without displaying the display transition image. The operation screen button BT43 is a button that is operated when a display transition image is to be displayed without displaying the captured image. For example, when the monitoring screen button BT42 is operated from the scene of FIG. 6(A), the registration and settlement device 10 displays the captured image without displaying the display transition image (not shown). When the operation screen button BT43 is operated from the scene of FIG. 6(A), the registration and settlement device 10 displays the display transition image without displaying the captured image, as shown in FIG. 6(B). Switching is possible even during playback.

[0089] In the scene of Fig. 6(A), the simultaneous screen button BT41 is selected, and therefore in Fig. 6(A), the simultaneous screen button BT41 is displayed in a display manner (bold frame in the figure) indicating that it is selected. In the scene of Fig. 6(B), the operation screen button BT43 is selected, and therefore in Fig. 6(B), the operation screen button BT43 is displayed in a display manner indicating that it is selected.

[0090] A mode in which both captured images and display transition images are played back (displayed) may be referred to as a simultaneous playback mode, a mode in which captured images are played back without playing back display transition images may be referred to as a captured image playback mode, and a mode in which display transition images are played back without playing back captured images may be referred to as a display transition image playback mode. In other words, the simultaneous screen button BT41, monitoring screen button BT42, and operation screen button BT43 can also be considered as mode switching buttons for switching between the simultaneous playback mode, captured image playback mode, and display transition image playback mode.

[0091] (Other screens) The registration and settlement device 10 may display a small screen SG31 as shown in FIG. 7(A) instead of the small screen SG30 (FIG. 5(B)). In other words, when the record confirmation button BT21 on the small screen SG20 (FIG. 5(A)) is operated from the scene of FIG. 5(A), the registration and settlement device 10 may display the small screen SG31 in front of the transaction start screen instead of the small screen SG20, as shown in FIG. 7(A). The small screen SG31 is a screen for specifying a receipt number (transaction number) to identify a transaction. When an operation to specify a receipt number is performed on the small screen SG31 (that is, when a transaction is identified), the registration and settlement device 10 displays a small screen SG40 above the small screen SG31, and displays an image of the transaction (captured image, display transition image) on the small screen SG40.

[0092] The registration and settlement device 10 may display small screen SG32 instead of small screen SG30 (FIG. 5(B)) or small screen SG31 (FIG. 7(A)). In other words, when the record confirmation button BT21 on small screen SG20 (FIG. 5(A)) is operated from the scene of FIG. 5(A), the registration and settlement device 10 may display small screen SG32 in front of the transaction start screen instead of small screen SG20. Small screen SG32 is a screen for extracting one or more transactions (basic transaction information) by specifying a time. When an operation for specifying a time is performed on small screen SG32, the registration and settlement device 10 displays one or more pieces of basic transaction information within the specified time range in a selectable manner. The small screen SG32 shown in FIG. 7(B) is not the small screen SG32 immediately after it is displayed (immediately after the record confirmation button BT21 is operated), but is the small screen SG32 displaying one or more pieces of basic transaction information in a selectable manner. When a transaction (basic transaction information) is selected on the small screen SG32 and the image playback button BT33 is operated, the registration settlement device 10 displays a small screen SG40 at the top of the small screen SG32 and displays an image of the transaction (captured image, display transition image) on the small screen SG40. When a transaction (basic transaction information) is selected on the small screen SG32 and the display button BT32 is operated, the registration settlement device 10 erases the displayed basic transaction information and displays the electronic journal information of the transaction.

[0093] The small screen SG30 (FIG. 5(B)), the small screen SG31 (FIG. 7(A)), and the small screen SG32 (FIG. 7(B)) may be switchable. For example, a button for switching to another screen may be provided on each of the small screen SG30, the small screen SG31, and the small screen SG32.

[0094] The embodiment has been described above, but according to the registration and settlement device 10 of the above embodiment, the operation status can be suitably confirmed. That is, the operation status can be suitably confirmed.

[0095] Conventionally, operational errors and incorrect deposits (wrong currency) have been confirmed using images captured by a surveillance camera, and it may take time to understand the situation or the situation may not be accurately understood using only the images captured by the surveillance camera. However, the registration and settlement device 10 of the above embodiment synchronizes the timing of the captured image and the display transition image (display history image) to display (play) on the same screen, making it possible to easily (without spending time) and accurately understand the situation. Furthermore, because the registration and settlement device 10 notifies the touch position in the display transition image, when an operational error or the like is discovered, it is possible to infer to some extent whether it was a simple operational error or an intentional operation (an operation suspected of being fraudulent).

[0096] Therefore, for example, if a store clerk and a customer check together and it is determined that there is no problem on the customer's side, the store (clerk) can accept the situation and take appropriate action (for example, by apologizing). On the other hand, if the check determines that there is a problem on the customer's side, corrections and other measures can be taken smoothly. In other words, it is less likely that a situation will arise in which a response is taken without dispelling the customer's distrust (or leaving room for the customer to object), and work can proceed smoothly. In other words, regardless of the results of the check, the high degree of reproducibility of the situation gives both the store clerk and the customer a sense of acceptance, and work can proceed smoothly.

[0097] The registered settlement device 10 can also effectively deal with pressing the wrong product preset button (intentionally pressing the button for a product cheaper than the product being purchased), deposit problems (for example, mistakenly thinking (or intentionally claiming) that a 10,000 yen bill was inserted when in fact a 5,000 yen bill was inserted, and then complaining about insufficient change), and the taking home of unregistered products (see below).

[0098] (Taking away unregistered products) The act of taking away unregistered products (also referred to as the act of taking away unpaid products) is the act of taking away products without settling (i.e., without paying for them). Examples of acts of taking away unregistered products include the act of moving products from the upstream counter 12 to the downstream counter 13 without scanning (putting products from a shopping basket into a shopping bag without even pretending to scan), the act of moving products from the upstream counter 12 to the downstream counter 13 by intentionally performing an insufficient (inappropriate) scanning operation to prevent the product's barcode from being read (putting products from a shopping basket into a shopping bag while disguising the scanning operation), the act of moving a set product from the upstream counter 12 to the downstream counter 13 by unintentionally or intentionally reading the barcode of a single item (e.g., a single can of beer) instead of the barcode of the set product (e.g., a six-pack of canned beer), and the act of moving a product from the upstream counter 12 to the downstream counter 13 without realizing that the scanning operation failed without any intention (a simple scanning error).

[0099] By understanding the situation of product movement (for example, understanding the situation of product movement from image recognition of the image captured by the imaging unit 110 or the detection results of the detection unit 112) and understanding the registration status before and after the product movement (a situation in which the product has not been added), it is generally possible to recognize whether an unregistered product has been taken home, but by playing back and checking the captured image and the display transition image, it is possible to recognize more accurately and easily.

[0100] (Variations, etc.) Although the embodiments have been described above, the device configuration, data configuration, processing flow, display and output modes, etc. can be appropriately changed as shown in the following (1) to (18). In addition, the following (1) to (18) can be appropriately combined with each other.

[0101] (1) The size of image GA1 (circular image, fingerprint-like image, fingertip-like image, etc.) may be changeable whether the display transition image is being played back or is paused. For example, an enlargement button and a reduction button may be arranged on the small screen SG40, and the registration settlement device 10 may enlarge and display image GA1 with the touch position at the center when the enlargement button is operated, and reduce and display image GA1 with the touch position at the center when the reduction button is operated. If the display size of image GA1 is large, image GA1 stands out even during playback display, allowing the user to immediately recognize that a touch has occurred even during playback display. On the other hand, if the display size of image GA1 is small, the touch position can be accurately confirmed even when various buttons are densely arranged. In addition, the initial display size of image GA1 (the display size before the enlargement button or reduction button is operated) may be set.

[0102] (2) The coordinate information (X coordinate value, Y coordinate value) of the touch position may be displayed in the display transition image. For example, the registration settlement device 10 may display the coordinate information of the touch position near (above or below) the image GA1 showing the touch position. This gives the customer the impression that the touch operation has been properly memorized (recorded).

[0103] (3) The display / non-display of the image GA1 may be switchable. For example, a display (ON) button and a non-display (OFF) button may be arranged on the small screen SG40, and the registration settlement device 10 may display the image GA1 when the display button is operated, and may not display the image GA1 when the non-display button is operated. Similarly, the display / non-display of the coordinate information in (2) above may be switchable.

[0104] (4) The touch position may be immediately notified when a touch operation is performed. For example, a button that can be switched to a display mode that immediately notifies the touch position (touch position immediate display mode) may be provided on the small screen SG40, and the registration and settlement device 10 may switch to the touch position immediate display mode when the button is operated. In the touch position immediate display mode, the registration and settlement device 10 displays a mock screen for operation confirmation (e.g., a mock registration screen, a mock settlement screen, etc.), and when a touch operation is performed on the mock screen, the image GA1 may be displayed at the touch position at the time of the touch operation (and while the touch is being performed). For example, if necessary (e.g., when the customer is not satisfied with the touch position), the store clerk may switch to the touch position immediate display mode, ask the customer to touch any position on the mock screen, and indicate that the image GA1 will be displayed at the actual touch position. This allows for smooth handling.

[0105] (5) The screens for selecting a transaction (small screen SG30, small screen SG31, small screen SG32) may be erased, and a screen (small screen SG40) displaying a moving image (captured image, display transition image) may be displayed. In other words, in the above embodiment, when a transaction is selected on the screen for selecting a transaction, the registered settlement device 10 maintains the display of the screen for selecting a transaction and displays a new screen displaying a moving image of the transaction. However, when a transaction is selected on the screen for selecting a transaction, the registered settlement device 10 may erase the screen for selecting a transaction and display a screen displaying a moving image of the transaction.

[0106] (6) The screen for selecting a transaction (small screen SG30, small screen SG31, small screen SG32) and the screen for displaying a moving image (small screen SG40) may be a common screen. In other words, in the above embodiment, when a transaction is selected on the screen for selecting a transaction, the registration settlement device 10 displays a moving image of the transaction on a screen (small screen SG40) different from the screen for selecting the transaction (small screen SG30, small screen SG31, small screen SG32). However, when a transaction is selected on the screen for selecting a transaction, the registration settlement device 10 may display a moving image of the transaction on the screen for selecting the transaction.

[0107] (7) Product preset buttons may be displayed on the normal screen. In other words, in the above embodiment, when the vegetable button on the registration screen is operated, the registration and settlement device 10 displays a small screen SG13 on which product preset buttons for individual vegetables are arranged in front of the registration screen. However, when the vegetable button on the registration screen is operated, the registration and settlement device 10 may display a screen (normal screen) on which product preset buttons for individual vegetables are arranged instead of the registration screen. Since it becomes possible to arrange the product preset buttons at a certain distance from each other, touching the product preset buttons can be reduced or it can be made more difficult to claim that a touch was an error. Note that when a product preset button or the back button is operated, the registration screen can be returned to.

[0108] (8) A screen where a store clerk intervenes may be displayed as a normal screen. In other words, in the above embodiment, an example was described in which the registration and settlement device 10 displays a screen where a store clerk intervenes on a small screen (e.g., small screen SG20, small screen SG30, small screen SG31, small screen SG32, small screen SG40), but it may also be displayed on a normal screen. For example, the registration and settlement device 10 may display a moving image (captured image, display transition image) on a normal screen.

[0109] (9) The captured image may be a moving image including audio. The same applies to the display transition image. Audio-related buttons (e.g., an audio ON button, an audio OFF button, and a volume button) may be arranged on the small screen SG40. When a captured image including audio and a display transition image including audio are played simultaneously, the audio of one of the images may be muted (or the audio output may be significantly reduced compared to the other).

[0110] (10) The display transition image may be a moving image obtained by recording (storing) the display content on the display unit 103. In other words, in the above embodiment, it has been explained that the registration and settlement device 10 generates the display transition image based on the operation log information, but the display transition image may also be generated by recording the display content on the display unit 103 (that is, the display transition image may be a recording of the display content).

[0111] When the display transition image is a moving image generated by recording the display content, image GA1 is displayed in the display transition image, just as when the display transition image is a moving image generated based on operation log information. For example, the registration and settlement device 10 may superimpose image GA1 based on operation log information on a recorded moving image of the display content on the display unit 103, store the superimposed image as a display transition image, and display the superimposed image based on the operation log information on the stored display transition image. In other words, the superimposed image GA1 on a recorded moving image of the display content may be referred to as a display transition image, or the recorded moving image of the display content (the moving image without image GA1 superimposed on it) may be referred to as a display transition image.

[0112] (11) The registration settlement device 10 does not need to have either the non-cash settlement unit (payment terminal) 105 or the cash settlement unit 106 (it is sufficient to have either one).

[0113] (12) Moving images (captured images, display transition images) may be stored in another device. In other words, in the above embodiment, an example was described in which a registered settlement device 10 stores its own moving images (moving images related to a transaction (customer) in its own registered settlement device 10) in the memory unit 102 of its own registered settlement device 10. However, a registered settlement device 10 may store its own moving images in another device (e.g., a transaction status management device 40, a store controller 50, another registered settlement device 10 (a representative registered settlement device 10), or a headquarters server 60). For example, a registered settlement device 10 that has generated a moving image may transmit the generated moving image to another device, and the other device that receives the moving image may store the moving image, or the registered settlement device 10 that has generated a moving image may access the other device and store the generated moving image in the memory unit of the other device.

[0114] (12-1) If there is only one registered settlement device 10 in the sales system 1 (i.e., if there is only one device to be monitored (device that generates video images)), the video images are stored in another device so that it is clear which transaction the video images are for.

[0115] (Images are stored in other devices / only one device is monitored) The registered settlement device 10 transmits a captured image of a certain transaction together with the transaction identification information of the transaction to another device, and the receiving device stores the captured image in association with the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the captured image of a certain transaction in a memory unit of the other device in association with the transaction identification information of the transaction.

[0116] In addition, the information name of the captured image (data name, file name) or the storage location of the captured image (another device, folder, path) may be different (separated) for each transaction so that it is possible to identify which transaction the captured image is for.

[0117] (Display transition images are stored on other devices / only one device is monitored) It has been explained that the registration and settlement device 10 may generate a display transition image that includes a transaction identification number, or may generate a display transition image that does not include transaction identification information. In the former case (when the display transition image includes transaction identification information), the registration and settlement device 10 simply transmits the display transition image of a certain transaction to another device, and the other device, which is the receiving side, simply stores the display transition image. Alternatively, the registration and settlement device 10 accesses the other device and simply stores the display transition image of a certain transaction in the memory unit of the other device.

[0118] On the other hand, in the latter case (when the display transition image does not include transaction identification information), the registration settlement device 10 transmits the display transition image of a certain transaction together with the transaction identification information of the transaction to another device, and the receiving device stores the display transition image in association with the transaction identification information. Alternatively, the registration settlement device 10 accesses the other device and stores the display transition image of a certain transaction in the memory of the other device in association with the transaction identification information of the transaction. Note that the information name of the display transition image or the storage location of the display transition image may be different for each transaction so that it is possible to identify which transaction the display transition image is for.

[0119] (12-2) If there are two or more registered settlement devices 10 in the sales system 1 (i.e., if there are two or more devices to be monitored), the video image is stored in another device so that it is clear which registered settlement device 10 and which transaction it is for.

[0120] (Images are stored in another device / More than two devices are monitored) The registered settlement device 10 transmits a captured image of a transaction to another device along with device identification information that identifies the registered settlement device 10 and the transaction identification information of the transaction, and the other device that receives the image stores the captured image in association with the device identification information and the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the captured image in the memory of the other device in association with the device identification information of the registered settlement device 10 and the transaction identification information of the transaction. Note that the information name of the captured image or the storage location of the captured image may be different for each registered settlement device 10 and each transaction, so that it is possible to identify which registered settlement device 10 and which transaction the captured image is for.

[0121] (Save the display transition image to another device / Monitor two or more devices) If the display transition image includes transaction identification information, the registered settlement device 10 transmits the display transition image of a certain transaction to another device together with device identification information that identifies the registered settlement device 10, and the other receiving device stores the display transition image in association with the device identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image in the memory of the other device in association with its own device's device identification information. Note that the information name of the device identification information or the storage location of the device identification information may be different for each registered settlement device 10, so that it is possible to identify which registered settlement device 10 the device identification information belongs to.

[0122] On the other hand, if the display transition image does not include transaction identification information, the registered settlement device 10 transmits the display transition image of a certain transaction to another device along with device identification information that identifies the registered settlement device 10 and the transaction identification information of the transaction, and the other device that receives the display transition image stores the display transition image in association with the device identification information and the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image of a certain transaction in the memory of the other device in association with the device identification information of its own device and the transaction identification information of the transaction. Note that the information name of the display transition image or the storage location of the display transition image may be different for each registered settlement device 10 and each transaction, so that it is possible to identify which registered settlement device 10 and which transaction the display transition image belongs to.

[0123] Note that the captured image may include both device identification information and transaction identification information, may include transaction identification information but not device identification information, or may include device identification information but not transaction identification information. For a captured image that includes both device identification information and transaction identification information, both the device and the transaction may be identified from the captured image itself (neither the device identification information nor the transaction identification information may need to be stored in association). For a captured image that does not include device identification information but does include transaction identification information, the transaction may be identified from the captured image itself (neither the device identification information nor the transaction identification information may need to be stored in association with the transaction identification information). For a captured image that does not include transaction identification information but does include device identification information, the device may be identified from the captured image itself (neither the device identification information nor the transaction identification information may need to be stored in association with the device identification information). The same applies to display transition images.

[0124] (13) A moving image in a certain registered settlement device 10 may be displayed (played back) on a device different from the registered settlement device 10 (for example, a transaction status management device 40, another registered settlement device 10). Note that while a moving image may be transmitted when it is stored (see (12) above), the moving image may also be transmitted when it is displayed even when the device that displays the moving image and the device that stores the moving image are different.

[0125] (13-1) When there is one registered settlement device 10 in the sales system 1 (i.e., when there is one device to be monitored), a transaction is specified and a moving image is displayed. For example, a screen (e.g., small screen SG30, small screen 31, small screen SG32) on which a transaction can be specified (i.e., a transaction identification number can be identified) may be displayed, and the specified moving image may be displayed on that screen. The screen on which the above-mentioned transaction can be specified may have a search function or a list display function, like small screen SG30 or small screen SG32.

[0126] (13-2) If there are two or more registered settlement devices 10 in the sales system 1 (i.e., if there are two or more devices to be monitored), a device and a transaction are specified and a video image is displayed. For example, a screen on which a device and a transaction can be specified (i.e., device identification information and transaction identification number can be specified) may be displayed, and the specified video image may be displayed on that screen. The screen on which the transaction can be specified may have a search function or a list display function.

[0127] (14) A device other than the registration and settlement device 10 may be monitored. In other words, in the above embodiment, an example was described in which video images (captured images, display transition images) of the registration and settlement device 10, which can execute the registration process and the settlement process, were acquired. However, video images of other devices may also be acquired. For example, video images of the settlement device 230 (not shown), the settlement device 250 (not shown), or the charging device 260 may be acquired. A label issuing device 270 (not shown) that selects products and prints and issues labels may be installed in the store, and video images of the label issuing device 270 may be acquired. Furthermore, a weighing device 280 (not shown) that weighs products may be installed in the store, and video images of the weighing device 280 may be acquired. Note that the label issuing device 270 may have an object scanning function and may display (or identify) candidate products by object scanning in a selectable manner. The registration and settlement device 10, the settlement device 230, the settlement device 250, and the weighing device 280 may also have an object scanning function. The weighing device 280 may also have a label printing function. A ticket vending machine 290 (not shown) may be installed inside the store, and video images of the ticket vending machine 290 may be acquired. The ticket vending machine 290 may be one installed in a restaurant or the like. A self-service checkout device installed in a restaurant or the like may also be monitored (video images of the self-service checkout device installed in a restaurant or the like may be acquired).

[0128] Related to (12) to (14) above, the device that generates the video (the device to be monitored), the device that stores the video, and the device that displays (plays) the video may all be the same device or may all be different devices. For example, video from device A (e.g., registration and settlement device 10, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc.) may be stored in device A and displayed on device A or device B (e.g., transaction status management device 40). Alternatively, video from device A may be stored in device B and displayed on device B or device C (e.g., registration and settlement device 10, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc., other than device A). When there are two or more devices to be monitored, the device that displays the video may accept designation of the device and transaction and display the designated video from the designated device.

[0129] (15) Video images from other imaging units (cameras) may be displayed (played). In other words, while the above embodiment describes an example in which an image captured by the imaging unit 110 is displayed, images captured by other imaging units (e.g., imaging unit 1110, imaging unit 2110, imaging unit 3110) may be displayed. One image selected from images captured by multiple imaging units (e.g., imaging unit 110, imaging unit 1110, imaging unit 2110, imaging unit 3110) may be displayed, or multiple display areas for captured images may be provided to simultaneously display multiple captured images. In other words, one or more captured images captured in respective areas (an area in which a customer can be imaged, an area in which a customer's operations (scanning operation, screen operation, deposit operation, movement of a product) can be imaged, an area in which an upstream product (before movement) can be imaged, and an area in which a downstream product (after movement) can be imaged) and a display transition image may be displayed. Note that a selection button (or a switching button) may be provided to accept a selection (switching) operation between displaying one captured image and displaying multiple captured images. Furthermore, when one captured image is displayed, a button may be provided that allows the user to select (switch) the captured image to be displayed. The same applies when multiple captured images are displayed (for example, when two out of three are displayed). Furthermore, the captured images (number and type) to be displayed may be automatically switched not by operation but by settings (by setting in advance for each event a camera image with a suitable shooting accuracy according to the event), or may be switched automatically using AI.

[0130] An externally acquired captured image may be displayed. For example, at least one of the imaging units 110, 1110, 2110, and 3110 may not be an imaging unit included in the registration and settlement apparatus 10, but may be an imaging unit separate from the registration and settlement apparatus 10 (a camera installed on a ceiling, wall, or pillar).

[0131] (16) Still images may be used for confirmation instead of moving images. In other words, in the above embodiment, an example has been described in which captured images and display transition images are displayed as moving images on the image display screen (small screen SG40). However, captured images and display transition images may be displayed as still images. As in the case of moving images, both captured still images and display transition images of still images may be displayed (simultaneous playback mode), or either one may be displayed (captured image playback mode, display transition image playback mode).

[0132] A device that displays still images (such as the registration and settlement device 10 or the transaction status management device 40) may display a still image (captured image or display transition image) at the time an event occurs. For example, a device that has a next button and a previous button arranged on the image display screen and displays still images may display a still image of the next event occurring in response to operation of the next button, and a still image of the previous event occurring in response to operation of the previous button. Furthermore, a device that has event-corresponding buttons arranged on the image display screen and displays still images may display a still image of the event occurring corresponding to the event-corresponding button operated in response to operation of the event-corresponding button.

[0133] The display of moving images may also be controlled by a next button or a previous button. That is, a device that displays moving images may display a moving image of the next event occurring when the next button is pressed (skip to the next event), and display a moving image of the previous event occurring when the previous button is pressed (skip to the previous event (backskip)).

[0134] (Generating still images) The monitored devices (registered settlement devices 10, settlement devices 230, settlement devices 250, charging devices 260, label issuing devices 270, weighing devices 280, ticket vending machines 290, etc.) may generate still images from the captured moving images after generating moving images, or may generate still images from the beginning without generating moving images. The same applies to display transition images.

[0135] In the former case (when a captured image of an image is generated and then a captured image of a still image is generated from the captured image of a moving image), a still image at the time when an event occurred may be extracted from the moving image. The same applies to the display transition image. The time when an event occurred may be identified by referring to electronic journal information.

[0136] In the latter case (when capturing images is generated as still images from the beginning without generating moving images), still images may be generated continuously and a still image at the time of event occurrence may be obtained (still images at other times may be discarded). For the display transition image, a still image at the time of event occurrence may be generated by referring to operation log information, etc. The time of event occurrence may be identified by recognizing the occurrence of the event itself, or by referring to electronic journal information.

[0137] (Moving image mode, still image mode) The system may have a moving image mode in which the captured image and the display transition image are confirmed (displayed) as moving images, and a moving image mode in which the captured image and the display transition image are confirmed as still images, and these modes may be switchable by a button, etc. Also, one of the captured image and the display transition image may be confirmed as a moving image, and the other may be confirmed as a moving image.

[0138] (17) Although the explanation has been omitted, events to be stored in the electronic journal information, events to be included in the basic transaction information (events displayed as "ten thousand" or "P" in the transaction list display area HR31), and events to be used as headings when displaying captured images and display transition images (events displayed below (or above) the playback bar, events arranged as event-related buttons) are set in advance. In other words, N events (event 1, event 2, ..., event N) are set as events to be stored in the electronic journal information, and M of the N events are A events are set as events to be included in the basic transaction information, and M B Events are set as headline events. A and M B may be the same or different numbers.A and M B Even if the number of events is the same, the events to be set may be different.

[0139] The above settings may be changeable as needed (for example, reconfigurable from a maintenance screen). In other words, the events stored as electronic journal information, the events included in the basic transaction information, and the events that become headings may be changeable as needed.

[0140] (18) Some or all of the functions (input / output, storage, processing (including judgment)) of each device (registered settlement device 10, transaction status management device 40, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc.) may be realized by a device other than the device described as the entity that executes the function. Each device may function as a so-called thin client specialized for input / output interfaces, etc. For example, a cloud server may execute a program to control the operation of each device, and each device may display a screen using a browsing function, etc.

[0141] <Summary of the embodiment> [Technical field] The present invention relates to an information processing device and a program. [Background technology] As a function for checking (inspecting) whether or not an unauthorized operation has been performed, a cash register is known that displays a video of the operation status captured by a camera (see, for example, Patent Document 1). [Prior art document] [Patent documents] [Patent Document 1] JP 2018-120635 A [Summary of the Invention] [Problem to be solved by the invention] However, there is room for improvement in the method of checking the operation status using the conventional technology as shown in Patent Document 1. The present invention has been made in view of such circumstances, and aims to provide a technology for conveniently checking the operation status. [Means for solving the problem]

[0142] (1) One aspect of the present invention for solving the above-described problems is an information processing device (e.g., the registration settlement device 10, the settlement device 230, the settlement device 250, the charging device 260, the label issuing device 270, the weighing device 280, the ticket vending machine 290) that performs processing based on the operation of a customer, and that has imaging means (e.g., the imaging unit 110, the imaging unit 210, the imaging unit 220) that images predetermined areas (e.g., an area where an image of the customer can be captured, an area where an image of the customer's operation can be captured, an area where an image of the product on the upstream side (before movement) can be captured, an area where an image of the product on the downstream side (after movement) can be captured). The information processing device is characterized in that it comprises an image capturing unit (image capturing unit 1110, image capturing unit 2110, image capturing unit 3110), and a display means (for example, display unit 103) that can accept touch operations and displays information according to the progress of processing, and the display means displays an image (moving image or still image) captured by the image capturing unit and a display transition image (moving image or still image) showing the transition of the display content by the display means on the same screen, with the time of capturing and the time of display coinciding (for example, see Figure 5 (C)). According to the information processing device of (1), the operation status can be conveniently confirmed.

[0143] (2) The information processing device described in (1) is characterized in that it is provided with a log information acquisition means for acquiring operation log information (for example, generation of operation log information by CPU 101), and the display means displays, as the display transition image, an image generated based on the log information acquired by the log information acquisition means. According to the information processing device of (2), it is possible to appropriately generate a display transition image based on an operation while suppressing the capacity.

[0144] (3) The information processing device described in (2) is characterized in that the log information acquisition means acquires log information of operations including touch operations, and the display means displays the display transition image that notifies the position of the touch operation (for example, by image GA1). According to the information processing device of (3), it is possible to easily check which position has been operated.

[0145] (4) The information processing device described in (1) is characterized in that it is provided with a log information acquisition means for acquiring log information of operations including touch operations, and the display means displays touch position information indicating the position of the touch operation acquired by the log information acquisition means by superimposing it on the display transition image. According to the information processing device of (4), it is possible to easily check which position has been operated.

[0146] (5) An information processing device according to any one of (2) to (4), characterized in that the captured image and the display transition image at the time of occurrence of an event that occurred between the start and end of processing and was specified by operation (for example, an event specified by an event-corresponding button, the next event specified by a next button, or the previous event specified by a previous button). According to the information processing device of (5), it is possible to conveniently check the necessary parts.

[0147] (6) An information processing device according to any one of (1) to (4), characterized in that it is provided with a switching means for switching between a first mode (simultaneous playback mode) in which the display means displays the captured image and the display transition image, a second mode (captured image playback mode) in which the display means displays the captured image without displaying the display transition image, and a third mode (display transition image playback mode) in which the display means displays the display transition image without displaying the captured image. According to the information processing device of (6), the display can be switched appropriately depending on the confirmation content.

[0148] (7) An information processing device according to any one of (1) to (4), characterized in that it is provided with a transmitting means for transmitting the captured image and the display transition image (transmitting when storing in another device, transmitting when displaying on another device). The information processing device (7) can be flexibly operated.

[0149] (8) One aspect for solving the above-mentioned problem is a program for causing an information processing device, which has a display unit capable of accepting touch operations and progresses processing based on customer operations, to function as a computer, the program causing the computer to function as an image capture means for capturing an image of a predetermined area and a display control means for controlling the display of the display unit, the program being characterized in that the display control means displays the image capture and a display transition image showing the transition of the display content of the display unit on the same screen, with the time of capture coinciding with the time of display. According to the program (8), the operating status can be conveniently confirmed.

[0150] Note that a program for implementing the functions of each of the above-described devices (such as the registration and settlement device 10 and the transaction status management device 40) may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform the processing of each device. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program on a computer system. The term "computer system" here includes hardware such as an operating system and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including the Internet, a WAN, a LAN, a dedicated line, and other communication lines. The term "computer-readable recording medium" refers to portable media such as floppy disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Thus, a recording medium storing a program may be a non-transitory recording medium such as a CD-ROM. The term "recording medium" also includes internal or external recording media accessible from a distribution server for distributing the program. The program code stored on the distribution server's recording medium may be different from the program code executable on a terminal device. In other words, the format in which the program is stored on the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that is executable on the terminal device. The program may be divided into multiple parts, each of which may be downloaded at different times and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a storage medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be a program that realizes part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-described functions in combination with a program already stored in the computer system. [Explanation of symbols]

[0151] 1...Sales system, 10...Registration and settlement device, 40...Transaction status management device, 50...Store controller, 60...Head office server, 230...Settlement device, 250...Settlement device, 260...Charging device, 270...Label issuing device, 280...Weighing device, 290...Ticket vending machine

Claims

1. A system including an information processing device that progresses transaction processing based on a customer's operation, an imaging unit that images a predetermined area; a display unit capable of receiving touch operations; a switching unit for switching the display of the display unit; Equipped with The display unit The image captured by the imaging unit and a display transition image showing a transition of the display content of the display unit can be displayed, The switching unit is The display unit can be switched between a first mode in which the captured image and the display transition image are displayed so that the time of image capture and the time of display coincide with each other, and a second mode in which the captured image is displayed without displaying the display transition image on the display unit. An information processing system comprising:

2. A storage unit that stores event information of events that occur in a transaction. Equipped with The display unit an image captured by the imaging unit can be displayed, and when an operation for designating one of the events is performed, the image captured at the time of occurrence of the event can be displayed; The event is This includes taking home unregistered products without registering them.

2. The information processing system according to claim 1, wherein:

3. The display unit As the display transition image, an image generated based on the operation log information is displayed.

3. The information processing system according to claim 1 or 2.

4. The display unit The display transition image notifying the position of the touch operation included in the log information is displayed.

4. The information processing system according to claim 3.

5. The display unit Touch position information indicating the position of the touch operation included in the operation log information is superimposed on the display transition image and displayed.

4. The information processing system according to claim 3.

6. A program for causing an information processing device that processes a transaction based on a customer's operation to function as a computer, The computer imaging means for imaging a predetermined area; a display means capable of accepting a touch operation; a switching means for switching the display of the display means; It functions as The display means The image captured by the imaging means and a display transition image showing the transition of the display content by the display means can be displayed, The switching means The display means can switch between a first mode in which the captured image and the display transition image are displayed so that the time of image capture and the time of display coincide with each other, and a second mode in which the display means displays the captured image without displaying the display transition image. A program characterized by:

Images