Information processing system, information processing device, and program

Abstract

To suitably confirm an operation state.SOLUTION: The present invention relates to an information processing apparatus that advances processing according to operation by a customer, and comprises imaging means for capturing an image of the customer and display means capable of accepting touch operation and also displaying information corresponding to the progress of the processing. The display means displays the image captured by the imaging means and a display transition image showing transition of display details by the display means on the same screen while making the point in time of capturing the image and the point in time of displaying the images coincident with each other.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program. [Background technology]

[0002] There is known a cash register that displays a video of the operation status captured by a camera as a function for checking (inspecting) whether or not an unauthorized operation has been performed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-120635 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is room for improvement in the method of checking the operation status according to the conventional technology as shown in Patent Document 1.

[0005] The present invention has been made in consideration of the above circumstances, and has an object to provide a technique for conveniently checking an operation status. [Means for solving the problem]

[0006] One aspect for solving the above-mentioned problem is an information processing device that progresses processing based on a customer's operation, comprising an imaging means for imaging the customer and a display means capable of accepting touch operations and displaying information according to the progress of the processing, wherein the display means displays, on the same screen, an image captured by the imaging means and a display transition image showing the transition of the display content by the display means, with the time of imaging coinciding with the time of display. [Brief description of the drawings]

[0007] [Figure 1] 2 is an example of the appearance of a registration and settlement device. [Diagram 2] 2 is a configuration example of a registration settlement device. [Diagram 3] 13 is a display example of a registration settlement device. [Figure 4] 13 is a display example of a registration settlement device. [Diagram 5] 13 is a display example of a registration settlement device. [Figure 6] 13 is a display example of a registration settlement device. [Figure 7] 13 is a display example of a registration settlement device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] (Sales System) The registration settlement device 10 of this embodiment is one of the devices constituting the sales system 1 (not shown). The sales system 1 can be introduced into various business formats (supermarkets, convenience stores, drug stores, home centers, home appliance retailers, etc.). In addition to the registration settlement device 10, the sales system 1 includes a transaction status management device 40 (not shown), a store controller 50 (not shown), and a headquarters server 60 (not shown). The registration settlement device 10, the transaction status management device 40, and the store controller 50 are installed in a store. These are connected to each other so that they can communicate with each other via a LAN (which may be wired or wireless). The headquarters server 60 is installed outside the store (for example, a head office or a data center). The devices installed in the store (the registration settlement device 10, the transaction status management device 40, the store controller 50) and the device installed outside the store (the headquarters server 60) can communicate with each other (directly or via the store controller 50). The number of devices installed is not particularly limited. The headquarters server 60 may be a cloud server. The headquarters server 60 may be composed of two or more servers.

[0009] (Registration settlement device 10) The registration and settlement device 10 is a so-called fully self-service cash register that registers products based on the customer's operation and settles the payment based on the customer's operation. In other words, the registration and settlement device 10 executes a registration process for registering products and a settlement process for settling the registered products, and both processes are basically executed based on the customer's own operation. In addition, the registration and settlement device 10 can execute an operation confirmation and verification process (described later) based on the operation of a store clerk.

[0010] (Transaction status management device 40) The transaction status management device 40 is a device that manages the devices installed in the store (monitoring and controlling the processing status, operation status, etc.). The transaction status management device 40 may also manage the operators of the devices installed in the store (monitoring the operation contents, etc. of the customer while using the device, and checking the operation contents, etc. of the customer after use). The transaction status management device 40 comprises at least a CPU, a memory unit, a display unit (which may be configured as a touch panel), an operation unit (keyboard, mouse, etc.), and a communication unit. The transaction status management device 40 is used by store clerks.

[0011] (Store Controller 50) The store controller 50 is a server that mainly handles communication of information between devices installed inside the store and devices installed outside the store. The store controller 50 includes at least a CPU, a storage unit, and a communication unit.

[0012] In addition, devices other than the above-mentioned devices may be installed in the store. For example, a registration settlement device 210 (not shown) that executes registration processing and settlement processing based on the operation of a store clerk, a registration device 220 (not shown) that executes registration processing based on the operation of a store clerk, and a settlement device 230 (not shown) that acquires registration information by the registration device 220 and executes settlement processing based on the operation of a customer may be installed. In addition, a settlement device 250 (not shown) that acquires registration information by a mobile registration terminal 240 (not shown) that executes registration processing based on the operation of a customer at a product sales floor and executes settlement processing based on the operation of the customer may be installed. In addition, a charging device 260 (not shown) that charges electronic money may be installed in the store.

[0013] FIG. 1 is an example of the appearance of the registration settlement device 10. The registration settlement device 10 includes a main body 11, an upstream counter 12, and a downstream counter 13. The upstream counter 12 is a counter provided upstream of the main body 11 on the customer flow line. For example, products that have not yet been registered and shopping baskets containing products that have not yet been registered are placed on the upstream counter 12. The downstream counter 13 is a counter provided downstream of the main body 11 on the customer flow line. For example, products that have been registered and shopping bags (my bags, paid shopping bags, etc.) containing products that have been registered are placed on the downstream counter 13. The upstream counter 12 (as well as the downstream counter 13) may be a separate entity attached to the registration settlement device 10 (main body 11).

[0014] A cash payment unit (change machine) 106 is provided at the bottom of the main body 11. A printing unit 108, a scanner unit 104, a display unit 103, a non-cash payment unit (payment terminal) 105, an imaging unit (camera) 110, and a support pole unit 111 are provided at the top of the main body 11. A light emitting unit 112, a monitoring display unit 113, and a detection unit (sensor unit) 120 are provided on the support pole unit 111 in this order from the top.

[0015] FIG. 2 is a configuration example of the registration settlement device 10. Although some of the components have already been mentioned in FIG. 1, the registration settlement device 10 includes a CPU 101, a memory unit 102, a display unit 103, a scanner unit 104, a non-cash settlement unit 105, a cash settlement unit 106, a communication unit 107, a printing unit 108, an audio output unit 109, an imaging unit 110, a light emitting unit 112, a monitoring display unit 113, and a detection unit 120. These can communicate with each other via a bus. Although not shown in the figure, the registration settlement device 10 includes a clock unit (clock function). For example, the registration settlement device 10 can manage the current time, the elapsed time from a certain point in time, and the remaining time until a certain time by using the clock unit. The registration settlement device 10 may include an audio input unit (microphone).

[0016] The CPU 101 is a central processing unit, and controls the operation of the registration and settlement device 10 by reading and executing programs stored in the storage unit 102. For example, the CPU 101 controls the entire transaction from product registration to settlement. In other words, the CPU 101 executes (progresses) the registration process and the settlement process.

[0017] (Generation of transaction information, electronic journal information, and operation log information) CPU 101 generates various information. For example, CPU 101 generates information (hereinafter referred to as "transaction information") indicating transaction details including product information (product registration information) of products registered in each transaction and payment information (information including payment type). CPU 101 also generates transaction history information (hereinafter referred to as "electronic journal information") that records events (hereinafter referred to as "events"; described below) that occurred in each transaction, and operation history information (hereinafter referred to as "operation log information") that records the history of operations (described below) in each transaction. CPU 101 generates transaction information, electronic journal information, and operation log information even without explicit instructions (such as operations from an operator).

[0018] (Events recorded in electronic journal information) Events stored in the electronic journal information include, for example, screen transitions (display / deletion), display / deletion of small screens, scan results (e.g., barcode reading OK, barcode reading NG, non-file (PLU failure)), product registration (product registration by scanning a barcode, product registration using a product preset button), product cancellation, increase / decrease in purchase quantity, selection of payment method (selected payment method), display / deletion of various messages (e.g., errors, alerts, guidance), occurrence / resolution of errors, a predetermined period of no operation where no operation is performed on the display unit 103 or the scanner unit 104 (scanning operation), notification from the non-cash payment unit 105 (described below), notification from the cash payment unit 106 (described below), calling a store clerk, etc.

[0019] Notifications from the non-cash payment unit 105 include, for example, notifications that various waiting states (e.g., waiting for a credit card to be read, waiting for a PIN (Personal Identification Number) to be entered) have continued for more than a predetermined time, notifications that an unauthorized credit card (one that has been reported lost) has been read, notifications that a PIN has been entered, notifications that the PIN entered is incorrect, notifications that credit (authorization) is not allowed, notifications that payment is complete, notifications that a credit card has been left behind, and in the case of electronic money, notifications of insufficient balance.

[0020] Notifications from the cash settlement unit 106 include, for example, notifications that various waiting states (e.g., waiting for cash to be inserted, waiting for change to be removed) have continued for more than a specified period of time, notifications regarding inserted coins (deposits) (that they have been inserted, the number of coins of each denomination), notifications regarding dispensed coins (change) (that they have been dispensed, the number of coins of each denomination), etc.

[0021] That is, for example, the registration settlement device 10 (CPU 101) stores in the electronic journal information, when screen A is displayed, that screen A has been displayed as an event, when small screen B is displayed, that small screen C has been displayed as an event, when product C is registered using the product preset button, that product C has been registered using the product preset button as an event, when product D is cancelled, that product D has been cancelled as an event, when a store clerk is called, that a store clerk has been called as an event, when a PIN is entered, the registration settlement device 10 stores in the electronic journal information, when a PIN is entered, that a PIN has been entered as an event, when a 5000 yen deposit (one 5000 yen note) is inserted, that a deposit of 5000 yen (one 5000 yen note) is inserted, that a deposit of 5000 yen (one 5000 yen note) is inserted, and that a deposit of 5000 yen (one 5000 yen note) is inserted as an event, when a PIN is entered, the registration settlement device 10 stores in the electronic journal information, when a PIN is entered, that a deposit of 5000 yen (one 5000 yen note) is inserted, that a deposit of 5000 yen (one 5000 yen note) is inserted as an event, when a PIN is entered, the registration settlement device 10 stores in the electronic journal information.

[0022] Furthermore, when the total amount (transaction amount) of registered products (registered products) reaches a predetermined amount (e.g., 10,000 yen) (or exceeds the predetermined amount), the registered settlement device 10 may store in the electronic journal information as an event that the total amount of registered products has reached the predetermined amount (or exceeded the predetermined amount). When the total amount of registered products by product preset buttons reaches a predetermined amount (or exceeds the predetermined amount), the registered settlement device 10 may store in the electronic journal information as an event that the total amount of registered products has reached the predetermined amount (or exceeded the predetermined amount). When the number of operations of the product preset buttons reaches a predetermined number (or exceeds the predetermined number), the registered settlement device 10 may store in the electronic journal information as an event that the number of operations of the product preset buttons has reached the predetermined number (or exceeded the predetermined number).

[0023] When the number of cancelled items (the number of items cancelled with the cancel button) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the number of cancelled items has reached (or exceeded) a predetermined number. When the number of items reduced in the purchase quantity (the number of items reduced with the minus button) reaches (or exceeds) a predetermined number, the registered settlement device 10 may store in the electronic journal information as an event that the number of items reduced in the purchase quantity has reached (or exceeded) a predetermined number. When the total amount of cancelled or reduced items (items cancelled with the cancel button or items reduced with the minus button) reaches (or exceeds) a predetermined amount, the registered settlement device 10 may store in the electronic journal information as an event that the total amount of cancelled or reduced items has reached (or exceeded) a predetermined amount.

[0024] When an unregistered product take-home action (described later) occurs, the registered settlement device 10 may store in the electronic journal information as an event that an unregistered product take-home action has occurred. When the number of occurrences (detection number) of unregistered product take-home actions (described later) reaches a predetermined number (or exceeds the predetermined number), the registered settlement device 10 may store in the electronic journal information as an event that the number of occurrences of unregistered product take-home actions (described later) has reached the predetermined number (or exceeded the predetermined number).

[0025] The registered settlement device 10 generates electronic journal information so that it is possible to know at what point (timing) in a transaction an event occurs. For example, the registered settlement device 10 may store in the electronic journal information the current time when a transaction starts and the current time when each event occurs. That is, by referring to the electronic journal information, it may be possible to know that a transaction starts at xx:xx, event a occurs at xx:xx, and event b occurs at xx:xx, △ seconds. The registered settlement device 10 may also store in the electronic journal information the current time when a transaction starts and the elapsed time from the start of a transaction when each event occurs. That is, by referring to the electronic journal information, it may be possible to know that a transaction starts at xx:xx, event a occurs xx seconds after the start of a transaction, and event b occurs △△ seconds after the start of a transaction.

[0026] (Operations that are stored as history in the operation log information) The operations stored as history in the operation log information are operations that can change the display contents of the display unit 103. In other words, the operations stored as history in the operation log information are operations that can change the display contents of the display unit 103. The registration settlement device 10 (CPU 101) stores operations that can change the display contents of the display unit 103 in the operation log information. Specifically, the registration settlement device 10 stores operations on the display unit 103 in the operation log information. In other words, since an operation on the display unit 103 may change the display information within the same screen or transition to another screen (since the display contents may change due to an operation on the display unit 103), the registration settlement device 10 stores operations on the display unit 103 in the operation log information. For example, the registration settlement device 10 stores the type of the touched button, the touch position (coordinates), and the like as operations on the display unit 103 in the operation log information. In addition to cases where the display content of display unit 103 changes due to a touch, even when the display content of display unit 103 does not change due to a touch (for example, when a button does not respond when touched (when a button that cannot be operated is touched) or when a location other than a button is touched), the type of touched button (determined from the touch position), the touch position, etc. may be stored in the operation log information.

[0027] The display contents of the display unit 103 may change depending on an operation on the display unit 103, but the display contents of the display unit 103 may also change depending on other operations. For example, the display contents of the display unit 103 may also change depending on an operation on the scanner unit 104 (scanning operation), an operation on the non-cash payment unit 105 (operation of a button on the non-cash payment unit 105, etc.), or an operation on the cash payment unit 106 (insertion of banknotes or coins, removal of change, etc.). Therefore, in addition to the operation on the display unit 103, the registered settlement apparatus 10 also stores the operation on the scanner unit 104, the operation on the non-cash payment unit 105, and the operation on the cash payment unit 106 in the operation log information.

[0028] The registered settlement device 10 (CPU 101) generates operation log information so that it is possible to know at what time point in a transaction an operation occurs. For example, the registered settlement device 10 may store in the operation log information the current time at the start of a transaction and the current time at the time of each operation. That is, by referring to the operation log information, it may be possible to know that a transaction started at xx:xx, an operation c occurred at xx:xx, and an operation d occurred at xx:xx,△. Also, the registered settlement device 10 may store in the operation log information the current time at the start of a transaction and the elapsed time from the start of a transaction at the time of each operation. That is, by referring to the operation log information, it may be possible to know that a transaction started at xx:xx, an operation c occurred xx seconds after the start of a transaction, and an operation d occurred △△ seconds after the start of a transaction.

[0029] The registered settlement device 10 may store something other than the operation in the operation log information. For example, when the display content of the display unit 103 changes due to something other than the operation (when there is something other than the operation that changes the display content of the display unit 103), the registered settlement device 10 may store something other than the operation that changes the display content of the display unit 103 as the operation log information. In the case of a configuration in which a message (alert, etc.) is displayed on the display unit 103 when a no-operation state in which no operation is performed on the display unit 103 or an operation (scanning operation) on the scanner unit 104 is continued for a predetermined time, the registered settlement device 10 may store in the operation log information the fact that the no-operation state has continued for a predetermined time (event). In other words, the registered settlement device 10 may store in the operation log information a part of the events stored in the electronic journal information. The registered settlement device 10 may store something other than the operation that changes the display content of the display unit 103 (one that does not correspond to an event stored in the electronic journal information) in the operation log information. Operations other than those that change the display content of the display unit 103 may be stored in at least one of the operation log information, the electronic journal information, and the third history information that is different from both the operation log information and the electronic journal information.

[0030] (Generation of display transition images) CPU 101 reproduces the display contents on display unit 103 (more specifically, the transition of the display contents, i.e., the history of changes in the display contents). Specifically, CPU 101 generates an image showing the transition of the display contents on display unit 103 (hereinafter referred to as a display transition image or a display history image). Since the display contents on display unit 103 change in response to an operation by an operator (such as a change in display information within the same screen or a transition to another screen), CPU 101 generates (assembles) a display transition image based on screen information of each screen and operation log information (i.e., an operation actually performed). Since the display transition image is an image expressing the transition of the display contents and is an image generated based on operation log information including a time component, it is a moving image.

[0031] In addition, when the display content of display unit 103 changes due to something other than an operation (for example, when a configuration is used in which a message is displayed on display unit 103 when a no-operation state continues for a predetermined time), and when something other than an operation that changes the display content of display unit 103 (that a no-operation state has continued for a predetermined time) is stored as operation log information, CPU 101 generates a display transition image by referring to the operation log information.

[0032] On the other hand, when the display content of display unit 103 changes due to something other than an operation and operations other than those that change the display content of display unit 103 are not stored as operation log information, CPU 101 generates a display transition image by referring to information that stores operations other than those that change the display content of display unit 103 (e.g., electronic journal information, third history information) in addition to the operation log information.

[0033] In addition, the CPU 101 may generate a display transition image when there is an explicit instruction (a specified operation by a store clerk, for example, an operation by the store clerk on the screens of Figures 5(B), 7(A), and 7(B)), or it may generate a display transition image even without an explicit instruction, as with transaction information, electronic journal information, and operation log information.

[0034] The memory unit 102 is a collective term for one or more various storage components (e.g., memory or storage, such as RAM, ROM, hard disk, SSD (Solid State Drive), etc.). The memory unit 102 stores various information. For example, the memory unit 102 stores programs executed by the CPU 101, information referenced by the CPU 101 (e.g., product master), information generated by the CPU 101 (e.g., transaction information, electronic journal information, operation log information, display transition image), captured images captured by the imaging unit 110, etc.

[0035] The product master is, for example, a file that stores product names, sales prices, etc. in association with product identification information (for example, JAN codes or product codes that are preset for each product preset button).

[0036] CPU 101 stores the transaction information in memory unit 102 so that it can be determined which transaction it corresponds to. For example, CPU 101 may generate transaction information including transaction identification information (transaction number) that is assigned (numbered) for each transaction and store the generated transaction information in memory unit 102, or may generate transaction information that does not include transaction identification information and store the generated transaction information in memory unit 102 in association with the transaction identification information. Similarly, CPU 101 stores electronic journal information and operation log information in memory unit 102 so that it can be determined which transaction it corresponds to. The transaction number is assigned (numbered) when the transaction begins (after operation of registration start button BT01, described below).

[0037] CPU 101 stores the display transition image in storage unit 102 so that it is possible to identify which transaction it is for. Specifically, a display transition image including the transaction identification number of the operation log information that was the basis for generating the display transition image may be generated and the generated display transition image may be stored in storage unit 102, or a display transition image not including the transaction identification information may be generated and the generated display transition image may be stored in storage unit 102 in association with the transaction identification information.

[0038] The CPU 101 stores the captured image captured by the imaging unit 110 in the storage unit 102 so that it is possible to identify which transaction it is. For example, the CPU 101 stores the captured image captured by the imaging unit 110 in the storage unit 102 in association with the transaction identification information.

[0039] That is, the transaction information, electronic journal information, operation log information, display transition images, and captured images are all stored in the storage unit 102 in such a way that it is possible to identify which transaction each of them pertains to.

[0040] The display unit 103 is a touch display that displays various information (e.g., a screen for registration processing, a screen for settlement processing) to an operator (customer) and also accepts operations by the operator. Note that the operation history (touch position) of the operator on the display unit 103 is stored in the storage unit 102 as operation log information, as described above.

[0041] The scanner unit 104 optically reads various pieces of information. For example, the scanner unit 104 scans a code attached to a product and reads the product code.

[0042] The non-cash settlement unit 105 is a settlement mechanism for non-cash settlement (e.g., credit settlement, electronic money settlement, etc.). The non-cash settlement unit 105 may be a settlement terminal separate from the registered settlement apparatus 10. The settlement terminal 105 includes, for example, a reading unit, a communication unit (a communication unit that communicates with the registered settlement apparatus 10, and a communication unit that communicates with an external server such as a credit server that executes credit settlement), an operation unit, and a display unit. The cash settlement unit 106 is a settlement mechanism for cash. The cash settlement unit 106 has an insertion port for banknotes and coins, and an ejection port for banknotes and coins. The cash settlement unit 106 detects currency inserted into the insertion port, identifies the number of coins inserted for each denomination, calculates the inserted amount, calculates the change amount, which is the difference between the inserted amount and the purchase amount, and ejects the change from the ejection port. The insertion of banknotes and coins and the removal of change may be detected by a sensor.

[0043] The communication unit 107 transmits and receives information to and from other devices (e.g., the transaction status management device 40, the store controller 50, etc.). For example, the communication unit 107 receives a product master from the store controller 50 (or the head office server 60). In addition, the communication unit 107 transmits transaction information to the store controller 50 (or the head office server 60).

[0044] The printing unit 108 prints and issues various media (receipts, coupons). A sensor may be used to detect the removal of a medium. The audio output unit 109 outputs audio (for example, a confirmation sound, a warning sound, audio guidance, etc.).

[0045] The imaging unit 110 captures an image of the operator (customer). Specifically, the imaging unit 110 captures an image of the vicinity of the front of the scanner unit 104. That is, the imaging unit 110 can capture an image of a scanning operation by a customer, an operation of inserting cash into the cash settlement unit 106 (or an operation of removing cash from the cash settlement unit 106), and the like. The image captured by the imaging unit 110 may be a still image or a moving image. For example, the imaging unit 110 captures a moving image during the period from the start of the transaction (when the registration start button BT01 is operated) to the end of the transaction (when a receipt is issued or when removal of the receipt is detected) under the control of the CPU 101. The images captured by the imaging unit 110 (still images, moving images) are stored in the storage unit 102 as described above.

[0046] The registration and settlement apparatus 10 may include an imaging unit 1110 (not shown) that images the face of the customer in addition to the imaging unit 110 that images the front area of ​​the scanner unit 104 (scanning operation by the customer, cash insertion operation, etc.). The registration and settlement apparatus 10 may capture a still image of the face of the customer at the start of the transaction (when the registration start button BT01 is operated) as a representative face image by the imaging unit 1110. The registration and settlement apparatus 10 may also include an imaging unit 2110 (not shown) that images the upstream side (the upper surface of the upstream counter 12) and an imaging unit 3110 (not shown) that images the downstream side (the upper surface of the downstream counter 13). The registration and settlement apparatus 10 may include multiple imaging units as described above, but does not image the PIN input area in the non-cash payment unit 105 (or stores it after processing). Like the imaging unit 110, the imaging unit 1110 (the same goes for the imaging units 2110 and 3110) is capable of capturing both moving images and still images.

[0047] The light-emitting unit 112 notifies various information by emitting light. For example, the light-emitting unit 112 notifies that an operation to call a store clerk has been performed by the operator. The light-emitting unit 112 may also notify related to operations such as an operation to cancel a product, an operation to reduce the purchase quantity, an operation of a product preset button, etc. For example, when the number of product cancellations (the same applies to the number of reductions in the purchase quantity and the number of operations of the product preset button) exceeds a predetermined threshold, the light-emitting unit 112 may emit light in a light-emitting mode that means that the number of product cancellations has exceeded the predetermined threshold.

[0048] The monitoring display unit 113 is, for example, a liquid crystal display, and displays various information to the surroundings (the operator, other customers, store clerks). For example, the monitoring display unit 113 may display the number of the device during standby (waiting for a customer), and may display an image (moving image or still image) captured by the imaging unit 110 during processing. The number of the device is indicated on both sides and the back of the monitoring display unit 113.

[0049] The detection unit 120 detects the presence of a person using a human presence sensor (e.g., a sensor using infrared rays). The detection unit 120 may be equipped with multiple human presence sensors with different detection positions. For example, the detection unit 120 may be composed of a first sensor that detects a person in a direction looking down near the front (front and vicinity) of the upstream counter 12 (i.e., detects the presence of a person near the front of the upstream counter 12), a second sensor that detects a person in a direction looking down near the front of the main unit 11 (i.e., detects the presence of a person near the front of the main unit 11), and a third sensor that detects a person in a direction looking down near the front of the downstream counter 13 (i.e., detects the presence of a person near the front of the downstream counter 13).

[0050] (Operation confirmation verification process) As described above, the registration and settlement device 10 can execute an operation confirmation verification process based on the operation of a store clerk. The operation confirmation verification process is a process for confirming whether or not (or whether the operation in the registration process or the settlement process) of a customer is correct (or incorrect), or for verifying whether or not the customer's operation is a correct operation (or an incorrect operation). The registration and settlement device 10 proceeds with the process (registration process, settlement process) based on the customer's operation, i.e., without the intervention of a store clerk, so there is a higher possibility of an incorrect operation (intentional incorrect operation, incorrect operation due to mistake) being performed compared to when the store clerk is involved. As a measure against incorrect operations, the store side (store clerk) executes an operation confirmation verification process as appropriate to verify whether or not the customer's operation is correct. Hereinafter, the function realized (provided by) the operation confirmation verification process may be referred to as an operation confirmation verification function (or an operation confirmation function, or an operation verification function).

[0051] Specifically, the registration settlement device 10 displays, as an operation confirmation verification process (operation confirmation verification function), on the display unit 103, a captured image (moving image) captured by the imaging unit 110 and a display transition image (moving image) generated based on operation log information. For example, the registration settlement device 10 displays the captured image and the display transition image side by side and in sync on the same screen (for example, the small screen SG40 in FIG. 5(C) etc.) based on a predetermined operation by the clerk on a predetermined screen dedicated to the clerk (for example, the small screen SG30 in FIG. 5(B), the small screen SG31 in FIG. 7(A), and the small screen SG32 in FIG. 7(B). Therefore, the clerk (the clerk and the customer when the clerk operates in the presence of the customer) can simultaneously check the captured image and the display transition image displayed in sync with the captured image.

[0052] 3 and 4 are examples of displays on the registration and settlement device 10 (examples of displays on the display unit 103). Specifically, FIGS. 3 and 4 show examples of displays from the start of product registration to the selection of a payment method. FIG. 3(A) is a transaction start screen (standby screen). Before the start of a transaction (while waiting for a customer), the registration and settlement device 10 displays the transaction start screen as shown in FIG. 3(A). A registration start button BT01 for starting a transaction is provided (displayed) on the transaction start screen. In addition, a store clerk call button is provided at the top of the transaction start screen (as are other screens).

[0053] 3(A), since the registration start button BT01 is provided relatively high up on the transaction start screen (close to the image capture unit 110), the customer's line of sight is relatively upward when the customer operates the registration start button BT01. Therefore, when the representative face image is captured, a suitable representative face image (with the line of sight relatively upward) can be obtained.

[0054] When the registration start button BT01 on the transaction start screen is operated, the registration and settlement device 10 displays an initial registration screen (a registration screen before registering one item of product) as shown in FIG. 3(B). The initial registration screen (as well as the registration screen) is provided with a total amount display area HR10 that displays total information (total number of items purchased, total amount), and a product information display area HR11 that displays information on the registered products. The initial registration screen (as well as the registration screen) is also provided with a checkout button BT12 that is operated when proceeding to checkout (settlement) (i.e., when completing registration). Note that the checkout button BT12 is a button that becomes operable after the registration of one item of product, so on the initial registration screen, which is the registration screen before the registration of one item of product, the checkout button BT12 is displayed in a manner indicating that it cannot be operated (grayed out, shown by a dashed line in the figure).

[0055] The initial registration screen (as well as the registration screen) is provided with various buttons (vegetable button, fruit button, fresh fish button, and meat button) that are operated when registering products that do not have barcodes. For example, when the vegetable button is operated, the registration and settlement device 10 displays product preset buttons for registering individual vegetables (cucumber, carrot, etc.) (see FIG. 4(A)). The same applies to the fruit button, fresh fish button, and meat button.

[0056] When the customer scans the barcode of the canned beer from the scene in Fig. 3(B) (initial registration screen), the registration and settlement device 10 displays a registration screen as shown in Fig. 3(C). The product information display area HR11 displays information about the product (canned beer) (product name, quantity, price). The total amount display area HR10 displays the total amount for the product.

[0057] The display column for the product (canned beer) in the product information display area HR11 is provided with a plus button (the button marked "+" in the figure) which is operated to increase the purchase quantity of the product (canned beer), a minus button (the button marked "-" in the figure) which is operated to decrease the purchase quantity of the product, and a cancel button (the button marked "Cancel" in the figure) which is operated to cancel the purchase of the product. Note that the minus button is a button which becomes operable when the purchase quantity is 2 or more, so in the scene in FIG. 3(C), the minus button is displayed in a manner which indicates that it cannot be operated.

[0058] When the vegetable button is operated from the scene (registration screen) of Fig. 3(C), the registration and settlement device 10 displays a small screen SG13 on top of the registration screen as shown in Fig. 4(A). The small screen SG13 is provided with product preset buttons (cucumber button, carrot button, etc.) for registering individual vegetables. Although not shown, the small screen SG13 displayed on top of the registration screen when the fruit button is operated from the scene (registration screen) of Fig. 3(C) is provided with product preset buttons (apple button, mandarin orange button, etc.) for registering individual fruits. The same applies when the fresh fish button or the meat button is operated.

[0059] The small screen SG13 is also provided with a NEXT PRODUCT button that is operated to display the next product (another product preset button) on the small screen SG13, and a CLOSE button that is operated to close (erase) the small screen SG13. The registration and settlement device 10 also closes the small screen SG13 when any of the product preset buttons is operated in addition to the CLOSE button.

[0060] When the product preset button for cucumber is operated from the scene in Figure 4 (A) (registration screen with small screen SG13 displayed in the foreground), the registration and settlement device 10 closes the small screen SG13 as shown in Figure 4 (B), adds information about the product (cucumber) to the product information display area HR11 of the registration screen, and updates the total information in the total amount display area HR10.

[0061] When the billing button BT12 is pressed from the scene (registration screen) of Fig. 4(B), the registration settlement device 10 displays a payment method selection screen (also called a payment type selection screen or a settlement method selection screen) as shown in Fig. 4(C). The payment method selection screen has buttons (cash button, credit button, electronic money button) corresponding to each payment method (cash payment, credit payment, electronic money payment).

[0062] Although not shown, the registered settlement device 10 displays (sequentially in response to the customer's operations, etc.) one or more settlement screens (one or more screens for cash payments, one or more screens for credit payments, one or more screens for electronic money payments) according to the payment method selected on the payment method selection screen (cash payment when the cash button is operated, credit payment when the credit button is operated, or electronic money payment when the electronic money button is operated) and proceeds with the settlement process. The registered settlement device 10 prints and issues a receipt as the final operation in the settlement process (or as the final operation of the transaction after the settlement process is completed).

[0063] The registration settlement device 10 is capable of reading the point card. There are no particular limitations on the timing of reading the point card before the start of registration. For example, the registration settlement device 10 may read the point card before the start of registration (operation of the registration start button BT01 to registration of one product item), may read the point card during product registration (registration of one product item to operation of the checkout button BT12), or may read the point card when selecting a payment method (operation of the checkout button BT12 to operation of one button on the payment method selection screen).

[0064] Fig. 5 is a display example (display example on the display unit 103) of the registration and settlement apparatus 10. Specifically, Fig. 5 shows a display example until a captured image and a display transition image are displayed.

[0065] For example, when the clerk scans the clerk's clerk code from the transaction start screen (FIG. 3(A)), the registered settlement device 10 displays the small screen SG20 in front of the transaction start screen, as shown in FIG. 5(A). The small screen SG20 is a maintenance screen for the clerk. Various buttons are provided on the small screen SG20. For example, the small screen SG20 has a record confirmation button BT21 that is operated to check the record (history).

[0066] The store clerk may monitor a customer who is registering or settling goods using the registered settlement device 10 (by directly monitoring with the naked eye in the checkout area or by monitoring with a monitoring monitor that displays images from a surveillance camera), and if there is anything suspicious about the customer's operation, he or she may move to the registered settlement device 10 (where the customer is) and, after the transaction is completed (or during the transaction), have the registered settlement device 10 read the store clerk code and display the small screen SG20. Also, if the store clerk finds a registered settlement device 10 that is emitting light from the light-emitting unit 112 (i.e., a registered settlement device 10 that is calling a store clerk, or a registered settlement device 10 in which the number of canceled goods or the number of purchase quantities has been reduced or the number of operations of the product preset buttons has exceeded a predetermined threshold), he or she may move to the registered settlement device 10 and, after the transaction is completed (or during the transaction), have the registered settlement device 10 read the store clerk code and display the small screen SG20.

[0067] When the record confirmation button BT21 is operated from the scene in Fig. 5(A) (the transaction start screen with the small screen SG20 displayed in the foreground), the registered settlement device 10 displays the small screen SG30 in front of the transaction start screen in place of the small screen SG20, as shown in Fig. 5(B). The small screen SG30 is a transaction confirmation screen (electronic journal search and viewing screen) for the store clerk. The small screen SG30 is provided with a transaction list display area HR31.

[0068] In the transaction list display area HR31 shown in Fig. 5(B), basic transaction information (described later) of each transaction is displayed in a selectable manner. That is, the registered settlement device 10 accepts touching (selection) of the basic transaction information of the transaction (display area of ​​the basic transaction information) as a selection operation of the transaction. In other words, when the store clerk selects a certain transaction, he / she touches (selects) the basic transaction information of the transaction (display area of ​​the basic transaction information).

[0069] (Basic transaction information) The basic transaction information is basic information about a transaction. The basic transaction information includes, for example, the transaction date and time (e.g., the transaction completion date and time), the receipt number (transaction number), the journal number, and event information within the transaction. The event information within the transaction is information about an event that occurred in the transaction, that is, an event recorded in the electronic journal information of the transaction, and is, for example, information about an event that the transaction amount (total amount) reached 10,000 yen (i.e., information indicating that the transaction amount was 10,000 yen or more), information about an event that a store clerk was called (i.e., information indicating that the transaction involved a store clerk call), information about an event that a product was registered using a product preset button (i.e., information indicating that the transaction involved an operation of the product preset button), information about an event that a product was canceled (i.e., information indicating that the transaction involved a cancellation operation (operation of the cancel button)), information about an event that a purchase quantity was reduced (i.e., information indicating that the transaction involved a reduction operation of the purchase quantity (operation of the minus button)), etc.

[0070] For example, "Man" in Figure 5 (B) indicates that the transaction amount is 10,000 yen or more. "Call" indicates that the transaction involved a call to a store clerk. "P" indicates that the transaction involved an operation of the product preset button. "Take" indicates that the transaction involved a cancellation operation. "Reduce" indicates that the transaction involved a decrease in the purchase quantity.

[0071] In addition, various buttons are provided on the small screen SG30. For example, the small screen SG30 is provided with a display button BT32 operated to display an electronic journal file, an image playback button BT33 operated to play (display) a captured image or a display transition image, and an end button BT34 operated to erase the small screen SG30 (display the small screen SG20 instead of the small screen SG30).

[0072] It is to be noted that the end button BT34 of the small screen SG30 may not be operable while the small screen SG40 (described later) is being displayed. Also, the end button BT34 of the small screen SG30 may be operable while the small screen SG40 is being displayed, and when the end button BT34 is operated while the small screen SG40 is being displayed, the small screen SG40 may be closed (deleted) and the small screen SG20 may be displayed instead of the small screen SG30. Alternatively, when the end button BT34 is operated while the small screen SG40 is being displayed, a confirmation small screen on which a confirmation button and a cancel button are arranged may be further displayed, and when the confirmation button is operated, the confirmation small screen and the small screen SG40 may be closed and the small screen SG20 may be displayed instead of the small screen SG30, and when the cancel button is operated, the confirmation small screen may simply be closed (that is, the operation of the end button BT34 may be canceled).

[0073] When the display button BT32 is operated following the selection of a transaction in the transaction list display area HR31 (i.e., when a transaction is selected), the registered settlement device 10 displays the electronic journal information of the transaction in the transaction list display area HR31 instead of the basic transaction information.

[0074] When the registration settlement device 10 reads a clerk code during a transaction (for example, while the registration screen is displayed), it displays a small screen SG20 (FIG. 5(A)) in front of the registration screen. However, if the record confirmation button BT21 is operated on the small screen SG20, the registration settlement device 10 may display electronic journal information for the transaction (electronic journal information up to a midpoint of the transaction) in the transaction list display area HR31 of the small screen SG30 displayed in place of the small screen SG20. In other words, when the record confirmation button BT21 is operated on the small screen SG20 displayed during a transaction, the registration settlement device 10 may display electronic journal information for the transaction in progress in the transaction list display area HR31 from the beginning (i.e., omitting the transaction selection operation or the operation of the display button BT32).

[0075] Furthermore, when the image playback button BT33 is operated following the selection of a transaction in the transaction list display area HR31, the registered settlement device 10 displays captured images and display transition images of the transaction. Note that the registered settlement device 10 may also display captured images and display transition images of a transaction when the image playback button BT33 is operated while electronic journal information of a transaction (including electronic journal information of a transaction in progress) is displayed in the transaction list display area HR31.

[0076] When a transaction (e.g., a transaction shown in Figures 3 and 4) is selected in the transaction list display area HR31 from the scene in Figure 5 (B) (the transaction start screen with small screen SG30 displayed in the foreground) and the image playback button BT33 is operated, the registered settlement device 10 displays small screen SG40 in the foreground of the transaction start screen, as shown in Figure 5 (C).

[0077] When the image playback button BT33 of the small screen SG30 is operated, the registration settlement device 10 plays the video from the start (beginning) of the transaction on the small screen SG40. When the small screen SG40 is displayed from the small screen SG31 (FIG. 7(A)) or the small screen SG32 (FIG. 7(B)), the video is also played from the start of the transaction. In other words, the video on the small screen SG40 in FIG. 5(C) does not show the playback content immediately after the image playback button BT33 of the small screen SG30 is operated (i.e., when the display of the small screen SG40 starts), but shows the point at which playback has progressed from the start of display (or after the playback point is specified after the display starts).

[0078] When the image playback button BT33 of the small screen SG30 is operated, the registered settlement device 10 may refer to the basic transaction information (such as "P") of the transaction and play (start playback) from a specific point (time point) on the small screen SG40 as shown in Fig. 5(C). Similarly, when displaying the small screen SG40 from the small screen SG31 (Fig. 7(A)) or the small screen SG32 (Fig. 7(B)), playback may be performed from a specific point.

[0079] Fig. 6 is a display example (display example of the small screen SG40) of the registration settlement device 10. Specifically, Fig. 6(A) is an enlarged view of the small screen SG40 shown in Fig. 5(A). The small screen SG40 is an image confirmation screen on which an image is reproduced and confirmed.

[0080] As shown in Fig. 6(A), the registration and settlement device 10 displays (plays) a captured image (symbol A in the same figure) and a display transition image (symbol B in the same figure) side by side on the small screen SG40. The captured image in Fig. 6(A) is an image captured by the imaging unit 110 when two products are registered, and the display transition image in Fig. 6(A) is a display transition image when two products are registered. In other words, the registration and settlement device 10 displays a captured image and a display transition image at the same timing (the timing when two products are registered in Fig. 6(A)) side by side.

[0081] Image GA1 in the display transition image is an image showing the customer's operation position (touch position). When the touch position (coordinates) is included in the operation log information, the registered settlement device 10 generates a display transition image in which the touch position is visualized by image GA1 (displaying image GA1 at the touch position). Note that the registered settlement device 10 may display image GA1 so that the center of image GA1 is the touch position (i.e., image GA1 may be displayed with the touch position at the center). Note that while image GA1 in the figure is simply a circular image, image GA1 may also be an image that evokes a touch (for example, a fingerprint-like image, a fingertip-like image).

[0082] According to FIG. 6(A), it can be seen that, although carrots (¥70) should have been registered, cucumbers (¥50) were actually registered. It can also be seen that the customer operated (touched) the upper right part of the product preset button for cucumbers. Note that since the display transition image in FIG. 6(A) shows (is being played back) the moment when the customer picked up a carrot and operated the product preset button for cucumbers (upper right part) on the small screen SG13, the registration information for cucumbers is not reflected (displayed) in the display transition image (product information display area HR11 of the registration screen displayed as the display transition image), but in the display transition image (not shown) immediately afterwards, the small screen SG13 is erased and the registration information for cucumbers is reflected.

[0083] The small screen SG40 is also provided with a playback bar (symbol C in the figure) that indicates the playback location (playback time point) of the video (captured image, display transition image). When a location on the playback bar is operated (touched), the registration and settlement device 10 plays the video from that location (i.e., changes the playback location). In other words, the store clerk can check the playback location of the video on the playback bar, and can operate the desired location on the playback bar to change (adjust) the playback location.

[0084] The lower part of the playback bar displays the location (time) of the event (start, registering 1st item, registering 2nd item (P), cash selection, ...) that occurred in the transaction. The "S" in "Registering 1st item S" indicates that the registration was made by scanning a barcode. In other words, "Registering 1st item S" indicates that (the location) the first item was registered by scanning a barcode. The "P" in "Registering 2nd item P" indicates that the registration was made by pressing a product preset button. In other words, "Registering 2nd item P" indicates that (the location) the second item was registered by pressing a product preset button. The information in the lower part of the playback bar is useful when checking or changing the playback location. The events shown in FIG. 6(A) are just an example, and other events (e.g., a cancel operation, a purchase quantity reduction operation, etc.) may be displayed. The location of the event may be displayed in the upper part of the playback bar instead of or in addition to the lower part of the playback bar.

[0085] Instead of or in addition to displaying the location of an event in association with the playback bar (displaying the location of the event at the bottom or top of the playback bar), buttons corresponding to each event (event-related buttons) may be provided. In the example shown in FIG. 6(A), the event-related buttons may be arranged on the small screen SG40 as a first-item registration S button, a second-item registration P button, a cash selection button, and a cash (300 yen) insertion button. When a button is operated, the registration settlement device 10 starts playing the video from the corresponding location. For example, when the second-item registration P button is operated, the registration settlement device 10 starts playing the video from the location where the second item is registered by the product preset button.

[0086] The small screen SG40 is also provided with a play (forward) button, a stop button, a fast-forward button, a rewind button, and a fast-rewind button. The play button is a button operated when playing and displaying a moving image at a normal speed (the speed at which real time passes). The stop button is a button operated when pausing and displaying a moving image (displaying it like a still image). The fast-forward button is a button operated when playing and displaying a moving image at a speed faster than normal. The rewind button is a button operated when playing and displaying a moving image in reverse at a speed faster than normal. Note that when the image play button BT33 on the small screen SG30 is operated (i.e., when the display of the small screen SG40 starts), the registered settlement device 10 may play and display the moving image from the beginning (or a specific location) at the normal speed, or may stop and display the beginning (or a specific location) of the moving image.

[0087] Further, the small screen SG40 is provided with a simultaneous screen (simultaneous image) button BT41, a monitoring screen (captured image) button BT42, an operation screen (display transition image) button BT43, and a close button BT44. The close button BT44 is a button that is operated when closing (erasing) the small screen SG40.

[0088] The simultaneous screen button BT41, the monitoring screen button BT42, and the operation screen button BT43 are buttons that accept a switching operation of the image display and notify the current display state. The simultaneous screen button BT41 is a button that is operated when a captured image and a display transition image are displayed simultaneously. The monitoring screen button BT42 is a button that is operated when a captured image is displayed without displaying a display transition image. The operation screen button BT43 is a button that is operated when a display transition image is displayed without displaying a captured image. For example, when the monitoring screen button BT42 is operated from the scene of FIG. 6(A), the registered settlement device 10 displays the captured image without displaying the display transition image (not shown). When the operation screen button BT43 is operated from the scene of FIG. 6(A), the registered settlement device 10 displays the display transition image without displaying the captured image, as shown in FIG. 6(B). Switching is possible even during playback.

[0089] In the scene of Fig. 6(A), the simultaneous screen button BT41 is selected, so in Fig. 6(A), the simultaneous screen button BT41 is displayed in a display manner (bold frame in the figure) indicating that it is selected. In the scene of Fig. 6(B), the operation screen button BT43 is selected, so in Fig. 6(B), the operation screen button BT43 is displayed in a display manner indicating that it is selected.

[0090] A mode in which both the captured image and the display transition image are played back (displayed) may be called a simultaneous playback mode, a mode in which the captured image is played back without playing back the display transition image may be called a captured image playback mode, and a mode in which the display transition image is played back without playing back the captured image may be called a display transition image playback mode. In other words, the simultaneous screen button BT41, the monitoring screen button BT42, and the operation screen button BT43 can be said to be mode switching buttons for switching between the simultaneous playback mode, the captured image playback mode, and the display transition image playback mode.

[0091] (Other screens) The registration settlement device 10 may display a small screen SG31 as shown in FIG. 7(A) instead of the small screen SG30 (FIG. 5(B)). In other words, when the record confirmation button BT21 of the small screen SG20 (FIG. 5(A)) is operated from the scene of FIG. 5(A), the registration settlement device 10 may display the small screen SG31 in front of the transaction start screen instead of the small screen SG20 as shown in FIG. 7(A). The small screen SG31 is a screen for specifying a receipt number (transaction number) to identify one transaction. When an operation for specifying a receipt number is performed on the small screen SG31 (that is, when one transaction is identified), the registration settlement device 10 displays a small screen SG40 at the top of the small screen SG31, and displays an image of the transaction (captured image, display transition image) on the small screen SG40.

[0092] The registration settlement device 10 may display the small screen SG32 instead of the small screen SG30 (FIG. 5(B)) or the small screen SG31 (FIG. 7(A)). In other words, when the record confirmation button BT21 of the small screen SG20 (FIG. 5(A)) is operated from the scene of FIG. 5(A), the registration settlement device 10 may display the small screen SG32 in front of the transaction start screen instead of the small screen SG20. The small screen SG32 is a screen for extracting one or more transactions (basic transaction information) by specifying a time. When an operation for specifying a time is performed on the small screen SG32, the registration settlement device 10 displays one or more pieces of basic transaction information within the specified time range in a selectable manner. The small screen SG32 shown in FIG. 7(B) is not the small screen SG32 immediately after display (immediately after the record confirmation button BT21 is operated), but the small screen SG32 displaying one or more pieces of basic transaction information in a selectable manner. When a transaction (basic transaction information) is selected on the small screen SG32 and the image playback button BT33 is operated, the registered settlement device 10 displays a small screen SG40 at the top of the small screen SG32 and displays an image of the transaction (captured image, display transition image) on the small screen SG40. When a transaction (basic transaction information) is selected on the small screen SG32 and the display button BT32 is operated, the registered settlement device 10 erases the basic transaction information being displayed and displays the electronic journal information of the transaction.

[0093] In addition, the small screen SG30 (FIG. 5(B)), the small screen SG31 (FIG. 7(A)), and the small screen SG31 (FIG. 7(B)) may be capable of being switched between displays. For example, a button for switching to another screen may be provided on each of the small screen SG30, the small screen SG31, and the small screen SG32.

[0094] The embodiment has been described above, but according to the registration and settlement device 10 of the above embodiment, the operation status can be preferably confirmed. That is, the operation status confirmation work can be preferably performed.

[0095] Conventionally, the situation of erroneous operation or erroneous deposit (wrong currency) was confirmed by images captured by a surveillance camera, and it may take time to grasp the situation or the situation may not be grasped accurately using only the images captured by the surveillance camera, but in the registration and settlement device 10 of the above embodiment, the captured image and the display transition image (display history image) are displayed (played) on the same screen at the same time, so the situation can be grasped easily (without taking time) and accurately. Furthermore, since the registration and settlement device 10 notifies the touch position in the display transition image, when an erroneous operation or the like is discovered, it is possible to guess to some extent whether it was a simple operational mistake or an intentional operation (an operation suspected of being fraudulent).

[0096] Therefore, for example, if a store clerk and a customer check together and it is found that there is no problem on the customer's side, the store (store clerk) can accept the situation and take appropriate action (for example, by apologizing). On the other hand, if it is found that there is a problem on the customer's side, it is possible to smoothly proceed with corrective measures. In other words, it is unlikely that a situation will arise in which measures are taken without dispelling the customer's distrust (or leaving room for the customer to argue), and work can proceed smoothly. In other words, regardless of the results of the check, because the degree of reproducibility of the situation is high, both the store clerk and the customer are convinced, and work can proceed smoothly.

[0097] The registered settlement device 10 is also capable of dealing effectively with problems such as pressing the wrong product preset button (intentionally pressing the button for a product cheaper than the product to be purchased), money deposit problems (for example, mistakenly thinking (or intentionally claiming) that a 10,000 yen note was inserted when in fact a 5,000 yen note was inserted, and then complaining that there is insufficient change), and also with the taking home of unregistered products (see below).

[0098] (Taking away unregistered products) The act of taking away unregistered products (also referred to as the act of taking away unpaid products) is the act of taking away products without settling (i.e., without paying for them). Examples of the act of taking away unregistered products include the act of moving products from the upstream counter 12 to the downstream counter 13 without scanning (putting products from a shopping basket into a shopping bag without even pretending to scan), the act of moving products from the upstream counter 12 to the downstream counter 13 by intentionally performing an insufficient (inappropriate) scanning operation so that the product's barcode is not read (putting products from a shopping basket into a shopping bag while disguising a scanning operation), the act of moving a set product from the upstream counter 12 to the downstream counter 13 by unintentionally or intentionally reading the barcode of a single product (one can of beer) instead of the barcode of a set product (e.g., a six-pack of canned beer), and the act of moving a product from the upstream counter 12 to the downstream counter 13 without realizing that the scanning operation has simply failed without any intention (simple scanning error).

[0099] By grasping the status of product movement (for example, grasping the status of product movement from image recognition of the image captured by the imaging unit 110 or the detection results of the detection unit 112) and grasping the registration status before and after the product movement (a situation in which the product has not been added), it is generally possible to recognize whether an unregistered product has been taken home; however, by playing back and checking the captured images and display transition images, it is possible to recognize more accurately and easily.

[0100] (Variations, etc.) Although the embodiment has been described above, the device configuration, data configuration, processing flow, display and output aspects, etc. can be appropriately changed, for example, as shown in the following (1) to (18). In addition, the following (1) to (18) may be appropriately combined with each other.

[0101] (1) The size of the image GA1 (circular image, fingerprint-like image, fingertip-like image, etc.) may be changeable even during playback or stop of the display transition image. For example, an enlargement button and a reduction button may be arranged on the small screen SG40, and the registration settlement device 10 may enlarge and display the image GA1 with the touch position as the center when the enlargement button is operated, and may reduce and display the image GA1 with the touch position as the center when the reduction button is operated. If the display size of the image GA1 is large, the image GA1 stands out even during playback display, so that it is possible to immediately recognize that a touch has occurred even during playback display. On the other hand, if the display size of the image GA1 is small, it is possible to accurately confirm the touch position even when various buttons are densely arranged. In addition, the initial display size of the image GA1 (the display size before the operation of the enlargement button or reduction button) may be set.

[0102] (2) The coordinate information (X coordinate value, Y coordinate value) of the touch position may be displayed in the display transition image. For example, the registration settlement device 10 may display the coordinate information of the touch position near (at the top or bottom) the image GA1 showing the touch position. This gives the customer the impression that the touch operation is properly memorized (recorded).

[0103] (3) The display / non-display of the image GA1 may be made switchable. For example, a display (ON) button and a non-display (OFF) button may be arranged on the small screen SG40, and the registration settlement device 10 may display the image GA1 when the display button is operated, and may not display the image GA1 when the non-display button is operated. Similarly, the display / non-display of the coordinate information in (2) above may be made switchable.

[0104] (4) The touch position may be notified immediately when a touch operation is performed. For example, a button that can be switched to a display mode that immediately notifies the touch position (touch position immediate display mode) may be arranged on the small screen SG40, and the registration settlement device 10 may switch to the touch position immediate display mode when the button is operated. In the touch position immediate display mode, the registration settlement device 10 may display a simulated screen for operation confirmation (e.g., a simulated registration screen, a simulated settlement screen, etc.), and when a touch operation is performed on the simulated screen, the image GA1 may be displayed at the touch position at the timing of the touch operation (while the touch is being performed). For example, the store clerk may switch to the touch position immediate display mode as necessary (e.g., when the customer is not satisfied with the touch position), ask the customer to touch the simulated screen at any position, and show that the image GA1 is displayed at the position where the customer actually touched. This allows for smooth handling.

[0105] (5) The screens for selecting a transaction (small screens SG30, SG31, SG32) may be erased, and a screen (small screen SG40) displaying a moving image (captured image, display transition image) may be displayed. That is, in the above embodiment, an example has been described in which, when a transaction is selected on the screen for selecting a transaction, the registered settlement device 10 maintains the display of the screen for selecting a transaction and displays a new screen displaying a moving image of the transaction, but, when a transaction is selected on the screen for selecting a transaction, the registered settlement device 10 may erase the screen for selecting a transaction and display a screen displaying a moving image of the transaction.

[0106] (6) The screen for selecting a transaction (small screen SG30, small screen SG31, small screen SG32) and the screen for displaying a moving image (small screen SG40) may be a common screen. In other words, in the above embodiment, an example has been described in which, when a transaction is selected on the screen for selecting a transaction, the registered settlement device 10 displays a moving image of the transaction on a screen (small screen SG40) different from the screen for selecting a transaction (small screen SG30, small screen SG31, small screen SG32). However, when a transaction is selected on the screen for selecting a transaction, the registered settlement device 10 may display a moving image of the transaction on the screen for selecting the transaction.

[0107] (7) Product preset buttons may be displayed on a normal screen. In other words, in the above embodiment, the registration and settlement device 10 has been described as displaying a small screen SG13 on which product preset buttons for individual vegetables are arranged in front of the registration screen when the vegetable button on the registration screen is operated. However, the registration and settlement device 10 may display a screen (normal screen) on which product preset buttons for individual vegetables are arranged instead of the registration screen when the vegetable button on the registration screen is operated. Since it is possible to arrange the product preset buttons at a certain distance from each other, touching mistakes of the product preset buttons can be reduced, or it can be made difficult to claim that a touching mistake occurred. Note that when the product preset button or the back button is operated, it is sufficient to return to the registration screen.

[0108] (8) A screen in which a store clerk is involved may be displayed as a normal screen. In other words, in the above embodiment, the registered settlement device 10 has been described as displaying a screen in which a store clerk is involved on a small screen (e.g., small screen SG20, small screen SG30, small screen SG31, small screen SG32, small screen SG40), but it may be displayed on a normal screen. For example, the registered settlement device 10 may display a moving image (captured image, display transition image) on a normal screen.

[0109] (9) The captured image may be a moving image including sound. The same applies to the display transition image. Buttons related to sound (for example, a sound ON button, a sound OFF button, and a volume button) may be arranged on the small screen SG40. When a captured image including sound and a display transition image including sound are played back simultaneously, the sound of one of them may be muted (or the sound output may be significantly reduced compared to the other).

[0110] (10) The display transition image may be a moving image obtained by recording (storing) the display contents on the display unit 103. That is, in the above embodiment, it has been described that the registration settlement device 10 generates the display transition image based on the operation log information, but the display transition image may be generated by recording the display contents on the display unit 103 (that is, the display transition image may be a recording of the display contents).

[0111] When the display transition image is a moving image generated by recording the display contents, the image GA1 is displayed in the display transition image in the same manner as when the display transition image is a moving image generated based on the operation log information. For example, the registration settlement device 10 may superimpose the image GA1 based on the operation log information on the moving image recorded of the display contents on the display unit 103, store it as a display transition image, and display it, or may store the moving image recorded of the display contents on the display unit 103 as a display transition image, and superimpose the image GA1 based on the operation log information on the stored display transition image. In other words, the image in which the image GA1 is superimposed on the moving image recorded of the display contents may be referred to as the display transition image, or the moving image recorded of the display contents (the moving image without the image GA1 superimposed) may be referred to as the display transition image.

[0112] (11) The registered settlement device 10 does not need to have either the non-cash payment unit (payment terminal) 105 or the cash payment unit 106 (it is sufficient to have either one of them).

[0113] (12) The moving image (captured image, display transition image) may be stored in another device. In other words, in the above embodiment, an example was described in which the registered settlement device 10 stores the moving image of the registered settlement device 10 itself (moving image related to a transaction (customer) in the registered settlement device 10) in the memory unit 102 of the registered settlement device 10 itself, but the registered settlement device 10 may store the moving image of the registered settlement device 10 itself in another device (e.g., the transaction status management device 40, the store controller 50, another registered settlement device 10 (representative registered settlement device 10), the headquarters server 60). For example, the registered settlement device 10 that has generated the moving image may transmit the generated moving image to another device, and the other device that is the receiving side may store the moving image, or the registered settlement device 10 that has generated the moving image may access another device and store the generated moving image in the memory unit of the other device.

[0114] (12-1) When there is only one registered settlement device 10 in the sales system 1 (i.e., when there is only one device to be monitored (device that generates video images)), the video images are stored in another device so that it is possible to identify which transaction the video images are for.

[0115] (Images are stored in other devices / One device is monitored) The registered settlement device 10 transmits a captured image of a certain transaction together with transaction identification information of the transaction to another device, and the other device, which is the receiving side, stores the captured image in association with the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the captured image of a certain transaction in a memory unit of the other device in association with the transaction identification information of the transaction.

[0116] In addition, the information name of the captured image (data name, file name) or the storage location of the captured image (another device, folder, path) may be made different (separate) for each transaction so that it is possible to identify which transaction the captured image refers to.

[0117] (Save the display transition image to other devices / Monitor only one device) It has been explained that the registration and settlement device 10 may generate a display transition image that includes a transaction identification number, or may generate a display transition image that does not include transaction identification information. In the former case (when the display transition image includes transaction identification information), the registration and settlement device 10 simply transmits a display transition image of a certain transaction to another device, and the other device, which is the receiving side, simply stores the display transition image. Alternatively, the registration and settlement device 10 accesses the other device and simply stores the display transition image of a certain transaction in the memory unit of the other device.

[0118] On the other hand, in the latter case (when the display transition image does not include transaction identification information), the registered settlement device 10 transmits the display transition image of a certain transaction together with the transaction identification information of the transaction to another device, and the other device, which is the receiving side, stores the display transition image in association with the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image of a certain transaction in the memory unit of the other device in association with the transaction identification information of the transaction. Note that the information name of the display transition image or the storage location of the display transition image may be different for each transaction so that it is possible to identify which transaction the display transition image is for.

[0119] (12-2) If there are two or more registered settlement devices 10 in the sales system 1 (i.e., if there are two or more devices to be monitored), the video image is stored in another device so that it is clear which registered settlement device 10 and which transaction it is for.

[0120] (Store captured images in other devices / Monitor two or more devices) The registered settlement device 10 transmits a captured image of a transaction together with device identification information for identifying the registered settlement device 10 and transaction identification information of the transaction to another device, and the other device that receives the image stores the captured image in association with the device identification information and transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the captured image in a storage unit of the other device in association with the device identification information of the registered settlement device 10 and the transaction identification information of the transaction. Note that the information name of the captured image or the storage location of the captured image may be different for each registered settlement device 10 and each transaction, so that it is possible to know which registered settlement device 10 and which transaction the captured image is for.

[0121] (Save the display transition image to another device / Monitor two or more devices) When the display transition image includes transaction identification information, the registered settlement device 10 transmits the display transition image of a certain transaction to another device together with device identification information that identifies the registered settlement device 10, and the other device that is the receiving side stores the display transition image in association with the device identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image in the memory unit of the other device in association with the device identification information of the registered settlement device 10. Note that the information name of the device identification information or the storage location of the device identification information may be different for each registered settlement device 10 so that it is possible to know which registered settlement device 10 the device identification information belongs to.

[0122] On the other hand, if the display transition image does not include transaction identification information, the registered settlement device 10 transmits the display transition image of a certain transaction to another device together with device identification information for identifying the registered settlement device 10 and the transaction identification information of the transaction, and the other device, which is the receiving side, stores the display transition image in association with the device identification information and the transaction identification information. Alternatively, the registered settlement device 10 accesses the other device and stores the display transition image of a certain transaction in the memory of the other device in association with the device identification information of its own device and the transaction identification information of the transaction. Note that the information name of the display transition image or the storage location of the display transition image may be different for each registered settlement device 10 and each transaction, so that it is possible to know which registered settlement device 10 and which transaction the display transition image belongs to.

[0123] The captured image may include both device identification information and transaction identification information, may not include device identification information but includes transaction identification information, or may not include transaction identification information but includes device identification information. For a captured image that includes device identification information and transaction identification information, both the device and the transaction may be identified from the captured image itself (neither device identification information nor transaction identification information may be stored in association). For a captured image that does not include device identification information but includes transaction identification information, the transaction may be identified from the captured image itself (neither device identification information nor transaction identification information may be stored in association). For a captured image that does not include transaction identification information but includes device identification information, the device may be identified from the captured image itself (neither device identification information nor transaction identification information may be stored in association). The same applies to the display transition image.

[0124] (13) A moving image in a certain registered settlement device 10 may be displayed (played back) on a device other than the registered settlement device 10 (for example, a transaction status management device 40, another registered settlement device 10). Note that while a moving image may be transmitted when it is stored (above (12)), the moving image may also be transmitted when it is displayed when the device that displays the moving image and the device that stores the moving image are different.

[0125] (13-1) When there is one registered settlement device 10 in the sales system 1 (i.e., when there is one device to be monitored), a transaction is specified and a moving image is displayed. For example, a screen (e.g., small screen SG30, small screen 31, small screen SG32) on which a transaction can be specified (i.e., a transaction identification number can be specified) may be displayed, and the specified moving image may be displayed on that screen. The above-mentioned screen on which a transaction can be specified may have a search function or a list display function, like small screen SG30 and small screen SG32.

[0126] (13-2) When there are two or more registered settlement devices 10 in the sales system 1 (i.e., when there are two or more devices to be monitored), a device and a transaction are specified and a video is displayed. For example, a screen on which a device and a transaction can be specified (i.e., device identification information and a transaction identification number can be specified) may be displayed, and the specified video may be displayed on the screen. The above-mentioned screen on which a transaction can be specified may be equipped with a search function and a list display function.

[0127] (14) A device other than the registered settlement device 10 may be monitored. In other words, in the above embodiment, an example of acquiring a moving image (captured image, display transition image) of the registered settlement device 10 capable of executing the registration process and the settlement process has been described, but a moving image of another device may be acquired. For example, a moving image of the settlement device 230 (not shown), the settlement device 250 (not shown), or the charge device 260 may be acquired. A label issuing device 270 (not shown) that selects products, prints and issues labels may be installed in the store, and a moving image of the label issuing device 270 may be acquired. A weighing device 280 (not shown) that weighs products may be installed in the store, and a moving image of the weighing device 280 may be acquired. The label issuing device 270 may have an object scanning function and may display (or identify) candidate products by object scanning in a selectable manner. The registered settlement device 10, the settlement device 230, the settlement device 250, and the weighing device 280 may also have an object scanning function. Weighing device 280 may also have a label printing and issuing function. Furthermore, a ticket vending machine 290 (not shown) may be installed in the store, and video images of ticket vending machine 290 may be acquired. Ticket vending machine 290 may be one that is installed in a restaurant or the like. A self-service payment device installed in a restaurant or the like may also be subject to monitoring (video images of a self-service payment device installed in a restaurant or the like may be acquired).

[0128] In relation to the above (12) to (14), the device that generates the video (the device to be monitored), the device that stores the video, and the device that displays (plays) the video may all be the same device, or may all be different devices. For example, the video of device A (e.g., the registered settlement device 10, the settlement device 230, the settlement device 250, the charge device 260, the label issuing device 270, the weighing device 280, the ticket vending machine 290, etc.) may be stored in the device A and displayed in the device A or device B (e.g., the transaction status management device 40), or the video of device A may be stored in device B and displayed in device B or device C (e.g., the registered settlement device 10, the settlement device 230, the settlement device 250, the charge device 260, the label issuing device 270, the weighing device 280, the ticket vending machine 290, etc., other than device A). When there are two or more devices to be monitored, the device that displays the video may accept the designation of the device and the transaction, and display the designated video of the designated device.

[0129] (15) A moving image of another imaging unit (camera) may be displayed (played). That is, in the above embodiment, an example of displaying an image captured by the imaging unit 110 has been described, but an image captured by another imaging unit (for example, imaging unit 1110, imaging unit 2110, imaging unit 3110) may be displayed. One image selected from images captured by a plurality of imaging units (for example, imaging unit 110, imaging unit 1110, imaging unit 2110, imaging unit 3110) may be displayed, or a plurality of display areas for captured images may be provided to simultaneously display a plurality of captured images. That is, one or more captured images captured in each area (area in which a customer can be captured, area in which a customer's operation (scanning operation, screen operation, deposit operation, movement of a commodity) can be captured, area in which an upstream side (before movement) commodity can be captured, area in which a downstream side (after movement) commodity can be captured) and a display transition image may be displayed. Note that a selection button (or a switching button) may be provided to accept a selection (switching) operation of displaying one captured image / displaying a plurality of captured images. Furthermore, when one captured image is displayed, a button that allows the captured image to be selected (switched) to be displayed may be provided. The same applies when multiple captured images are displayed (for example, when two of three are displayed). Furthermore, the captured images (number, type) to be displayed may be automatically switched not by operation but by settings (a camera image with a suitable shooting accuracy according to the event may be set in advance for each event), or may be automatically switched using AI.

[0130] An externally acquired captured image may be displayed. For example, at least one of the imaging units 110, 1110, 2110, and 3110 may not be an imaging unit included in the registered settlement device 10, but may be an imaging unit (a camera installed on a ceiling, wall, or pillar) separate from the registered settlement device 10.

[0131] (16) Still images may be used for confirmation instead of moving images. In other words, in the above embodiment, an example has been described in which a captured image or a display transition image is displayed as a moving image on the image display screen (small screen SG40), but a captured image or a display transition image may be displayed as a still image. As in the case of moving images, both a captured image of a still image and a display transition image of a still image may be displayed (simultaneous playback mode), or either one of them may be displayed (captured image playback mode, display transition image playback mode).

[0132] A device that displays still images (such as the registration and settlement device 10 and the transaction status management device 40) may display a still image (captured image, display transition image) at the time an event occurs. For example, a device that has a next button and a previous button arranged on the image display screen and displays still images may display a still image of the next event occurring in response to operation of the next button, and a still image of the previous event occurring in response to operation of the previous button. Also, a device that has event-corresponding buttons arranged on the image display screen and displays still images may display a still image of the event occurring at the time of the event corresponding to the operated event-corresponding button in response to operation of the event-corresponding button.

[0133] The display of moving images may also be controlled by a next button or a previous button. That is, a device that displays moving images may display a moving image of the next event occurring (skip to the next event time) in response to the operation of the next button, and display a moving image of the previous event occurring (skip to the previous event time (backskip)) in response to the operation of the previous button.

[0134] (Generating still images) The monitored devices (registered settlement devices 10, settlement devices 230, 250, charging devices 260, label issuing devices 270, weighing devices 280, ticket vending machines 290, etc.) may generate still images from the captured images of the moving images after generating the captured images of the moving images, or may generate the captured images as still images from the beginning without generating the moving images. The same applies to the display transition images.

[0135] In the former case (when a captured image of an image is generated and then a captured image of a still image is generated from the captured image of a moving image), a still image at the time when an event occurs may be extracted from the moving image. The same applies to the display transition image. The time when an event occurs may be specified by referring to electronic journal information.

[0136] In the latter case (when capturing images is generated as still images from the beginning without generating moving images), still images may be generated continuously and a still image at the time of occurrence of the event may be obtained (still images at other times may be discarded). For the display transition image, a still image at the time of occurrence of the event may be generated by referring to operation log information, etc. The time of occurrence of the event may be identified by recognizing the occurrence of the event itself, or by referring to electronic journal information.

[0137] (Motion image mode, Still image mode) The camera may have a moving image mode in which the captured image and the display transition image are confirmed (displayed) as moving images, and a moving image mode in which the captured image and the display transition image are confirmed as still images, and these modes may be switchable by a button, etc. Also, the camera may be configured so that one of the captured image and the display transition image is confirmed as a moving image, and the other is confirmed as a moving image.

[0138] (17) Although the explanation has been omitted, events to be stored in the electronic journal information, events to be included in the basic transaction information (events displayed as "ten thousand" or "P" in the transaction list display area HR31), and events to be used as headings when displaying captured images and display transition images (events displayed at the bottom (or top) of the playback bar, events arranged as event-related buttons) are set in advance. In other words, N events (event 1, event 2, ..., event N) are set as events to be stored in the electronic journal information, and M of the N events is A events are set as events to be included in the basic transaction information, and M B M events are set as headline events. A and M. B may be the same or different numbers. Also, MA and M. B Even if the number of events is the same, the events to be set may be different.

[0139] The above settings may be changeable as appropriate (for example, reconfigurable from a maintenance screen). In other words, the events to be stored as electronic journal information, the events to be included in the basic transaction information, and the events to be used as headings may be changeable as appropriate.

[0140] (18) Some or all of the functions (input / output, storage, processing (including judgment)) of each device (registered settlement device 10, transaction status management device 40, settlement device 230, settlement device 250, charging device 260, label issuing device 270, weighing device 280, ticket vending machine 290, etc.) may be realized in a device other than the device described as the entity that executes the function. Each device may function as a so-called thin client specialized for input / output interfaces, etc. For example, a cloud server may execute a program for controlling the operation of each device, and each device may display a screen using a browsing function, etc.

[0141] <Overview of the embodiment> [Technical field] The present invention relates to an information processing device and a program. [Background technology] There is known a cash register that displays a video of the operation status captured by a camera as a function for checking (inspecting) whether or not an unauthorized operation has been performed (see, for example, Patent Document 1). [Prior art document] [Patent documents] [Patent Document 1] JP 2018-120635 A [Summary of the invention] [Problem to be solved by the invention] However, there is room for improvement in the method for checking the operation status using the conventional technology as shown in Patent Document 1. The present invention has been made in consideration of such circumstances, and has an object to provide a technology for conveniently checking the operation status. [Means for solving the problem]

[0142] (1) One aspect of the invention for solving the above-mentioned problems is an information processing device (e.g., the registration settlement device 10, the settlement device 230, the settlement device 250, the charge device 260, the label issuing device 270, the weighing device 280, the ticket vending machine 290) that performs processing based on the operation of a customer, and which has imaging means (e.g., the imaging unit 110, the imaging device 220, the imaging device 230, the imaging device 250, the charging device 260, the label issuing device 270, the weighing device 280, the ticket vending machine 290) that images predetermined areas (e.g., an area in which the customer can be imaged, an area in which the customer's operation can be imaged, an area in which the upstream side (before movement) products can be imaged, an area in which the downstream side (after movement) products can be imaged). The information processing device is characterized in that it comprises an image capturing unit (imaging unit 1110, imaging unit 2110, imaging unit 3110) and a display means (e.g., display unit 103) that can accept touch operations and display information according to the progress of processing, and the display means displays an image captured by the imaging means (moving image or still image) and a display transition image (moving image or still image) showing the transition of the display content by the display means on the same screen by matching the time of capture with the time of display (e.g., see FIG. 5(C)). According to the information processing device of (1), the operation status can be conveniently confirmed.

[0143] (2) The information processing device described in (1) is characterized in that it is equipped with a log information acquisition means for acquiring operation log information (e.g., generation of operation log information by CPU 101), and the display means displays, as the display transition image, an image generated based on the log information acquired by the log information acquisition means. According to the information processing device of (2), it is possible to appropriately generate a display transition image based on an operation while suppressing capacity.

[0144] (3) The information processing device described in (2) is characterized in that the log information acquisition means acquires log information of operations including touch operations, and the display means displays the display transition image that notifies the position of the touch operation (e.g., by image GA1). According to the information processing device of (3), it is possible to easily check which position has been operated.

[0145] (4) The information processing device described in (1) is characterized in that it is provided with a log information acquisition means for acquiring log information of operations including touch operations, and the display means displays touch position information indicating the position of the touch operation acquired by the log information acquisition means by superimposing it on the display transition image. According to the information processing device of (4), it is possible to easily check which position has been operated.

[0146] (5) An information processing device according to any one of (2) to (4), characterized in that the captured image and the display transition image are displayed at the time of occurrence of an event that occurs between the start and end of processing and is specified by an operation (for example, an event specified by an event-corresponding button, the next event specified by a next button, or the previous event specified by a previous button). According to the information processing device of (5), the necessary parts can be conveniently confirmed.

[0147] (6) An information processing device described in any of (1) to (4), characterized in that it is provided with a switching means for switching between a first mode (simultaneous playback mode) in which the display means displays the captured image and the display transition image, a second mode (captured image playback mode) in which the display means displays the captured image without displaying the display transition image, and a third mode (display transition image playback mode) in which the display means displays the display transition image without displaying the captured image. According to the information processing device of (6), the display can be appropriately switched depending on the confirmation content.

[0148] (7) An information processing device according to any one of (1) to (4), characterized in that it is provided with a transmission means for transmitting the captured image and the display transition image (transmitting when storing in another device, and transmitting when displaying in another device). The information processing device of (7) can be flexibly operated.

[0149] (8) One aspect for solving the above-mentioned problem is a program for causing an information processing device, which has a display unit capable of accepting touch operations and progresses processing based on customer operations, to function as a computer, the program causing the computer to function as an image acquisition means for acquiring an image of a predetermined area and a display control means for controlling the display of the display unit, the program being characterized in that the display control means displays the image and a display transition image showing the transition of the display content of the display unit on the same screen by matching the time of image capture with the time of display. According to the program (8), the operating status can be conveniently confirmed.

[0150] In addition, a program for realizing the functions of each device described above (such as the registration settlement device 10 and the transaction status management device 40) may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing of each device. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. The "computer system" may also include a plurality of computer devices connected via a network including the Internet, a WAN, a LAN, a dedicated line, and other communication lines. The "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. The recording medium also includes a recording medium provided inside or outside the computer system that can be accessed from a distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format executable by a terminal device. That is, the format in which the program is stored in the distribution server does not matter as long as the program can be downloaded from the distribution server and installed in a form executable by the terminal device. The program may be divided into multiple parts, downloaded at different times, and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a program that holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a network. The program may also be a program for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system. [Explanation of symbols]

[0151] 1...Sales system, 10...Registration and settlement device, 40...Transaction status management device, 50...Store controller, 60...Head office server, 230...Settlement device, 250...Settlement device, 260...Charge device, 270...Label issuing device, 280...Weighing device, 290...Ticket vending machine

Claims

1. A system including an information processing device that processes a transaction based on a customer's operation, An imaging unit that images a predetermined area; A storage unit that stores event information of an event that occurs in a transaction; a display unit that can receive touch operations and displays information according to the progress of processing; Equipped with An image captured by the imaging unit can be displayed, and when an operation for designating one of the events is performed, the image captured at the time of occurrence of the event can be displayed. An information processing system comprising:

2. The event is: This includes taking home unregistered products without registering them.

2. The information processing system according to claim 1 .

3. A display transition image showing the transition of the display content of the display unit can be displayed, and the captured image and the display transition image are displayed on the same screen with the time of capture and the time of display being aligned.

2. The information processing system according to claim 1 .

4. The information processing system according to claim 3, characterized in that an image generated based on operation log information is displayed as the display transition image.

5. The information processing system according to claim 4, characterized in that the display transition image is displayed to notify the position of the touch operation contained in the log information.

6. The information processing system according to claim 3, characterized in that touch position information indicating the position of the touch operation contained in the operation log information is displayed superimposed on the display transition image.

7. A switching unit that switches between a first mode in which the captured image and the display transition image are displayed, a second mode in which the display transition image is displayed on the captured image without being displayed, and a third mode in which the display transition image is displayed without being displayed on the captured image.

7. The information processing system according to claim 3, further comprising:

8. A transmission unit that transmits the captured image and the display transition image.

7. The information processing system according to claim 3, further comprising:

9. An information processing device that progresses processing based on a customer's operation, An imaging means for imaging a predetermined area; a display means capable of receiving a touch operation and displaying information according to the progress of a process; Equipped with The display means includes: An information processing apparatus characterized in that an image captured by said imaging means and a display transition image showing the transition of the display content by said display means are displayed on the same screen with the image capture time and the display time coinciding with each other.

10. A program for causing an information processing device, which has a display unit capable of receiving touch operations and progresses processing based on a customer's operation, to function as a computer, The computer, A captured image acquisition means for acquiring a captured image of a predetermined area; A display control means for controlling the display of the display unit Function as a The display control means A program characterized in that the captured image and a display transition image showing the transition of the display content of the display unit are displayed on the same screen with the capture time and the display time coinciding with each other.

Images