Sales processing device, program, and control method

Abstract

To improve visibility of an operator.SOLUTION: A sales processor includes: display means for displaying information relating to a transaction; and display control means for displaying a part of a display area in the display means and an area different from the part of the display area in such a way that visibility thereof is different from that of the part of the display area.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present invention relates to a sales processing device, a program, and a control method. [Background technology]

[0002] There is a POS (Point Of Sales) provided with a display (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5559252 Summary of the Invention [Problem to be solved by the invention]

[0004] Displays have become larger in recent years. The larger the display, the greater the amount of information that can be displayed, which has created a new problem: it is difficult for customers and store clerks to know where to look on the display.

[0005] The present invention has been made in view of the above circumstances, and has an object to improve visibility for an operator. [Means for solving the problem]

[0006] One aspect of the present invention for solving the above-mentioned problems is a sales processing device comprising a display means for displaying information relating to a transaction, and a display control means for displaying a portion of a display area on the display means and an area other than the portion so as to have different visibility.

[0007] Another aspect of the present invention is a program for causing a computer to function as a sales processing device equipped with a display means for displaying information relating to a transaction, and the program for causing the computer to function as a display control means for displaying a portion of a display area on the display means and an area other than the portion so as to have different visibility.

[0008] Another aspect of the present invention is a control method for a sales processing device equipped with a display means for displaying information related to a transaction, the control method including a display control step for displaying a portion of a display area on the display means and an area other than the portion so as to have different visibility. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a shopping system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of installation of a settlement device in this embodiment. [Diagram 3] 1 is a diagram showing an example of the appearance of a settlement device according to an embodiment of the present invention; [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a settlement device in the present embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of the configuration of a customer terminal device in the present embodiment. [Figure 6] 1 is a diagram illustrating an example of the configuration of a transaction management device in this embodiment. [Figure 7] FIG. 1 is a diagram illustrating an example of the configuration of a transaction status management device according to an embodiment of the present invention. [Figure 8] 3A to 3C are diagrams illustrating an example of customer information, store information, and cart information according to the present embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a definition of a reserved product in the present embodiment. [Figure 10] A sequence diagram showing an example of processing procedures executed by a customer terminal device, a settlement device, and a transaction management device in this embodiment in response to the accounting of one transaction in self-service operation. [Figure 11]FIG. 2 is a diagram showing an example of a screen of a shopping application in the present embodiment. [Figure 12] 11A to 11C are diagrams illustrating an example of a registered product presentation screen, a product registration screen, and a guide image according to the embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a settlement screen in the present embodiment. [Figure 14] FIG. 13 is a diagram showing an example of a reserved product processing screen in the present embodiment. [Figure 15] FIG. 13 is a diagram showing an example of a status confirmation screen in the present embodiment. [Figure 16] FIG. 4 is a diagram showing an example of a product information input screen in the present embodiment. [Figure 17] 10 is a flowchart showing an example of a processing procedure executed by the customer terminal device and the transaction management device in this embodiment in relation to product registration. [Figure 18] 11 is a flowchart showing an example of a processing procedure executed by the settlement device in this embodiment in relation to releasing the reserved item. [Figure 19] FIG. 13 is a diagram showing an example of a status confirmation screen in the second modified example. [Figure 20] FIG. 13 is a diagram showing an example of a settlement screen in the third modified example. [Figure 21] FIG. 13 is a diagram showing an example of a screen display. [Figure 22] FIG. 13 shows a settlement screen. [Diagram 23] FIG. 13 shows a product information input screen and a payment method selection screen. [Figure 24] FIG. 13 is a diagram showing an example of a screen that gradually darkens. [Diagram 25] FIG. 13 is a diagram for explaining a lower or upper position. [Figure 26] FIG. 13 is a diagram for explaining rightward or leftward shifting. [Figure 27] FIG. 13 is a diagram showing a display example in which a partial area and an area surrounding the partial area have different visibility. [Figure 28] FIG. 13 is a diagram showing a display example of a darkened area. [Figure 29]FIG. 13 is a diagram showing an example of dynamically changing a region of interest. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Embodiment> [Example of shopping system configuration] Fig. 1 shows an example of the configuration of a shopping system (an example of a product sales data processing system) according to this embodiment. The shopping system in Fig. 1 corresponds to accounting (customer terminal-compatible accounting) in which, for example, a customer takes out a product to be purchased from a product shelf, registers the taken-out product using a customer terminal device 50, and the same customer also uses a payment device 40 to settle the payment for the registered product. The accounting here refers to a procedure carried out in response to one transaction, including the registration of the product and the settlement of the registered product. The shopping system of this embodiment includes a management device 10, a transaction management device 60, a settlement device 40, a customer terminal device 50, a transaction status management device 80, and a credit card settlement server 70.

[0011] The management device 10, settlement device 40, and transaction status management device 80 are installed in a store and are communicatively connected via a LAN 19 (which may be wired or wireless). The management device 10 is a business terminal used by store staff for store management, etc., and is capable of communicating with the transaction management device 60.

[0012] The transaction status management device 80 is a business terminal that monitors the status of transactions performed by customers using the customer terminal device 50 in the store, and is capable of communicating with the transaction management device 60. Although four settlement devices 40 are illustrated in FIG. 1, the number of settlement devices 40 within one store is not particularly limited.

[0013] The customer terminal device 50 is operated by customers (such as shoppers who are members of the store, hereinafter also referred to simply as customers). In addition to typical communication and imaging functions (camera), the customer terminal device 50 has a recognition function that scans barcodes attached to products to read product codes, that is, recognizes barcodes attached to products. Note that the recognition function of the customer terminal device 50 only needs to be able to read product codes, and does not need to be able to recognize which product the read product code belongs to. In other words, the customer terminal device 50 only needs to be able to read a product code from a barcode that exists as an object in an image captured by the imaging function (an image acquired as a through image).

[0014] Such a customer terminal device 50 may be a smartphone, a tablet terminal, or the like owned by a customer. A shopping application having a function of registering products in accordance with the shopping system of this embodiment is installed in the smartphone, tablet terminal, or the like serving as the customer terminal device 50.

[0015] Furthermore, the customer terminal device 50 may be, for example, a terminal loaned from a store to a customer. An application having a function of registering products in accordance with the shopping system of this embodiment is also installed in the customer terminal device 50 loaned from the store to a customer. The customer terminal device 50 loaned from the store to a customer may be one that is held by the customer or may be one that is attached to a shopping cart and used by the customer.

[0016] The customer terminal device 50 also captures an image (e.g., releases the shutter) of the product (e.g., the peripheral portion with a barcode attached) and generates a captured image (image data). The customer terminal device 50 may capture an image by releasing the shutter in response to an operation by the customer who is the operator, but in this embodiment, it is preferable that the customer terminal device 50 captures an image by releasing the shutter at its own discretion.

[0017] In addition, the customer terminal device 50 may be equipped with an image recognition technology that extracts feature points from an image (through image, captured image) and recognizes an imaged subject (object, etc.). For example, the customer terminal device 50 may use the image recognition technology to identify (estimate) an imaged product.

[0018] The settlement device 40 is a settlement device capable of at least cash payment as a settlement method. The settlement device 40 is configured to be able to communicate with the transaction management device 60. The settlement device 40 may be configured to be able to communicate with the transaction management device 60, for example, via the management device 10.

[0019] The transaction management device 60 manages transactions carried out in accordance with the shopping system of this embodiment.

[0020] The credit card payment server 70 executes payment processing corresponding to the credit card in response to a credit card payment request transmitted from the settlement device 40, the customer terminal device 50, etc.

[0021] [About the payment device] Fig. 2 is a diagram showing an example of installation of the settlement device 40. Fig. 2(A) is a perspective view of the settlement device 40 etc. as seen from the customer side. Fig. 2(B) is a perspective view of the settlement device 40 etc. as seen from the store clerk side. As shown in FIG. 2(A), a counter is placed on the right side of the settlement device 40 when viewed from the customer side.

[0022] Fig. 3 is a diagram showing an example of the appearance of the settlement device 40. Fig. 3(A) is a perspective view of the settlement device 40 as seen from the customer side. Fig. 3(B) is a perspective view of the settlement device 40 as seen from the store clerk side. Fig. 4 is a diagram showing an example of the configuration of the settlement device 40. In Fig. 3 and Fig. 4, the same parts are given the same reference numerals.

[0023] An example of the configuration of settlement device 40 shown in Fig. 4 will be described below with reference to Fig. 3. Settlement device 40 includes a CPU 401, a ROM 402, a RAM 403, a hard disk 404, a customer display unit 405, a customer scanner unit 406, a card payment unit 408, a change machine 409, a store clerk display unit 410, a key operation unit 411, a store clerk scanner unit 412, a printing unit 413, an audio output unit 414, and a communication unit 415. These can communicate with each other via a bus.

[0024] The CPU 401 is a central processing unit, and controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 . The ROM 402 is a read-only memory, and stores various types of information used by the CPU 401, including programs.

[0025] The RAM 403 is a read / write memory and stores various information, such as information read from the ROM 402 or the hard disk 404, information acquired from the outside, and information generated during processing.

[0026] The hard disk 404 stores various information. For example, the hard disk 404 may store programs executed by the CPU 401 instead of the ROM 402. Also, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information acquired from the outside, information generated during processing, and the like.

[0027] The customer-side display unit 405 is a touch display for customers, which displays various information to customers and accepts various inputs from customers. Customer-side scanner unit 406 is a scanner unit for customers, and for example, scans a code (for example, a code other than a barcode, such as a barcode or a two-dimensional code) attached to a product and reads the product code. Customer-side scanner unit 406 may also scan a code (barcode, two-dimensional code, etc.) printed on a bill (registered trademark) and read information required for settlement. Customer-side scanner unit 406 may also scan a code (two-dimensional code, barcode, etc.) displayed on the display unit of customer terminal device 50 and read information required for settlement.

[0028] Although the customer-side scanner unit 406 is used when a customer registers a product, the customer may register the product by other methods. For example, if a preset key (a button for ordering the product) corresponding to the product is displayed on the customer-side display unit 405, the customer may operate (press) the preset key to register the product.

[0029] The card payment unit 408 is a payment mechanism that uses various cards (credit cards, prepaid cards such as transportation cards, etc.). The card payment unit 408 of this embodiment includes a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient that the card payment unit 408 includes at least the card recognition unit. The card recognition unit may also recognize various cards (e.g., membership cards, point cards, etc.) that are not directly used for payment (settlement).

[0030] The change machine 409 (cash settlement unit) is a cash settlement mechanism, which has an input port for banknotes and coins, and an output port for banknotes and coins, calculates the amount of money inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. The change machine 409 faces the customer and is operated by the customer. When banknotes or coins are inserted into the input port, they are detected by a sensor (detecting that they have been inserted, detecting the number of coins by denomination, etc.).

[0031] That is, the change machine 409 is used when the price of registered products is settled with cash (money) in the settlement device 40. The change machine 409 has a banknote insertion port for inserting banknotes, a coin insertion port for inserting coins, a banknote discharge port for discharging banknotes, a coin discharge port for discharging coins, a counting unit for counting the inserted or discharged money, a money transport mechanism between the insertion port or discharge port and the storage unit, the above-mentioned sensor, etc. The banknote insertion port and the coin insertion port are also referred to as deposit insertion ports. The banknote discharge port and the coin discharge port are also referred to as change discharge ports. The banknote insertion port and the banknote discharge port may be a common port, and the coin insertion port and the coin discharge port may be a common port.

[0032] The change machine 409 also counts the coins replenished during the closing process and stores them in a storage unit. The change machine 409 also counts the coins to be dispensed during the closing process and discharges them from a change discharge port. The closing process is a process of adjusting the amount of money stored in the change machine 409 (cash on hand / cash on hand) to a reference amount after the store is closed or before the store is opened.

[0033] The store clerk's display unit 410 is a touch display for the store clerk, which displays various information to the store clerk and receives various inputs from the store clerk. The key operation unit 411 is made up of various keys (buttons) and receives various inputs from the store clerk. The store clerk scanner unit 412 is a scanner unit for store clerks, and for example, scans a barcode attached to a product and reads the product code. The store clerk scanner unit 412 also scans a barcode attached to the name tag of the store clerk and reads the store clerk code.

[0034] Although the store clerk's scanner unit 412 is used when the store clerk registers a product, the store clerk may register the product by other methods. For example, if a key corresponding to a product (e.g., a key corresponding to a sports newspaper) is arranged on the key operation unit 411, the store clerk may operate (press) that key to register the product. Also, if a preset key corresponding to a product is displayed on the store clerk's display unit 410, the store clerk may operate that preset key to register the product.

[0035] The printing unit 413 prints and issues various media (receipts, accounting vouchers, etc.). The printing unit 413 can be rotated freely in the direction (the direction of the media issuing port) from the clerk side to the customer side and from the customer side to the clerk side. The orientation of the printing unit 413 may be changed manually, or may be automatically changed (mechanically controlled, etc.) according to, for example, the transition (switching) of the operation modes described later. In addition, the correctness of the orientation of the printing unit 413 may be detected by a sensor or the like.

[0036] The voice output unit 414 outputs voice. For example, the voice output unit 414 outputs voice guidance and the like. The communication unit 415 executes communication with other devices (other settlement devices 40 and management device 10).

[0037] Note that the settlement device 40 of the present embodiment can be switched between a total of four operation modes, including a customer terminal corresponding settlement mode set to perform settlement processing under customer terminal corresponding accounting, and three accounting modes (first to third accounting modes) corresponding to three different patterns of accounting (product registration, settlement). The first accounting mode is an operation mode in which a clerk performs operations corresponding to product registration and settlement on the settlement device 40. The second accounting mode is an operation mode in which a clerk performs an operation corresponding to product registration on the settlement device 40, and a customer performs an operation corresponding to settlement on the same settlement device 40. The third accounting mode is an operation mode in which a customer performs operations corresponding to product registration and settlement on the settlement device 40. Thereby, in a store introducing the shopping system of the present embodiment, for some of the plurality of settlement devices 40, by setting the customer terminal corresponding settlement mode, it corresponds to the customer terminal corresponding settlement, and for other settlement devices 40, for example, by setting the first accounting mode or the second accounting mode, a clerk can interact with a customer to perform accounting. Note that those in which the third accounting mode is set in other settlement devices 40 may be included. Such a mode switch may be performed, for example, according to a predetermined time period within the business hours of the store. Note that the settlement device 40 of the present embodiment may be configured as a dedicated device specialized for the above-described customer terminal-compatible settlement mode.

[0038] [Configuration Example of Customer Terminal Device] FIG. 5 shows a configuration example of the customer terminal device 50. The customer terminal device 50 in the figure includes a CPU 501, a storage unit 502, a RAM 503, a display unit 504, an operation unit 505, an imaging unit 506, and a network-compatible communication unit 507. The CPU 501 is a central processing unit, and controls the operation of the customer terminal device 50 by reading and executing programs stored in the storage unit 502. The storage unit 502 is an auxiliary storage device of the CPU 501, and stores various types of information used by the CPU 501, including programs. The RAM 503 is the main storage device of the CPU 501. The display unit 504 displays an image according to the display control of the CPU 501. The display unit 504 may be configured as a touch panel capable of operating on the screen. The operation unit 505 collectively represents various operation elements provided in the customer terminal device 50 and various input devices connected to the customer terminal device 50. The imaging unit 506 performs imaging according to the control of the CPU 401. In addition, the imaging unit 506 of the present embodiment can also function as a part that reads a two-dimensional code in a code reader that processes two-dimensional codes. The network-compatible communication unit 507 is a part that communicates wirelessly via a network. The customer terminal device 50 can communicate with the transaction management device 60 through the network-compatible communication unit 507.

[0039] [Configuration Example of Transaction Management Device] FIG. 6 shows a configuration example of the transaction management device 60. The transaction management device 60 in the figure includes a CPU 601, a storage unit 602, a RAM 603, and a network-compatible communication unit 604. The CPU 601 is a central processing unit, and controls the operation of the transaction management device 60 by reading and executing programs stored in the memory unit 602 . The storage unit 602 is an auxiliary storage device for the CPU 601, and stores various types of information used by the CPU 501, including programs. The RAM 603 is a main storage device for the CPU 601 . The network compatible communication unit 604 is a part that performs communication via a network.

[0040] The memory unit 602 of the transaction management device 60 stores customer information, store information, product master (product information), and cart information.

[0041] [Example of transaction status management device configuration] 7 shows an example of the configuration of a transaction status management device 80. The transaction status management device 80 in the figure includes a CPU 801, a storage unit 802, a RAM 803, an in-store compatible communication unit 804, a network compatible communication unit 805, a display unit 806, and an operation unit 807. The CPU 801 is a central processing unit, and controls the operation of the transaction status management device 80 by reading and executing programs stored in the storage unit 802 . The storage unit 602 is an auxiliary storage device for the CPU 801, and stores various types of information used by the CPU 801, including programs. The RAM 803 is the main storage device of the CPU 801 . The in-store communication unit 804 communicates with terminals within the store, such as the settlement device 40 and the management device 10, via the LAN 19. The network compatible communication unit 805 is a part that communicates with the outside via a network. In the case of the transaction status management device 80, the network-compatible communication unit 805 can communicate with the transaction management device 60 on the cloud. The display unit 806 displays images under the control of the CPU 801. The display unit 806 may be configured as a touch panel. The operation unit 807 collectively indicates the operators provided in the transaction status management device 80 and the input devices connected to the transaction status management device 80 .

[0042] [Information stored in the transaction management device] The customer information stored in the memory unit 602 is information for managing individual customers. The transaction management device 60 generates customer information when a customer is registered (a customer's registration may be considered to have been completed when the customer information of that customer is stored). The transaction management device 60 also updates the customer information as appropriate based on cart information, etc. The transaction management device 60 may update the customer information, for example, by referring to the cart information at a specified time every day.

[0043] 8(A) shows an example of customer information stored in the memory unit 602 of the transaction management device 60. The customer information in this figure includes customer identification information, customer name, customer registration date, cancellation information, customer rank, number of points, etc. The customer identification information is identification information that uniquely identifies a customer. The customer identification information may be application identification information that is uniquely assigned to each installed shopping application when the shopping application of this embodiment is installed. The application identification information may be, for example, a number that is uniquely (for example, sequentially) assigned to each installed shopping application. Alternatively, the application identification information may be terminal identification information that is unique to the customer terminal device 50 in which the shopping application is installed. The terminal identification information may be, for example, a telephone number assigned to a customer who is the user of the customer terminal device 50, or hardware identification information that is uniquely assigned to the hardware of the customer terminal device 50, such as a MAC (Media Access Control) address. The customer name is the customer's name or nickname, etc. The customer registration date is the date and time when the customer was registered. Cancellation information is information regarding the cancellation of registered products after registration. The customer rank is a rank according to the customer's purchasing history. When generating customer information for a new customer, the customer identification information, customer name, and customer registration date are generated, but since this is before the start of actual trading (product registration), other information (cancellation information, etc.) is not generated.

[0044] The transaction management device 60 generates and stores customer identification information, for example, when registering a customer (for example, when the customer terminal device 50 downloads or installs a shopping application to use a shopping service provided by the transaction management device 60 from an external device (for example, a specified server that provides all applications, the transaction management device 60 itself)). In addition, for example, when registering a customer, the transaction management device 60 acquires information entered into the name field of a registration form (input form) using the customer terminal device 50, and stores it as the customer name. In addition, the transaction management device 60 acquires, for example, the current date and time when registering the customer, and stores it as the customer registration date.

[0045] In addition, instead of or in addition to storing the customer information in its own memory unit, the transaction management device 60 may store some or all of the customer information in another device (such as a file server accessible to the transaction management device 60).

[0046] 8(B) shows an example of store information stored in the memory unit 602 of the transaction management device 60. The store information in the figure includes store identification information, store name (branch name), store identification information 1, and store identification information 2. The store identification information is identification information that uniquely identifies a store. The store identification information shown in Fig. 8(B) is composed of a store (trade name) or company code and a branch code. The store name is the name of the store. The store name shown in Fig. 8(B) is composed of the store (trade name) or company and the branch name. The store identification information 1 is information on a two-dimensional code (QR code (registered trademark) or the like) for identifying a store where a transaction takes place (a store where a product is bought and sold). The store identification information 2 is store location information (GPS information) for identifying the store where the transaction is to be carried out. In the example shown in FIG. 8(B), the store identification information and the store specifying information 1 are different, but the store identification information and the store specifying information 1 may be the same.

[0047] The transaction management device 60 may obtain and store store information from an external source (such as a server (not shown) at a headquarters that manages each store). The transaction management device 60 may store some or all of the store information in another device (such as a file server accessible by the transaction management device 60) instead of or in addition to storing the store information in its own memory.

[0048] Cart information is information for managing individual transactions. The transaction management device 60 generates the cart information at the start of a transaction. In addition, the transaction management device 60 updates the cart information as the transaction progresses (each time a product is registered) (the storage of a product in the cart information may be interpreted as the registration of that product).

[0049] FIG. 8(C) is an example of cart information stored in the memory unit 602 of the transaction management device 60. The cart information in the figure includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, product information in the cart (registered product information, reserved product information), cancellation information, and the like.

[0050] The cart identification information is identification information that uniquely identifies the cart information. For example, the cart identification information includes store identification information, a date, and a serial number (for example, a serial number for each store and each date). The transaction start date and time is the date and time when the transaction starts. The transaction start date and time may be, for example, the date and time when the cart information is generated. The transaction start date and time may be the date and time when the first item of product is registered (the date and time when the registered product information (registered product 1) in FIG. 8(C) is stored). The date and time when the cart information is generated and the date and time when the first item of product is registered may both be stored separately.

[0051] The transaction end date and time is the end date and time of the transaction. For example, the transaction end date and time may be the settlement date and time. The customer identification information is the customer identification information that identifies the customer of the transaction. Note that when generating the cart information, the cart identification information, the transaction start date and time, and the customer identification information are generated, but since it is before the start of the actual transaction (product registration), other information (such as the transaction end date and time) is not generated. The settlement date and time may be the settlement start date and time or the settlement end date and time. As the transaction end date and time, both the settlement start date and time and the settlement end date and time may be separately stored.

[0052] The registered product information (total) is information that is updated each time a product is registered. The registered product information (total) includes the quantity (number of products), the estimated subtotal amount (for example, the subtotal amount before discounts, etc.), the subtotal amount, etc. The registered product information (registered product 1) is the registration information of one item of product. The registered product information (registered product 2) is the registration information of two items of product. In the example shown in FIG. 8(C), the illustration of the registered product information (registered product 3) to the registered product information (registered product 5) is omitted. The registered product information (N; N is an integer) includes the product code, the product name, the price, etc.

[0053] The registered product information (N) may include the registration date and time of the Nth item of product. That is, the transaction management device 60 may store the registration date and time of the registered product as the registered product information. The registration date and time of each product may also be used as a basis for determining the necessity of applying services such as time services and the effects after application.

[0054] The reserved product information (total) is information that is updated each time a reserved product is registered. A reserved product is a product that has not been normally registered due to a predetermined factor and is temporarily registered in a reserved state. The reserved product information (total) includes the quantity (number of products) of the reserved products, the quantity of NON-FILE among the reserved products, the quantity of read NG (operation error confirmation required) among the reserved products, etc.

[0055] NON-FILE refers to a product whose barcode or product code was successfully scanned in the store (the product code was read), but the product code is not stored as product information in the product master (a product whose product information is not registered).

[0056] A read NG is when a product code scan at a store fails (when the product code cannot be read), or is one aspect of a case where a product code scan at a store fails. In other words, a read NG refers to a timeout process when it is possible to determine that a product has been imaged for a certain period of time using image recognition technology but has not been recognized as a barcode, or an event where a timeout process has been performed. For example, if the barcode cannot be correctly acquired (recognized) despite the customer performing an operation to have the barcode read due to wrinkles in the package, smudges or dirt on the barcode print, or the barcode being printed on a transparent film, it will be determined that the barcode cannot be read.

[0057] Reserved Product Information (Reserved Product 1) is information on reserved product 1. Reserved Product Information (Reserved Product 2) is information on reserved product 2. Reserved Product Information (Reserved Product 3) is information on reserved product 3.

[0058] The reserved product information (reserved product N; N is an integer) includes the reserved product type (information indicating whether the reserved product is NON-FILE or cannot be read) and image data (image data captured when reading was NG). For example, if the Nth item is a reserved product due to NON-FILE, the reserved product information (reserved product N) includes the reserved product type "1 (NON-FILE)", product code, and image data. Also, if the Nth item is a reserved product due to reading being NG, the reserved product information (reserved product N) includes the reserved product type "2 (cannot be read)" and image data.

[0059] 9 shows an example of reserved product definition corresponding to cart information in this embodiment. In the figure, an example is shown in which reserved products are classified into eight categories. Classification numbers "1" to "8" are assigned to each of the eight reserved product classifications. The classification of the reserved product with classification number 1 is “NON-FILE” as explained in FIG. The classification of the reserved product with classification number 2 is “reading not allowed” as explained in FIG.

[0060] The classification of the reserved product with classification number 3 is "pseudo scan input." A reserved product with "pseudo scan input" is a product for which the customer has attempted to have the customer terminal device 50 read the product's code, but the customer terminal device 50 has not been able to read the code properly.

[0061] As a specific example, a situation where a product is brought into the imaging range of the imaging unit 506 but the product code is not brought into the imaging range of the imaging unit 506 corresponds to a "pseudo scan input". Additionally, a "pseudo scan insertion" situation corresponds to a situation in which the product is brought into the imaging range of the imaging unit 506, but an image of a part that does not have a code is captured. Additionally, "pseudo scan insertion" refers to a situation in which a product is brought into the imaging range of imaging unit 506 with the code portion hidden by a finger or the like. In addition, a "pseudo scan input" situation occurs when the code on a product is captured by the imaging unit 506, but the product is passed through the imaging range of the imaging unit 506 at a speed that prevents the customer terminal device 50 from reading the code.

[0062] The classification of the reserved product with classification number 4 is “non-scan-inserted.” A reserved product with the “non-scan-inserted” classification is a product that was added to the shopping cart without performing an operation to position the product within the imaging range of imaging unit 506. This non-scanning insertion can occur, for example, when the customer terminal device 50 is fixedly attached to a shopping cart and is configured to be able to capture an image of the inside of a shopping basket placed in the shopping cart. For example, when the customer terminal device 50 determines based on the captured image that a product has been placed in the shopping cart even though the product's code has not been scanned, it can determine that a non-scanning insertion has occurred.

[0063] The classification of reserved items with classification number 5 is "age verification items." "Age verification items" are items such as alcohol and tobacco that require a store clerk to verify the customer's age before selling them.

[0064] The classification of reserved goods with classification number 6 is "pharmaceuticals." "Pharmaceuticals" are goods that require a qualified person, such as a pharmacist, to confirm with the purchaser.

[0065] The classification of reserved items with classification number 7 is "anti-theft tag items." "Anti-theft tag items" are items that have anti-theft tags attached and require a store clerk to remove the anti-theft tag from the item when it is sold.

[0066] The classification of the reserved product with classification number 8 is a "cancelled product." A "cancelled product" is a product that was once registered but was cancelled by a customer's cancellation operation on the customer terminal device 50. A "cancelled product" is taken from the customer by a store clerk at the time of payment, for example.

[0067] The product master is stored, for example, in the management device 10. The product master stores product information for each product sold in the store. The product information for one product includes a number of predetermined information items, including a product code and a product name. Transaction management device 60 stores the product master data acquired from management device 10. Transaction management device 60 uses the stored product master data for processes related to product registration, etc.

[0068] For reserved items that fall under "NON-FILE" in classification number 1, the product code was obtained by code scanning in response to the product registration operation, but the product information associated with the obtained product code was not registered in the product master. Therefore, it has not been determined which of the products registered in the product master the reserved items that fall under "NON-FILE" in classification number 1 are. For pending products that fall into one of the three categories of classification numbers 2 to 4, "Read NG", "Pseudo scan input", and "Non-scan input", the product code has failed to be acquired as a result of the code scan in response to the product registration operation. Therefore, it is not possible to identify which of the products registered in the product master the pending products that fall into one of the three categories of classification numbers 2 to 4, "Read NG", "Pseudo scan input", and "Non-scan input", are.

[0069] Among classification numbers 1 to 7, pending products that fall into any of the four classifications of classification numbers 1 to 4, "NON-FILE," "Read NG," "Pseudo-scan input," and "Non-scan input," are unspecified products. Unspecified products are products whose registration results are incomplete, for example, because it was not possible to identify which of the products registered in the product master they are. Unidentified products are further divided into code-acquired products, whose product codes have been acquired through product registration operations, such as classification number 1 "NON-FILE," and code-unacquired products, whose product codes have not been acquired through product registration operations, such as classification numbers 2 to 4 "Read NG," "Pseudo scan input," and "Non-scan input."

[0070] In addition, instead of or in addition to storing the cart information in its own memory, the transaction management device 60 may store some or all of the cart information in another device (such as a file server accessible by the transaction management device 60).

[0071] [Example of basic processing procedure for self-operation] The sequence diagram in Fig. 10 shows an example of a processing procedure executed by the customer terminal device 50, the settlement device 40, and the transaction management device 60 in response to the accounting of one transaction in a self-service operation. Specifically, the processing in Fig. 10 shows an example of the processing of the customer terminal device 50, the settlement device 40, and the transaction management device 60 from when a customer registers a product to be purchased from among products displayed in the store, until the settlement of the registered product is completed.

[0072] Step S1: The customer terminal device 50 acquires information for identifying a store (store identification information). For example, a two-dimensional code for identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output on a display screen, a medium with the two-dimensional code printed on it may be attached, etc.), and a customer visiting the store may scan (read) the two-dimensional code with customer terminal device 50, which may allow customer terminal device 50 to acquire store identification information. When a customer visiting the store starts a shopping application, a screen instructing the customer to scan the two-dimensional code may be displayed as a start-up screen, or when a customer visiting the store scans the two-dimensional code with customer terminal device 50, the shopping application may be started, and a screen informing the customer that they are currently connected to transaction management device 60 may be displayed as a start-up screen.

[0073] Also, for example, since a store is identified by its location, a customer who visits the store may obtain location information (GPS information) at the store using the customer terminal device 50 (i.e., the store's location information may be obtained as store identification information). When a customer who visits the store starts a shopping application, the location information may be obtained, and a screen may be displayed as a start-up screen informing the customer that the application is currently connected to the transaction management device 60. If multiple stores are detected from the location information and it is not possible to identify one, a selection screen may be displayed to allow the customer to make a selection. Alternatively, you can switch to a mode that forcibly acquires a 2D code.

[0074] 11(A) shows an example of a start-up screen of a shopping application displayed on the display unit 504 of the customer terminal device 50. The start-up screen in the figure displays the store name. This store name is displayed based on the store-specific information acquired in step S1. When a customer wishes to start registering a product for shopping, the customer operates a transaction start button BT10 arranged on the start-up screen. In response to the operation of the transaction start instruction button BT10, the customer terminal device 50 transmits a transaction start request including the acquired store specific information and customer identification information to the transaction management device 60. The transaction start request may be transmitted in response to the customer's operation on the customer terminal device 50 as described above, or may be transmitted in response to the acquisition of store specific information without receiving any particular operation. Regarding the customer identification information, the information entered in the name field of the registration form using the customer terminal device 50 during customer registration (when downloading or installing a shopping application on the customer terminal device 50) may be stored in the customer information of the transaction management device 60, but may also be stored in the memory of the customer terminal device 50 in addition to the transaction management device 60. When a store is identified (when the product registration initial screen described below is acquired), the store name, the services being offered (flyer information distributed on that day), and available coupon information may be displayed on a screen (the product registration initial screen or a screen other than the product registration initial screen). The service and coupon information may be acquired from the transaction management device 60 as screen information, for example.

[0075] In addition, the destination information (address of the transaction management device 60) may also be obtained when registering the customer (when downloading or installing the shopping application to the customer terminal device 50) and stored in the memory of the customer terminal device 50. In addition to the store-specific information, the destination information may also be converted into a two-dimensional code, and by scanning the two-dimensional code with the customer terminal device 50, the customer terminal device 50 may obtain the destination information along with the store-specific information.

[0076] Step S2: The transaction management device 60, which has received the customer identification information and store identification information as a transaction start request from the customer terminal device 50, generates cart information for the transaction. Note that in the initial state in which the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are stored, but since this is before the start of the actual transaction (product registration), other information (transaction end date and time, product information (registered product information, reserved product information), etc.) is not stored.

[0077] As described above, the transaction management device 60 stores store information such as that shown in Figure 8 (B), so when it receives store identification information (it also receives customer identification information) as a transaction start request from the customer terminal device 50, if the received store identification information is a two-dimensional code, it acquires the store identification information by referring to store identification information 1, and if the received store identification information is location information (GPS information), it acquires the store identification information by referring to store identification information 2. Note that if the store identification information received from the customer terminal device 50 is store identification information converted into a two-dimensional code, the transaction management device 60 need only acquire it as is.

[0078] In other words, the transaction management device 60 receives customer identification information and store identification information as a transaction start request from the customer terminal device 50, acquires the store identification information from the store identification information received from the customer terminal device 50, and further acquires the current date, issues (assigns) a serial number, combines the store identification information, current date, and serial number, and stores the combined information as cart identification information in the cart information. Furthermore, when the transaction management device 60 receives store specific information and customer identification information as a transaction start request from the customer terminal device 50, it acquires the current date and time and stores it as the transaction start date and time (generation date and time) in the cart information. Furthermore, when the transaction management device 60 receives store specific information and customer identification information as a transaction start request from the customer terminal device 50, it stores the customer identification information received from the customer terminal device 50 as customer identification information in the cart information.

[0079] Step S3: The transaction management device 60, which has generated the cart information for the transaction, generates product registration initial screen information (screen information of the product registration screen, which is the initial screen), and transmits it to the customer terminal device 50. Specifically, the transaction management device 60, for example, generates product registration initial screen information, and transmits the generated product registration initial screen information to the customer terminal device 50 together with the cart identification information.

[0080] Step S4: The customer terminal device 50, which has received the cart identification information and the product registration initial screen information from the transaction management device 60, stores the cart identification information and displays the registration screen on the display unit. Specifically, the customer terminal device 50 displays the product registration initial screen as shown in FIG. 11(B), for example.

[0081] Step S5: The customer terminal device 50 scans the barcode attached to the product by the customer's operation, and reads the product code. In the figure, it is assumed that the barcode has been successfully scanned. Steps S5 to S9 are repeatedly executed every time the barcode attached to the product is scanned.

[0082] The customer terminal device 50 that has acquired the barcode transmits to the transaction management device 60 the cart identification information and product registration information including the product code obtained by scanning.

[0083] Step S6: The transaction management device 60 receives the product registration information (cart identification information and product code) from the customer terminal device 50 and identifies the cart information of the transaction from the cart identification information included in the received product registration information.

[0084] Step S7: The transaction management device 60 updates the product data in the identified cart information. Specifically, when the received product registration information contains a product code for the Nth item, the transaction management device 60 stores the product code as the product code of the registered product information (registered product N) in the identified cart information, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). Additionally, the transaction management device 60 updates the registered product information (total) in the identified cart information.

[0085] Step S8: After updating the product data in the cart information, the transaction management device 60 generates product registration update screen information (screen information of the product registration screen, which is an update screen to which the registered product has been added) and transmits it to the customer terminal device 50. Specifically, the transaction management device 60 generates product registration update screen information such that the customer terminal device 50 displays a product registration update screen to which a list item of the currently registered product has been added to the registered product list, as shown in Fig. 11(C), for example, and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information.

[0086] The product registration screen (product registration update screen) shown in FIG. 11(C) is displayed on the customer terminal device 50 after "XX bread" is registered as the third product item. In other words, when "XX yogurt" is stored in the cart information as the first item, the transaction management device 60 generates product registration update screen information such that "XX yogurt" is displayed on the customer terminal device 50 and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information; when "XX chocolate" is stored in the cart as the second item, the transaction management device 60 generates product registration update screen information such that "XX yogurt" and "XX chocolate" are displayed on the customer terminal device 50 and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information; and when "XX bread" is stored in the cart information as the third item, the transaction management device 60 generates product registration update screen information such that "XX yogurt", "XX chocolate" and "XX bread" are displayed on the customer terminal device 50 as shown in FIG. 11(C), and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information.

[0087] In response to the registration of one product, a registered product display screen WD1 may be displayed to display the currently registered product, as shown in Fig. 12(A), for example. In addition, in the registered product display screen WD1 shown in the same figure, in response to the target product being available for eating in, a "Yes" button BT1 and a "No" button BT2 are displayed to confirm with the customer whether or not they would like to eat in. In this case, the price of the product is calculated so that the consumption tax rate for the target product is different when the "Yes" button BT1 is operated and when the "No" button BT2 is operated.

[0088] Step S9: The customer terminal device 50, which has received the cart identification information and product registration update screen information from the transaction management device 60, adds the product to the registration screen. Specifically, the customer terminal device 50 displays a product registration update screen such as that shown in Fig. 11(C). As described above, the product registration screen (product registration update screen) shown in Fig. 11(C) is displayed on the customer terminal device 50 after "XX bread" is registered as the third product item.

[0089] Step S10: The customer terminal device 50 accepts a payment instruction operation as an operation by the customer, for example, a touch of the "Proceed to Checkout" button shown in FIG.

[0090] Step S11: The customer terminal device 50 that has accepted the payment transition instruction operation generates a two-dimensional code. That is, the customer terminal device 50 converts information (e.g., cart identification information) required for executing the payment process for the purchased items by the customer terminal device 50 into a two-dimensional code. The customer terminal device 50 that has generated the two-dimensional code displays the generated two-dimensional code on the display unit. For example, a code display screen in which a two-dimensional code is arranged as shown in FIG. 11(D) is displayed on the display unit.

[0091] Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the customer terminal device 50. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the customer terminal device 50 that is directed by the customer within the recognition range of the customer-side scanner unit 406.

[0092] Step S13: The settlement device 40, which has read the two-dimensional code displayed on the display unit of the customer terminal device 50, requests the calculation of the subtotal amount from the transaction management device 60. For example, the settlement device 40 transmits a calculation request (subtotal calculation request information) requesting the calculation of the subtotal amount to the transaction management device 60 together with the cart identification information obtained from the two-dimensional code.

[0093] Step S14: The transaction management device 60, which has received the cart identification information and the sub-calculation output request information from the customer terminal device 50, identifies the cart information of the transaction from the cart identification information.

[0094] Step S15: After identifying the cart, the transaction management device 60 calculates the subtotal amount using information such as the price in the registered product information included in the cart information corresponding to the identified cart information.

[0095] Step S16: After calculating the subtotal, the transaction management device 60 updates the cart information (storing the subtotal (calculated subtotal)), and transmits the subtotal information indicating the calculated subtotal to the settlement device 40 together with the cart identification information.

[0096] Step S17: The settlement device 40 receives the cart identification information and the subtotal information (settlement information) from the transaction management device 60 and displays the subtotal amount on the customer side display unit 405.

[0097] Step S18: The settlement device 40, which has displayed the subtotal on the customer display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the amount of change, and, if there is change, dispenses the change and issues a receipt. In addition, when the settlement is completed, the settlement device 40 sends settlement completion information together with the cart information to the transaction management device 60, and the transaction management device 60 stores the transaction end date and time (settlement date and time) for that cart.

[0098] After scanning a product, the customer terminal device 50 transmits the product code to the transaction management device 60 (S5). However, the customer terminal device 50 may be configured not to transmit product codes for products (e.g., products scanned after moving to another store) other than those scanned in the store (the store when the customer visited the store and displayed the product registration initial screen). For example, the customer terminal device 50 acquires and stores location information (GPS information) when the customer visits the store (or when the product registration initial screen is displayed). The customer terminal device 50 also acquires location information when scanning each product, and compares the location information acquired when the customer visits the store (or when the product registration initial screen is displayed). If the two match (or approximately match), the customer terminal device 50 may permit the transmission of the product code of the product to the transaction management device 60, and if they do not match (or approximately match), the customer terminal device 50 may prohibit the transmission of the product code of the product to the transaction management device 60. This makes it possible to prevent inappropriate product registration (for example, registering a product to a cart created in another store, etc.).

[0099] When the settlement device 40 prohibits the transmission of the product code as described above, it may display an error message (e.g., "Registration is not possible because it is not in the XX store") on the customer side display unit 405 after scanning the product. Furthermore, the settlement device 40 may display the above message on the store clerk side display unit 410 instead of or in addition to the message on the customer side display unit 405.

[0100] The settlement device 40 reads the two-dimensional code displayed on the display of the customer terminal device 50 and then requests the transaction management device 60 to calculate the subtotal amount (S13), but may not request calculation of the subtotal amount for products other than the product scanned in the store (for example, products scanned in other stores). For example, the settlement device 40 refers to the store identification information of the store (it may store and refer to the store identification information of the store in its own settlement device 40, or may refer to the store identification information stored in another accessible device), and when it reads the two-dimensional code displayed on the display of the customer terminal device 50, it compares the cart identification information obtained from the two-dimensional code with the store identification information of the store. The settlement device 40 may permit the request to calculate the subtotal amount if the store identification information included in the cart identification information includes the store identification information of the store, and may prohibit the request to calculate the subtotal amount if the store identification information does not include the store identification information of the store. This makes it possible to prevent inappropriate payments (for example, payment for products registered at other stores, etc.).

[0101] When the settlement device 40 prohibits the request for the subtotal amount as described above, after reading the two-dimensional code, the settlement device 40 may display an error message (e.g., "Settlement cannot be completed because the transaction includes products not from the XXX store") on the customer-side display unit 405. The settlement device 40 may also display the above message on the store clerk-side display unit 410 instead of or in addition to the message on the customer-side display unit 405.

[0102] In the example of the processing procedure in Fig. 10, the transaction management device 60 calculates the subtotal amount. However, for example, the settlement device 40 may acquire cart information from the transaction management device 60 and calculate the subtotal amount based on the acquired cart information. Alternatively, the transaction management device 60 may transmit cart information to the management device 10 and request calculation of the subtotal amount, the management device 10 may calculate the subtotal amount using the received cart information, and the calculated subtotal amount may be transmitted to the settlement device 40 via the transaction management device 60.

[0103] [About the behavior of customer terminal devices when an unspecified product is registered] In FIG. 10, step S5 is a process corresponding to the case where the reading of the product code is successful and the product corresponding to the read product code is identified. However, when the customer terminal device 50 performs the product code reading process in response to step S5, the product registration result may indicate that the product has been registered as an unidentified product under any of classification numbers 1 to 4 in Figure 9 (unidentified result).

[0104] 12(B) shows, by way of a captured image of a product registration screen, an example corresponding to a case where the customer terminal device 50 has determined that an unidentified result has occurred as a pseudo scan of classification number 3. The figure shows the product registration screen displayed on the customer terminal device 50. The product registration screen in the figure shows a state in which a captured image is displayed of a customer attempting to have the customer terminal device 50 read the barcode of a product with the barcode portion covered by his or her finger.

[0105] In such a case, the customer terminal device 50 obtains a processing result indicating that the presence of a barcode was recognized from the captured image, but the code could not be extracted from the barcode. Since such a processing result is included in one type of processing result in a pseudo scan, the customer terminal device 50 detects that a pseudo scan has occurred. In this case, the customer terminal device 50 transmits to the transaction management device 60 product registration information indicating that the product is an unspecified product for which a pseudo scan of classification number 3 has been performed.

[0106] Furthermore, when transmitting product registration information in response to the occurrence of an unidentified result as described above, the customer terminal device 50 transmits captured images corresponding to the generated unidentified result. The captured images to be transmitted may be extracted from the buffered captured images, for example, by buffering the captured images captured by the imaging unit 506 for a certain period of time in order to read the product code. Furthermore, the captured images to be transmitted may be, for example, a predetermined number of still images, or may be a moving image for a predetermined period of time in response to the occurrence of an unidentified result.

[0107] Furthermore, when the product to be registered is designated as an unspecified product due to the occurrence of an unspecified result as described above, the customer terminal device 50 treats the unspecified product as a reserved product. In this case, the customer terminal device 50 may, at the timing when it is determined that an unspecified result has occurred, display a guide image WD2 that marks the unspecified product as a reserved product and encourages the customer to place the product in a bag for reserved products, as shown in FIG. 12(C). Specifically, the guide image WD2 in the figure displays a message saying, "If you cannot scan the product, please place the product in a special bag." Note that the form of the guide image WD2 is not particularly limited. The guide image WD2 in the figure is displayed so as to be superimposed on the product registration screen. A confirmation button BT3 is arranged in the guide image WD2 in the figure. In this case, the guide image WD2 is erased in response to an operation on the confirmation button BT3. Alternatively, the guide image WD2 may be erased in response to a certain time having elapsed since the start of display. After the guide image WD2 is erased, the display may return to the product registration screen of FIG. 11(B) or FIG. 11(C), for example.

[0108] Unspecified products are treated as reserved products as described above. On the product registration screen, a registered product list is displayed as shown in FIG. 11(C), but for reserved products classified as unspecified products, list items do not need to be displayed in the registered product list. Reserved products that are not classified as unspecified products are displayed as list items in the registered product list. Note that unspecified products may also have list items displayed in the registered product list to indicate that they are unspecified products.

[0109] When the customer has finished registering all the products to be purchased, the customer performs a settlement instruction operation and has the settlement device 40 read the two-dimensional code displayed on the customer terminal device 50. After reading the two-dimensional code, the settlement device 40 requests the transaction management device 60 to calculate the subtotal amount, as shown as step S13 in Fig. 9. Here, in this embodiment, when requesting the calculation of the subtotal amount, the settlement device 40 requests settlement information to be used in the settlement process.

[0110] In response to a request for settlement information, the transaction management device 60 identifies the corresponding cart information and generates settlement information based on the identified cart information. The settlement information includes registered product information in the cart information and information such as a subtotal amount according to the registered products. In addition, among the registered product information included in the settlement information, the registered product information corresponding to the insufficiently registered product includes a captured image corresponding to the unidentified result. The transaction management device 60 transmits the generated settlement information to the settlement device 40. When the settlement device 40 receives the settlement information, it displays a settlement screen on the customer side display unit 405 that reflects the contents of the received settlement information.

[0111] Next, a control for displaying a part of the display area and a different area from the part of the display area so as to have different visibility will be described. Fig. 13 shows an example of a settlement screen displayed on the customer side display unit 405 of the settlement device 40. The checkout screen in the figure includes a registered product list area AR3, which displays a list of products registered by the customer terminal device 50.

[0112] In the following description, the product information (registered product information, reserved product information) corresponding to each product list item in the registered product list area AR3 may also be referred to as a record. Additionally, registered product information and reserved product information corresponding to classifications such as age verification products (classification number 5), medicines (classification number 6), and security tag products (classification number 7) are also referred to as product identification records, since the products are identified. Of the product identification records, registered product information is also referred to as normal registration records, since it corresponds to products that have been properly registered, not reserved products. Of the reserved product information, that which corresponds to unspecified products (classification numbers 1 to 4) is also referred to as unspecified records.

[0113] If the registered products include reserved products, the list items of the reserved products are given priority and arranged from the top of the left column in the registered product list area AR3. In other words, the list items of the reserved products are arranged in order from the top in the registered product list area AR3. In the example shown in the figure, five reserved products are included in the registered products, so a reserved product list area AR31 including list items of the five reserved products from the top left is formed in the registered product list area AR3. The list items in the reserved product list area AR31 are made to differ from the list items of other products, for example in terms of background color, etc.

[0114] In this way, the settlement device 40 displays a part of the display area where the list items of the reserved item list area AR31 are displayed and a different area from the part of the display area with different visibility. In addition, the settlement device 40 displays the area AR32 including the number of reserved items and the estimated total, which are information that supplements the reserved item list area, with the same visibility as the reserved item list area AR31. This improves customer visibility. Once the products have been identified and the reserved products have been resolved, the amount of each product can be identified, and the estimated total display will change to show the total amount, which is the amount actually required for settlement.

[0115] In addition, in the reserved item list area AR31, first, list items corresponding to the reserved item identification records in which the item is identified are arranged in order, followed by list items corresponding to the unspecified items in which the item is not identified. In other words, in the reserved item list area AR31, the list items corresponding to the reserved item identification records in which the item is identified and list items corresponding to the unspecified items in which the item is not identified are arranged separately.

[0116] In the registered product list area AR3, list items corresponding to normal registration records are arranged following the pending product list area AR31. In the same figure, an example is shown in which the list items corresponding to normal registration records are arranged in the registered product list area AR3 in descending order of product price. Among the list items in the registered product list area AR3, the list item corresponding to the product specification record shows the product name, registered quantity, price according to the registered quantity, etc. of the specified product.

[0117] A Hold Release Declaration Button BT31 is located in the list item for each held item in the held item list area AR31. The Hold Release Declaration Button BT31 is a button that declares the release of the held item for the corresponding item. However, the Hold Release Declaration Button BT31 cannot be operated on the checkout screen in the figure, and can be operated on the held item processing screen described below. On the checkout screen, the Hold Release Declaration Button BT31 functions as an icon that indicates that the corresponding list item corresponds to a held item.

[0118] The reserved item release declaration button BT31 indicates the name of the reserved item classification to which the corresponding item falls. Specifically, for example, for age verification products (classification number 5), medicines (classification number 6), security tag products (classification number 7), and canceled products (classification number 8), the name indicating the corresponding classification is displayed on the hold release declaration button BT31. Also, for unspecified products (classification numbers 1 to 4), the name "hold product" is displayed collectively. In the figure, the display of the hold release declaration button BT31 shows an example in which the list item on the first line is displayed as "age verification" since it is an age verification product (classification number 5), the list item on the second line is displayed as "security tag" since it is a security tag product (classification number 7), and the list item on the third line is displayed as "pharmaceuticals" since it is a pharmaceutical product (classification number 6). Also, the list items on the fourth and fifth lines are each displayed as "hold product". In the list items in the fourth and fifth rows, thumbnail images TN are arranged. The thumbnail images TN are images captured by the customer terminal device 50 when a product registration operation is being performed for the corresponding unspecified product. The thumbnail images TN may be still images or moving images. Depending on whether or not a thumbnail image TN is present, the store clerk can visually distinguish between list items corresponding to unspecified products and list items corresponding to specified reserved products. Also, the store clerk can understand what the product registered as an unspecified product actually was based on the content displayed in the thumbnail image TN. In addition, the display indicating the corresponding classification on the hold release declaration button BT31 may be a picture such as an icon indicating the classification.

[0119] Among the list items of reserved goods, in the list items corresponding to identified goods such as age verification goods (classification number 5), pharmaceuticals (classification number 6), security tag goods (classification number 7), etc., the product name, registered quantity, price according to the registered quantity, etc. of the identified goods are displayed. In contrast, in the list item corresponding to an unidentified product, content according to the classification of the corresponding reserved product is displayed. Specifically, in the case of an unidentified product classified as NON-FILE (classification number 1), the read product code may be displayed together with an indication that it is NON-FILE, as in the list item in the fifth line of the figure. In addition, in the case of unidentified products (products for which a code has not been obtained) that are NG for reading (classification number 2), pseudo-scan input (classification number 3), and non-scan input (classification number 4), a message may be displayed indicating that the product code could not be read, as in the list item in the fourth line of the figure.

[0120] As shown in the figure, when the pending item list area AR31 is formed in the registered item list area AR3 on the checkout screen, a store clerk needs to perform an operation to process the pending items. When the checkout screen with the pending item list area AR31 formed in this way is displayed, the checkout device 40 executes a store clerk call to have the store clerk perform an operation to process the pending items. To call a store clerk, the checkout device 40 may, for example, cause a display such as a sign pole to be displayed in a predetermined manner, or send a notice of a store clerk call to a terminal device carried or monitored by the store clerk. As shown in the figure, when the checkout screen in which the reserved item list area AR31 is formed is displayed because the reserved item is included in the registration result, the checkout device 40 does not accept any operations related to checkout by the customer. Also, for example, in the area above the registered item list area AR3 on the checkout screen in the figure, a message is displayed to inform the customer that a store clerk who has been called will come to the checkout device 40.

[0121] The store clerk who goes to the settlement device 40 in response to the store clerk call performs a login operation to log in to the settlement device 40 as a store clerk. The login operation in this case may be, for example, an operation of having a scanner unit (customer-side scanner unit 406 or store clerk-side scanner unit 412) read a code (two-dimensional code or barcode) printed on the store clerk ID. Alternatively, it may be an operation of inputting a store clerk number or the like by, for example, entering a number. In response to the store clerk logging in, the display on the customer side display unit 405 transitions from the checkout screen to a reserved item processing screen.

[0122] Fig. 14 shows an example of a reserved item processing screen, in which the same parts as in Fig. 13 are given the same reference numerals. On the reserved item processing screen, the contents of the registered item list area AR3 are displayed in a state inherited from the checkout screen (FIG. 13). Therefore, in the initial display of the reserved item processing screen, the list items corresponding to the registered item information in the registered item list area AR3 are arranged in price order. In addition, the contents of the text and the like displayed in the reserved item release declaration button BT31 arranged in the list item for each reserved item in the reserved item list area AR31 are also inherited from the checkout screen (FIG. 13). Also, the display of some areas in which the list items in the reserved item list area AR31 are displayed and other areas with different visibility is also inherited. Furthermore, the visibility of area AR32 is also inherited to be displayed with the same visibility as that of the reserved item list area AR31. This also improves the visibility for store clerks.

[0123] On the left side of the area above the registered product list area AR3 on the reserved product processing screen in the same figure, it is indicated that this is a screen for product management, and a message is displayed to the store clerk instructing them to release the reserved products.

[0124] Also, sort buttons BT32 (price order button BT32-1, registration order button BT32-2, department order button BT32-3) are arranged on the right side of the upper region of the registered product list area AR3. The sort buttons BT32 are buttons for sorting list items corresponding to registered product information in the registered product list area AR3 according to the corresponding sort order. Of the sort buttons BT32, the price order button BT32-1 is a button for sorting list items corresponding to registered product information in price order. The registration order button BT32-2 is a button for sorting list items corresponding to registered product information in the order in which the customer registered them using the customer terminal device 50. The department order button BT32-3 is a button for sorting list items corresponding to registered product information by dividing them into departments to which the products correspond.

[0125] When a store clerk wishes to release a held item of a specific category (age verification items (category number 5), medicines (category number 6), security tag items (category number 7) from among the held items, the clerk performs the prescribed response procedure according to each category and then operates (e.g. taps) the hold release declaration button BT31 located next to the corresponding list item in the held item list area AR31. Note that the entire list item of held items may be made tappable. If the product to be released from hold is an age verification product, the store clerk will confirm that the customer is an adult as a customer service procedure, and then operate the hold release declaration button BT31 arranged on the corresponding list item. In addition, if the product to be released from hold is a medicine, the store clerk will, as a handling procedure, either explain the situation himself / herself if he / she is a licensed pharmacist, or ask another store clerk who is a licensed pharmacist to explain the situation, and then operate the hold release declaration button BT31 located on the corresponding list item. Furthermore, if the product to be released from hold has a security tag, the store clerk will, as a service of the store, remove the security tag attached to the product and operate the hold release declaration button BT31 located on the corresponding list item. When a hold is released for a held item of a category for which the item has been identified as described above, the display on the operated hold release declaration button BT31 changes from a state showing the relevant category as shown in FIG. 13 to a state indicating that the hold has been released, such as "Confirmed" (or "Released") as shown in FIG. 14.

[0126] In addition, when a store clerk wishes to release a pending item classified as an unspecified item (NON-FILE (classification number 1), NG to read (classification number 2), pseudo-scan input (classification number 3), non-scan input (classification number 4)), the store clerk first operates the hold release declaration button BT31 located in the list item in the held item list area AR31 that corresponds to the pending item to be released. In response to the operation of the hold release declaration button BT31 corresponding to the unspecified product, a status confirmation screen is displayed on the customer side display unit 405.

[0127] Fig. 15 shows an example of the status confirmation screen P10. The status confirmation screen P10 is a screen showing the status of the unspecified product corresponding to the operated hold release declaration button BT31 when it is registered by the operation of the customer terminal device 50. The status confirmation screen P10 may be displayed so as to be superimposed on the reserved product processing screen of Fig. 14, for example. The status confirmation screen P10 in the figure shows an example in which the unspecified product to be processed for hold release is classified as pseudo-scan input (classification number 3). The status confirmation screen P10 in the figure corresponds to a case where the customer terminal device 50 is held in a holder section provided in the cart, and in a state where it is held in the holder section, the customer terminal device 50 is capable of capturing an image of the inside of the shopping cart. The customer causes the customer terminal device 50 held in the holder section to scan the product code.

[0128] On the status confirmation screen P10 in the figure, the captured images G11, G12, and G13 are arranged from left to right in this order. The captured images G11, G12, and G13 are captured images obtained when the customer terminal device 50 was capturing images in an attempt to scan the product code of the corresponding unspecified product.

[0129] The order of the captured images G11, G12, and G13 arranged from left to right corresponds to the time course in which the images were captured. First, the second (center) captured image G12 is an image captured when the product to be processed is within the imaging range of the customer terminal device 50 during the product registration operation. The first captured image G11 is captured earlier than the second captured image G12, and is captured before the product to be processed falls within the image capturing range. The third captured image G13 is captured after the second captured image G12, and is captured at a timing when the product to be processed goes out of the imaging range.

[0130] In this case, the store clerk can understand the situation at the time of the product registration operation that caused the unidentified result by looking at the captured images G11, G12, and G13. Specifically, in the example shown in the same figure, the store clerk can first understand from the transition from captured image G11 to captured image G12 that the product to be processed was not in the shopping cart before the product registration operation was performed. Furthermore, the store clerk can see from captured image G12 that the product code on the product was hidden by the hand, resulting in an unidentified result of a false scan input (classification number 3). In addition, the store clerk can know what the product to be disposed of is from the captured image G12. Specifically, in the example shown in the figure, the store clerk knows that the product to be disposed of is a "can of yakitori." Furthermore, the store clerk can understand from the transition from captured image G12 to captured image G13 that the product to be processed was added to the shopping cart after the product registration operation was performed.

[0131] When the salesperson grasps the situation during the product registration operation as described above, the salesperson operates the screen transition button BT35 to display the product information input screen. In response to the operation of the screen transition button BT35, the product information input screen is displayed in place of the status confirmation screen P10. In addition, while the status confirmation screen P10 is displayed, the store clerk may perform an operation to have the scanner unit (customer-side scanner unit 406, store clerk-side scanner unit 412) of the settlement device 40 read the product code of the product to be processed that was placed in the shopping cart. When the settlement device 40 succeeds in reading the product code, a product information input screen is displayed.

[0132] 16 shows an example of a product information input screen P40. The product information input screen P40 is a screen on which a store clerk inputs product information of a product to be processed as an unspecified product. The product information input screen P40 may be displayed so as to be superimposed on the reserved product processing screen, for example, in the same manner as the status confirmation screen P10.

[0133] The product information input screen P40 has a product information area AR41, an input operation area AR42, a registration button BT41, a cancel button BT42, and a captured image area AR43 (a target captured image area AR43-1, a next captured image area AR43-2) arranged thereon.

[0134] The product information area AR41 is an area where product information about the corresponding unspecified product is displayed. The product information area AR41 in the figure displays the product name, product code, price, registered quantity, etc. The registered quantity can be input (registered) by operating the plus button and minus button arranged in the product information area AR41.

[0135] The input operation area AR42 is an area in which a software keyboard and buttons for inputting product information in the product information area AR41 are arranged. In the input operation area AR42, a keyboard having a key arrangement suitable for numeric input and a keyboard having a key arrangement suitable for character input can be switched according to the operation.

[0136] In the input operation area AR42, a preset call button BT421 is arranged. By operating the preset call button BT421, a preset key screen on which preset keys corresponding to each product are arranged is displayed. For example, if the unspecified product to be processed is either NG for reading (classification number 2), pseudo-scan input (classification number 3), or non-scan input (classification number 4), the store clerk may display the preset key screen if he knows the product name of the product to be processed. The store clerk can input product information of the product to be processed into the product information area AR41 by operating the preset key of the corresponding product from among the preset keys arranged on the displayed preset key screen.

[0137] Furthermore, in the input operation area AR42, a product code input declaration button BT422 is arranged. The product code input declaration button BT422 is a button that is operated when declaring that a product code input operation will be performed. The store clerk operates the product code input declaration button BT422, and then, for example, operates a keyboard arranged in the input operation area AR42 to input the numeric value of the product code printed or otherwise indicated on the product to be processed. This causes the product code to be input into the product information area AR41.

[0138] When the product information input screen P40 is displayed in response to a store clerk having the payment device 40 read a product code, the read product code has already been entered into the product information area AR41, and if the product is an unspecified product other than NON-FILE (classification number 1), for example, the payment device 40 has identified the product from the product master based on the product code, and the product name, price, etc. have also been input.

[0139] Furthermore, if the product to be processed is classified as NON-FILE (classification number 1), the product code is already entered in the product information area AR41 when the product information input screen P40 is displayed. In this case, the store clerk enters the product name, price, etc. in the product information area AR41 by operating the keyboard arranged in the input operation area AR42.

[0140] The register button BT41 is a button that is operated to confirm the information input in the product information area AR41 as described above, release the reservation of the corresponding unspecified product, and make the product a properly registered product. The CANCEL button BT42 is a button used to instruct the cancellation of the registration of the corresponding unspecified product.

[0141] The store clerk checks the information entered in the product information area AR41 and then operates the register button BT41. This causes the settlement device 40 to release the pending state of the unspecified product and process it to be included in the registered products. After the registration button BT41 is operated, the product information input screen P40 is cleared and the screen returns to the reserved product processing screen (FIG. 14). When the screen returns to the reserved product processing screen, the reserved product release declaration button BT31 for the list item of the product that was the subject of processing in the reserved product list area AR31 indicates that the reserved state has been released, for example by displaying "Confirmed."

[0142] Alternatively, if the store clerk determines that the item being processed should not be registered, he or she operates the cancel button BT42 to cancel the registration and removes the item from the shopping cart. Even after the cancel button BT42 is operated, the product information input screen P40 is cleared and the screen returns to the reserved product processing screen (FIG. 14). When the screen returns to the reserved product processing screen, the list item of the product that was the subject of processing in the reserved product list area AR31 has changed to a predetermined state indicating that the registration has been cancelled.

[0143] In addition, the classification number, classification name, etc. of the corresponding unspecified product may be displayed on the status confirmation screen P10 or the product information input screen P40.

[0144] In addition, the target image area AR43-1 in the captured image area AR43 is an area in which a target captured image is displayed. The target captured image is an image captured when the unspecified product to be processed this time is a product for which a code has not been obtained and is within the image capturing range during product registration operation. The captured image displayed in the target image area AR43-1 may be the same as the captured image G12 displayed on the status confirmation screen P10, for example. In the captured image area AR43, the next captured image area AR43-2 is an area where the next captured image is displayed. The next captured image is an image captured during a product registration operation performed following a product registration operation corresponding to a product whose code is not yet acquired and which is currently being processed. In other words, the next captured image is an image corresponding to a product registration operation in which the product code is successfully read for the first time and the product is identified after a product registration operation corresponding to a product whose code is not yet acquired and which is currently being processed. As the next captured image, if there is an image in which an item serving as a commodity is detected among the captured images obtained in response to the next commodity registration operation, the captured image including the detected commodity is selected.

[0145] When a customer performs a product registration operation for a certain product, and the customer terminal device 50 is unable to read the product code properly and registers the product as a product for which a code has not been acquired, the customer often tries to perform the product registration operation again for the same product. In such a case, the customer will perform the product registration operation twice in a row for the same product. In such a case, if the product code is read properly by the second product registration operation, the same product will be registered twice: as a pending product for which a code has not been acquired, and as a normal registration in which the product is identified based on the read product code. In the case where duplicate registrations have occurred in this way, the procedure for the store clerk to release the product on hold is to delete the registration of the product on hold as a product for which a code has not been obtained, and to leave the normal registration for the same product. To ensure that such processing is carried out smoothly, it is necessary to enable the store clerk to easily grasp whether the product for which a code has not been obtained that is the subject of release on hold has been duplicately registered or not.

[0146] Therefore, in this embodiment, the captured image area AR43 (target captured image area AR43-1, next captured image area AR43-2) is arranged on the product information input screen P40 to display the target captured image and the next captured image. If the target captured image and the next captured image show the same product, the store clerk can confirm that a duplicate registration has occurred. Therefore, the store clerk in this case, for example, operates the cancel button BT42 arranged on the product information input screen P40. As a result, the product to be processed that does not have a code is deleted from the list of registered products.

[0147] In the next captured image area AR43-2 in the figure, a quantity display section NM is arranged. The quantity display section NM is an image section that indicates the quantity (registered quantity) registered by, for example, a customer's operation for a commodity that has been properly registered by the commodity registration operation when the next captured image is captured. By providing the quantity display unit NM, the store clerk can grasp the registered quantities of the unspecified product to be released from hold and the products that have been correctly registered in overlap with the unspecified product to be released from hold at the stage of performing the hold release operation, and can accurately perform the tasks and operations related to the hold release.

[0148] If the product to be processed is an unspecified product as NON-FILE (classification number 1), the customer terminal device 50 notifies the customer that the product code has been successfully read but the product has not been specified. In this case, the customer can know that the product code has been successfully read, so there is little possibility that the same product will be registered again and duplicate registration will occur. Therefore, if the registration result of the product to be processed corresponds to NON-FILE, the captured image area AR43 (processing target image, next captured image) may not be displayed. However, some customers may mistakenly believe that reading the product code has failed even if a notification is given in response to the non-registration result as NON-FILE, and may perform the product registration operation again. For this reason, even if the registration result of the product to be processed corresponds to NON-FILE, the captured image area AR43 (processing target image, next captured image) may be displayed. If the captured image area AR43 is displayed, the store clerk can easily understand that the non-registration result of NON-FILE for the same product has been registered twice.

[0149] [Example of processing procedure] An example of a processing procedure executed by the customer terminal device 50 and the transaction management device 60 of this embodiment in relation to product registration will be described with reference to the flowchart of Fig. 17. In correspondence with Fig. 9, the processing in this figure shows the process of steps S5 to S11 more specifically as including processing corresponding to the registration result of an unspecified product.

[0150] First, an example of a processing procedure executed by the customer terminal device 50 will be described. Step S101: The customer terminal device 50 judges whether or not the reading of the product code has been successful while the product registration screen is being displayed.

[0151] Step S102: If the product code is successfully read, the customer terminal device 50 further determines whether the product registration result from the previous product registration operation was a code not obtained result (either reading NG (classification number 2), pseudo scan input (classification number 3), or non-scan input (classification number 4)). The result of code not being acquired corresponds to a case where a reserved product is registered as a product for which a code has not been acquired as a result of product registration.

[0152] Step S103: If it is determined in step S102 that the product registration result according to the previous product registration operation is a result of no code being acquired, the customer terminal device 50 acquires (extracts) an image in which the product is placed during the period corresponding to the current product registration operation from among the images captured by the imaging unit 506. The image acquired by the customer terminal device 50 in step S103 is used as the next captured image in the next captured image area AR43-2 of the product information input screen P40, which targets the product registered in the previous product registration operation and for which no code has been acquired, for which the pending release process is to be performed.

[0153] In addition, the format (video, still image), number of captured images, time, period, etc. of the captured images acquired by the customer terminal device 50 in step S103 may be varied depending on the classification of the detected product for which a code has not been acquired.

[0154] Step S104: If it is determined in step S102 that the code has not been acquired, or after the process of step S103, the customer terminal device 50 transmits the product registration information. The product registration information includes the product code read in response to step S101 and the cart identification information. If the process proceeds to step S104 after step S103, the customer terminal device 50 transmits product registration information further including the captured image acquired in step S103. If it is determined in step S102 that the code was not not acquired, and the product registration result corresponds to an unspecified product of NON-FILE (classification number 1), the customer terminal device 50 transmits product registration information indicating that the product registration result is NON-FILE (classification number 1).

[0155] Step S105: Upon receiving the registration completion notification sent from the transaction management device 60 in response to the sending of the product registration information in step S102, the customer terminal device 50 updates the product registration screen, for example as illustrated in FIG. 11(C).

[0156] Step S106: If it is determined in step S101 that the code was not read normally, or after the process of step S105, the customer terminal device 50 determines whether the product registration result according to the current product registration operation is a code not acquired result. A code not acquired result is a product registration result that corresponds to any of the reserved products of reading NG (classification number 2), pseudo-scan input (classification number 3), and non-scan input (classification number 4).

[0157] First, when the customer terminal device 50 recognizes that the entire barcode of the product has been captured in the captured image, but the reading of the barcode is not completed successfully (successful) after a certain period of time has elapsed, it determines that a code failure has occurred, indicating that reading was not successful (classification number 2).

[0158] In addition, if the product is recognized in the captured image but the barcode cannot be recognized at all and the barcode reading is not completed successfully, the customer terminal device 50 determines that a code not obtained result has occurred, which is a pseudo scan (classification number 3). Alternatively, the customer terminal device 50 may determine that a code not obtained result has occurred as a pseudo scan if the barcode reading is not completed normally because only a portion of the barcode of the product can be recognized in the captured image. Alternatively, the customer terminal device 50 may determine that a code not obtained result has occurred as a pseudo scan if the barcode of the product is recognized in the captured image but the barcode reading is not completed successfully because the product is moving too quickly. Such pseudo-scans, or results where the code was not obtained, may indicate that fraud is occurring, such as a customer simply pretending to have the product's barcode scanned, and then actually placing the product in their shopping cart.

[0159] In addition, the customer terminal device 50 set in the shopping cart can capture an image of the contents of the shopping basket placed in the shopping cart. When the customer terminal device 50 detects a state in which a new product has been added to the shopping basket even if no attempt to read the product's barcode is recognized in the captured image, the customer terminal device 50 determines that a code not obtained result has occurred as a non-scan input (classification number 4). For example, a customer may move an already registered product within the shopping cart in order to organize the contents of the shopping cart. The customer terminal device 50 may be configured to distinguish between such a movement of a product and the additional insertion of a new product corresponding to a result of not acquiring a code as a non-scan insertion, based on the captured image. The occurrence of such a result of no code being obtained, classified as a non-scan input, may indicate fraud, such as a customer simply putting an item into their shopping cart without having the customer terminal device 50 read the item's barcode.

[0160] In addition, when it is determined in step S106 that the product registration result is not a result of code not being acquired, this corresponds to a case where the customer terminal device 50 determines, as a result of analyzing the image captured by the imaging unit 506, that a product registration operation has not been performed.

[0161] Step S107: If it is determined in step S106 that the code was not acquired, the customer terminal device 50 acquires (extracts) images captured by the imaging unit 506 during the period corresponding to the current product registration operation. The captured images acquired by the customer terminal device 50 in step S107 are used as captured images G11, G12, and G13 to be displayed on the status confirmation screen P10 when the product with no code registered by the current product registration operation is subject to the hold release process. Also, the captured images acquired by the customer terminal device 50 in step S107 may also be used as the target captured image in the target captured image area AR43-1 on the product information input screen P40 when the product with no code registered by the current product registration operation is subject to the hold release process.

[0162] Step S108: The customer terminal device 50 transmits product registration information indicating the product registration result of the current code not being acquired to the transaction management device 60. The product registration information transmitted in step S108 also includes the captured image acquired in step S107.

[0163] Step S109: The customer terminal device 50 updates the product registration screen in response to receiving the registration completion notification transmitted from the transaction management device 60 in response to the transmission of the product registration information in step S108.

[0164] Step S110: If it is determined in step S106 that the code has not been acquired, or after the process of step S109, the customer terminal device 50 determines whether or not a settlement transition instruction operation has been accepted. If the settlement transition instruction operation has not been accepted, the process returns to step S101.

[0165] Step S111: When a payment transition instruction operation is received, the customer terminal device 50 converts, for example, the cart identification information into a two-dimensional code, and displays the generated two-dimensional code on the display unit 504, for example, using a code display screen as illustrated in FIG. 11(D).

[0166] Next, an example of a processing procedure executed by the transaction management device 60 will be described. Step S201: The transaction management device 60 determines whether or not product registration information transmitted from the customer terminal device 50 in step S104 or step S108 has been received.

[0167] Step S202: When the transaction management device 60 receives the product registration information, it identifies the cart information indicated by the cart identification information included in the received product registration information.

[0168] Step S203: The transaction management device 60 updates the cart information identified in step S202. That is, the transaction management device 60 generates registered product information or reserved product information based on the product code included in the product registration information received in response to step S201. The transaction management device 60 adds the generated registered product information or reserved product information to the cart information identified in step S202. As a result, one product is registered in the cart information. In addition, if the captured image is included in the product registration information received in step S201, the transaction management device 60 stores the captured image in association with the registered product information or reserved product information added to the cart information.

[0169] Step S204: After the process of step S203, the transaction management device 60 transmits a registration completion notice to the customer terminal device 50.

[0170] In the process of the figure, the customer terminal device 50 may delete captured images (i.e., target captured images, captured images not to be used as the next captured images) obtained by regular imaging at a predetermined timing that are not to be transmitted to the transaction management device 60. For example, when a product is identified and registered twice consecutively, it is determined that the captured image corresponding to the first registration will not be used as the next captured image. Therefore, the customer terminal device 50 may delete the captured image corresponding to the first registration at the timing when the product is identified and registered twice consecutively. Alternatively, the customer terminal device 50 may erase the captured images that were not transmitted to the transaction management device 60 at the timing when, for example, a settlement transition instruction operation is performed and product registration for all products is completed.

[0171] In the shopping system of the present embodiment, the customer terminal device 50 may transmit the product registration information for all registered products at once when the product registration for all products is completed, instead of transmitting the product registration information every time a product is registered. At this time, the customer terminal device 50 transmits the product registration information including the target captured image and the captured image to be used as the next captured image. When transmitting product registration information in bulk in this manner, the customer terminal device 50 may delete captured images that are not to be transmitted to the transaction management device 60 (target captured images, captured images that are not to be used as next captured images) for example, when product registration for all products is deemed to be completed.

[0172] An example of a process performed by the settlement device 40 in relation to releasing the reserved item will be described with reference to the flowchart of FIG. Step S301: The settlement device 40 waits for a store clerk who will release the hold to log in while displaying a settlement screen including a list of held items on the customer display unit 405 as illustrated in FIG.

[0173] Step S302: In response to the store clerk logging in, the settlement device 40 changes the display on the customer side display unit 405 from the settlement screen to the reserved item processing screen (FIG. 14).

[0174] Step S303: While the reserved item processing screen is displayed, the settlement device 40 determines whether or not an unspecified record designation operation has been performed. The unspecified record designation operation is an operation on the reserved item release declaration button BT31 arranged in a list item corresponding to an unspecified item among the list items of reserved items (unspecified records) in the reserved item list area AR31.

[0175] Step S304: When an unspecified record designation operation is performed, the settlement device 40 causes the customer display unit 405 to display a status confirmation screen P10 which indicates that the unspecified product corresponding to the list item designated by the unspecified record designation operation is to be processed for release from hold.

[0176] Step S305: With the status check screen P10 displayed, the settlement device 40 waits for a screen transition trigger to the product information input screen P40 to occur. The screen transition trigger is acceptance of an operation on the screen transition button BT35 on the status check screen P10, or completion (success) of reading the product code by the settlement device 40.

[0177] Step S306: When a screen transition trigger occurs, the settlement device 40 displays the product information input screen P40 for the unspecified product to be processed. Step S307: While the product information input screen P40 is displayed, the settlement device 40 inputs product registration information (product code, product name, price, quantity, etc.) in response to the product registration information input operation performed by the store clerk. Step S308: The settlement device 40 transmits the registered product information or a cancellation request to the transaction management device 60 in response to the operation of the registration button BT41 or the cancellation button BT42. The registered product information is sent in response to the operation of the registration button BT41, and has contents reflecting the information entered in the product information area AR41. The cancel request is sent in response to the operation of the cancel button BT42, and is a command requesting that the pending product information to be processed and stored in the cart information be cancelled.

[0178] Step S309: If it is determined in step S303 that the unspecified record designation operation has not been performed, the settlement device 40 determines whether or not a reserved item confirmation operation has been performed. The reserved item confirmation operation is an operation on the reserved item release declaration button BT31 in a list item corresponding to a reserved item that is not an unspecified item. In other words, reserved items that are not unspecified items are age verification items (classification number 5), medicines (classification number 6), and security tag items (classification number 7), which have been identified. For these reserved items that are not unspecified items, the store clerk performs the specified response work and then operates the reserved item release declaration button BT31 in the corresponding list item, thereby releasing the reserved item.

[0179] Step S310: When a reserved product confirmation operation is performed, the settlement device 40 transmits to the transaction management device 60 a reserved product release request corresponding to the reserved product for which the reserved product confirmation operation has been performed this time.

[0180] Step S311: In response to receiving the registered product information or the cancellation request sent from the settlement device 40 in step S308, the transaction management device 60 updates the corresponding cart information. That is, when the transaction management device 60 receives the registered product information, it changes the reserved product information of the target product to product registration information based on the received registered product information. At this time, the transaction management device 60 may change, for example, the reserved release flag included in the product information to indicate that the product has been released from reserved. Furthermore, when the transaction management device 60 receives a cancellation request, it changes the reserved product information of the target product to indicate that it has been canceled. Furthermore, in response to receiving the hold release notification sent from the settlement device 40 in step S310, the transaction management device 60 changes the product information type so that the reserved product information of the target product in the cart information is managed as registered product information. In this case, the transaction management device 60 may, for example, change the hold release flag included in the product information to indicate that it has been released from hold. After updating the cart information as described above, the transaction management device 60 transmits settlement information corresponding to the updated cart information to the settlement device 40. The settlement device 40 receives the transmitted settlement information.

[0181] Step S312: The settlement device 40 determines whether or not all reserved items have been released from the reserved state. If there are any reserved items that have not yet been released from the reserved state, the process returns to step S302. When the process returns from step S312 to step S302, the settlement device 40 displays the reserved product processing screen updated based on the contents of the settlement information received in step S311.

[0182] Step S313: If it is determined in step S312 that all reserved items have been released from reservation, the settlement device 40 displays a settlement screen reflecting the contents of the settlement information received in step S311. Since no list items of reserved items are displayed in the registered item list area AR3 of the settlement screen displayed in step S312, the settlement device 40 is in a state where it can execute settlement processing in response to subsequent operations by the customer.

[0183] <Modification> Modifications of the present embodiment will be described below. The modifications described below may be combined with the above embodiment and other modifications as appropriate.

[0184] [First Modification] In the above embodiment, the transaction management device 60 identifies the product based on the product code read by the customer terminal device 50. Product identification is not limited to such a process. For example, the customer terminal device 50 may transmit an image of the product to the transaction management device 60, and the transaction management device 60 may identify the product by comparing the received image with a database of product images.

[0185] [Second modified example] The next captured image may also be displayed, for example, on the status confirmation screen P10. Fig. 19 shows an example of the status confirmation screen P10 according to this modified example. In the status confirmation screen P10 in the figure, in addition to the captured images G11, G12, and G13 captured when the product registration operation was being performed for the unspecified product, the next captured image G21 is displayed. In this case, the store clerk will be able to determine whether or not a duplicate registration has occurred at the stage when the status confirmation screen P10 is displayed before the product information input screen P40 is displayed. The arrangement of the captured images G11, G12, and G13 and the next captured image G21 on the status check screen P10 is not limited to the example shown in the figure. For example, the next captured image G21 may be arranged in an area above or below the area where the captured images G11, G12, and G13 are arranged. Alternatively, the next captured image may be arranged so as to fit within the range of any of the captured images G11, G12, and G13. Alternatively, the next captured image may be displayed on the status check screen P10 in a state where the captured images G11, G12, and G13 are displayed but the next captured image G21 is not displayed, in response to an operation on a button for displaying the next captured image, for example. Incidentally, a quantity display section NM similar to that illustrated in FIG. 16 may be arranged in correspondence with the next captured image G21 shown in FIG.

[0186] [Third Modification] On the checkout screen (Figure 13) or the pending product processing screen (Figure 14), for example, a list item of a pending product as an unspecified product (unspecified record) may be arranged in a chronological order relative to the list item of the product that is first successfully registered by a subsequent product registration operation (successfully registered record), regardless of the sort order. Fig. 20 shows an example of a checkout screen corresponding to this modified example. In this figure, the list items of unspecified products for which codes have not been acquired (reserved products that fall into any of the following categories: reading NG (classification number 2), pseudo-scan input (classification number 3), and non-scan input (classification number 4)) are arranged in a reverse order to the list items of products that were first successfully registered after the registration of the unspecified products. In the figure, an example of the arrangement order is shown in which the list item of the product for which the code has not been acquired is placed first, followed by the list item of the product that is registered correctly first after the registration of the product for which the code has not been acquired. In addition, in the figure, the list item of the product for which the code has not been acquired and the list item of the product that is registered correctly are surrounded by a single frame FL. This makes it easier to visually grasp the pair of the list item of the product for which the code has not been acquired and the list item of the product that is registered correctly, which may be registered twice. In addition, the settlement device 40 displays the partial area where the area FL1 is displayed and the area different from the partial area so that the visibility is different. Note that in FIG. 13 and FIG. 14, there is one partial area (only AR31), but in FIG. 20, AR31 and FL1 correspond to the partial area. In other words, there may be multiple partial areas.

[0187] [Fourth Modification] In the above embodiment, when a reserved item is released from reservation, the corresponding list item displayed in the reserved item list area AR31 of the reserved item processing screen is shown to have been released from reservation by, for example, changing the display of the release reservation declaration button BT31. In this case, the list item of the reserved item that was the subject of release from reservation is placed in the same position in the reserved item list area AR31 even after the item is released from reservation. However, when the list item of the pending product is released from the hold, it may be moved to the registered product list area AR3 outside the held product list area AR31 and arranged as a normal registration record according to the currently set arrangement order (sort order).

[0188] [Fifth Modification] In the above embodiment, the captured image corresponding to the product registration operation in which the product is first identified after the unspecified product is registered is displayed as the next captured image. In contrast to this, in this modified example, even if the next product registration operation after an unspecified product is registered results in a registration result that the product is an unspecified product, the captured image corresponding to the next product registration operation can be displayed as the next captured image. Alternatively, even if the next product registration operation performed within a certain time after an unspecified product is registered results in a registration result that the product is an unspecified product, the captured image corresponding to the next product registration operation can be displayed as the next captured image. For example, if a customer performs a product registration operation for a certain product, and the product is registered as an unspecified product, when the customer performs the product registration operation again for the same product, the product may be registered as an unspecified product in the same way. In other words, when registering a product, the same product may be registered twice, resulting in duplication of registration results as an unspecified product. With the configuration of this modified example, the store clerk can easily recognize such double registrations and can take measures to deal with them quickly. Furthermore, if the next product registration operation after an unspecified product has been registered as described above results in the product being registered as an unspecified product, or if a product registration operation performed within a certain period of time after an unspecified product has been registered results in the product being registered as an unspecified product, the transaction management device 60, for example, may process the same product as being registered as an unspecified product and treat it as a single unspecified record.

[0189] [Sixth Modification] In addition, the customer terminal device 50 may be configured to constantly transmit the captured images obtained by the imaging unit 506 to the transaction management device 60, and the transaction management device 60 may then be configured to acquire captured images corresponding to the code not being obtained result depending on the time the code not being obtained result occurred, the classification result, the image analysis result, etc.

[0190] [Seventh Variation] In addition, the transaction management device 60 may be configured to store the captured images that are constantly transmitted by the customer terminal device 50, and when the customer terminal device 50 or the transaction management device 60 determines that a code not obtained result has occurred based on the captured images, the relevant captured image may be managed as corresponding to the code not obtained result, for example by assigning a flag or identification information to the captured image.

[0191] [Eighth Modification] The next captured image displayed on the settlement device 40 may be displayed as a result of a search by a store clerk. For example, when an unspecified product is the target for release from hold, the store clerk performs an operation on the settlement device 40 to instruct the settlement device 40 to search for a captured image corresponding to the product that was successfully registered first after the unspecified product was registered. The settlement device 40 requests the transaction management device 60 to search for the instructed captured image. The transaction management device 60 executes the search in response to the request, and transmits the searched captured image to the settlement device 40. The settlement device 40 displays the received captured image as the next captured image.

[0192] [Ninth Variation] In addition, based on the results of analyzing changes in the image obtained by the customer terminal device 50 capturing an image of the inside of the shopping cart, the customer terminal device 50 or the transaction management device 60, etc., may be able to detect the occurrence of situations such as when a registered item is not added to the shopping cart, or when an item is added to the shopping cart without being registered, for example, by a person accompanying the customer. In addition, when the occurrence of the above-mentioned situation is detected, the customer terminal device 50 may provide guidance to the customer, for example by display, to have the customer place the registered product in the shopping cart, or to have the customer register an unregistered product that is in the shopping cart.

[0193] [Tenth Modification] On the product information input screen P40 in Fig. 16, it may be possible to search for products registered in the corresponding transaction. Furthermore, the search possible in this modified example may be a fuzzy search that searches for products that "contain" the input keyword. For example, a button for instructing a fuzzy search is placed on the product information input screen P40 of this modified example. When this button is operated, a character input keyboard with, for example, the Japanese syllabary (or the alphabet) is displayed in the input operation area AR42. The store clerk operates the character input keyboard to input a keyword, which may be part of the name of the product to be searched for. At this time, it may be possible to convert the hiragana characters entered as the keyword into a word. The store clerk then operates the search execution button placed on the product information input screen P40. In response to the operation of the search execution button, the transaction status management device 80 executes a fuzzy search and displays the search results. The search results may be displayed, for example, as a pop-up screen. In addition, the search results may display the product name, price, registered number, etc. of each searched product. The sales clerk performs an operation to identify an unspecified product to be processed this time from among the products presented as the search results, and select the identified product. In response to this operation, for example, the display of the product information input screen P40 may be returned to. In the displayed input operation area AR42, the product name, price, registration quantity, product code, etc. of the selected product are input. The sales clerk operates the registration button BT41 after checking the contents displayed in the input operation area AR42. This enables the sales clerk to efficiently release the reservation of the unspecified product without having to, for example, input a number with many digits as a product code or narrow down the product classification.

[0194] [Eleventh Modification] For example, a moving image (or a time lapse image) of a captured image corresponding to a product registration operation when a target unspecified product is registered may be displayed by an operation on a screen such as the status confirmation screen P10 in Fig. 15 (and Fig. 19) or the product information input screen P40 in Fig. 16. The operation to display the moving image may be, for example, an operation on a video display button arranged on the status confirmation screen P10 or the product information input screen P40, or may be, for example, an operation on the captured image G12 on the status confirmation screen P10 or the target captured image area AR43-1 or the next captured image area AR43-2 on the product information input screen P40. For example, when the next captured image is displayed, a moving image from when the product registration operation is started when the target unspecified product is registered to when the product corresponding to the next captured image is captured may be displayed. The moving image may be deleted, for example, when the settlement is completed. However, if the transaction is ended without completing the settlement for some reason, the moving image may be stored for a certain period of time after the transaction is ended so that the store clerk can check it even after the transaction is ended. The moving image may be stored in the transaction management device 60 or the transaction status management device 80 in association with the date, member identification information, etc. The stored moving image can be played back on a terminal such as the transaction status management device 80.

[0195] [Twelfth Modification] In the above embodiment and modified examples, part of the processing executed by the customer terminal device 50, the transaction status management device 80, etc. may be executed by a higher-level device such as a transaction management device 60 connected to the customer terminal device 50 via communication. Also, in the above embodiment and modified examples, part of the processing executed by the settlement device 40 may be executed by a higher-level device such as a transaction management device 60 connected to the settlement device 40 via communication. As one of the above aspects, the shopping system of this embodiment may be provided with the configuration of a thin client system. As an example, the transaction management device 60 may be configured as a thin client-compatible server (e.g., a cloud server), and the customer terminal device 50, transaction status management device 80, etc. may function as thin clients. In this case, the thin client terminals such as the customer terminal device 50 and transaction status management device 80 transmit information such as captured images and commands corresponding to operations performed on the thin client terminals to the transaction status management device 80. The transaction status management device 80 may execute processing in response to information received from the thin client terminal, and transmit image data or web page data as a screen reflecting the processing results to the thin client terminal. The customer terminal device 50, transaction status management device 80, etc. as thin client terminals display the transmitted image data or web page data by an image viewer or web browser.

[0196] [13th Variation] Next, a control for displaying a part of the display area (hereinafter sometimes referred to as a "focused area") and a different area from the part of the display area with different visibility will be described. The background to such control is the recent increase in the size of displays. For example, warning screens and operation screens are displayed on displays by pop-ups or the like, but as displays have become larger, customers and store clerks are confused about where to look, which leads to delayed operation and easy occurrence of erroneous operation.

[0197] In particular, when customers purchase products, they get very close to the POS main unit, so they can only see a localized portion of the display due to their viewing angle. Furthermore, in recent years, customers may perform operations to determine payment methods, but they may not know where to look on the display, which can make them panic and result in incorrect operations. Even if the operation is performed correctly, there is also the issue that the time required to handle the customer increases because it takes more time to think about where to look and what to do.

[0198] As a result of the display becoming larger, there is a problem that visibility for the operator is reduced.

[0199] An example of control for solving such problems will be described. In the above embodiment, an area in which pending products are displayed is taken as an example of a portion of the area, but the present invention is not limited to this. As shown below, the attention area displays attention information that calls the operator's attention (e.g., information indicating that a store clerk is being called), or reception information that accepts the operator's operation (e.g., information that accepts the selection of a payment method). The settlement device 40 of this embodiment is an example of a sales processing device. The operator refers to a person who operates the settlement device 40, such as a store clerk or a customer.

[0200] Fig. 21 is a diagram showing an example of a screen display. As shown in Fig. 21, the screen is a portrait-oriented screen. Fig. 21(A) is a diagram showing a screen in an idle state waiting for payment processing. Fig. 21(B) is a diagram showing a screen indicating that a store clerk is being called, which is displayed as a pop-up in area AR502 when payment processing is performed.

[0201] As shown in FIG. 21B, the attention area AR502 and the area AR501 different from the attention area AR502 are displayed with different visibility. In this way, by displaying with different visibility, in the case of FIG. 21B, the operator can narrow down to two areas AR501 and AR502, and can select an area to be noted from these two areas, so that the operator's visibility can be improved compared to the case where the visibility is not different. Note that the area AR502 is superimposed on the area AR501, but is not included in the area AR501, and the area AR501 is the entire area within the broken line of the area AR501 excluding the area AR502. That is, the area AR501 and the area AR502 are different areas. In this way, the area AR501 and the area AR502 may be different areas, or the area AR502 may be superimposed on the area AR501. In the following description, two regions that appear to be overlapped may be different regions, or one region may overlap the other region.

[0202] Fig. 21C is a diagram showing a state where a screen for inputting information required for maintenance is displayed as a pop-up in an area AR503 when performing maintenance on an apparatus. As shown in Fig. 21C, an area of ​​interest AR503 and an area AR501 different from the area of ​​interest AR503 are displayed with different visibility. By displaying the areas with different visibility in this way, even in the case of Fig. 21C, the operator can narrow down the areas to two areas AR501 and AR503, and can select an area to pay attention to from these two areas. This improves the operator's visibility compared to when the areas are not displayed with different visibility.

[0203] In the following description, a grayed-out state such as AR501 may be referred to as a blackout display, and a region in which a blackout display is performed may be referred to as a blackout region.

[0204] Next, a control for displaying the visibility of the auxiliary area, which displays information that supplements the information displayed in the attention area, with the same visibility as the attention area will be described. FIG. 22 is a diagram showing a checkout screen. FIG. 22(A) is a diagram showing a checkout screen similar to FIG. 13, which displays the subtotal amount of the transaction and a list of registered products displayed on the checkout device after steps 12 to 17. All products registered by the customer are displayed, and if the number of products does not fit on one page, the product information is displayed on the next page as well. By pressing the "Reserved Products" button on this screen, only the reserved products present in the transaction are extracted and displayed together on one screen. FIG. 22(B) shows a screen (pop-up) that displays only reserved products together.

[0205] In comparison with the checkout screen of FIG. 22(A), in the checkout screen shown in FIG. 22(B), the area AR511 is displayed in a dark state, but the areas AR510, 512, and AR513, which should be kept in focus, are not displayed in a dark state. In addition, the information displayed in the area AR510, which displays the estimated total (or total), the total number of purchases, and the number of items by price range, and the area AT513, which displays the number of reserved items, are information that supplements the information displayed in the area AR512, which displays the reserved item list. In this way, in FIG. 22(B), the visibility of the auxiliary areas AR510 and 513, which display information that supplements the information displayed in the attention area AR512, is displayed with the same visibility as the attention area. In this way, the operator can more accurately grasp the displayed contents by the auxiliary information, compared to the case where the areas other than the attention area are uniformly displayed in a dark state. Furthermore, when the "Could not be scanned" field displayed in the reserved item list of FIG. 22(B) is pressed, the screen transitions to the next screen of FIG. 23 in order to move to the task of identifying the items that could not be scanned.

[0206] FIG. 23(A) is a diagram showing the status confirmation screen of FIG. 15 and the product information input screen of FIG. 16 side by side on one screen. That is, the screen for status confirmation and the product information input screen for input based on the confirmation are integrated into one screen, and the screen is processed collectively without screen transition. The product information input screen of FIG. 23(A) shows an area AR520 for input operation, an area AR521 showing the estimated total (or total) and the status confirmation screen, and an area AR522. Of these, the area AR522 is displayed in a darkened state. In FIG. 23(A), the area AR520 corresponds to the attention area, and the information shown in the area AR521 is information that supplements the information displayed in the area AR520. Thus, in FIG. 23(A), the visibility of the auxiliary area AR521 that displays information that supplements the information displayed in the attention area AR520 is displayed with the same visibility as the attention area. In this way, the operator can more accurately grasp the display contents by the auxiliary information compared to the case where the areas other than the attention area are uniformly darkened. Furthermore, the product information input screen in Fig. 23(A) is a screen that requires input operations, unlike the checkout screen shown in Fig. 22, but even when such input operations are required, the operation can be performed more accurately.

[0207] FIG. 23(B) is a diagram showing a payment method selection screen. The payment method selection screen in FIG. 23(B) shows areas AR526, AR527, and 528 for performing input operations, and area AR529 showing the estimated total (or total) and the like. Of these, areas AR527 and 528 are displayed in black. The payment method selection screen in FIG. 23(B) is a screen that requires input operations, but even when such input operations are required, it is possible to operate more accurately. In particular, when there is a queue at the POS, customers tend to panic and are likely to make erroneous operations. Since selecting a payment method is an important operation, it is very effective to improve the visibility of the operator in order to avoid erroneous operations as much as possible.

[0208] Fig. 24 is a diagram showing an example of a screen that is gradually darkened. In Fig. 24(A), area AR530 corresponds to the attention area, area AR531 that displays the total amount of the transaction, the total number of items purchased, and the number of items by price range, and area AR532 for the checkout button, which is an operation button for declaring the end of the transaction and transitioning to a payment type selection screen (e.g. Fig. 23(B)), correspond to the auxiliary area, and area AR533 corresponds to the darkened area. Of these, area AR530 is an area where a customer performs an input operation to select items to eat or drink in the store.

[0209] For products displayed in the area AR530 that can be switched between take-out and eat-in, a check box is displayed at the beginning of the product name, and when the customer selects a product to be eaten in the store (eat-in) by touching or the like, a check mark is placed on the check box. When the customer presses the checkout button displayed in the area AR532 in a checked state, the tax rate of the checked product is changed and the recalculated amount is displayed, and the screen shown in FIG. 24(B) is displayed. The area AR534, which is a screen requesting approval to eat the checked product in the store, corresponds to the attention area, and the area AR535 corresponds to the blackout area. Pressing "Yes" in the area AR534 transitions to the payment type selection screen (FIG. 23(B)), and pressing "No" causes the previous product selection screen, area 530, to be displayed again. In this way, the display is gradually blacked out, thereby improving the operator's visibility. The latest area to be displayed later is controlled to be the area of ​​interest, and the previous area of ​​interest is controlled to be the area of ​​non-interest (blacked out), while the area AR531 is controlled to be constantly maintained as the auxiliary area.

[0210] Next, a setting example for setting the display position of the attention area will be described. When the screen is vertically long and there are many operators who are relatively short, the display position of the attention area is set toward the bottom of the display area. Here, the bottom will be described with reference to FIG. 25. Area AR540 in FIG. 25 is the entire display area of ​​the display. AR541 is the attention area. Dashed line 542 indicates a line segment that divides area AR540 vertically into equal parts. Dashed line 543 indicates a line segment that divides area AR541 vertically into equal parts.

[0211] In light of this, "lower" means that line segment 543 is present below line segment 542. In other words, the area where region AR541 intersects with the lower half of region AR540 is larger than the area where region AR541 intersects with the upper half of region AR540. In this way, when there are many operators who are relatively short, it is possible to improve visibility for many operators.

[0212] When the screen is vertically long and there are many relatively tall operators, the display position of the attention area is set toward the top of the display area. "To the top" means that line segment 543 is located above line segment 542. In other words, the area where area AR541 intersects with the lower half of area AR540 is smaller than the area where area AR541 intersects with the upper half of area AR540. In this way, when there are many relatively tall operators, the visibility of many operators can be improved.

[0213] The ability to set the display position of the attention area allows for applications other than those described above. Specifically, the display position can be set according to the height at which the housing of a POS such as a settlement device is installed. When the housing or display is installed high, the display position of the attention area is set low. On the other hand, when the housing or display is installed low, the display position of the attention area is set high. In this way, the display position can be set flexibly, thereby improving the operator's visibility according to the usage mode of the store. Specifically, the display position of the attention area at the POS can be determined by measuring the distance from where the customer is standing to the bottom (or top) of the screen display and inputting the measured value into the setting screen of the POS.

[0214] In the twelfth modified example, the case where the screen is vertically long has been described, but the display position may also be set in the case where the screen is horizontally long. For example, if the POS cabinet is on the customer's left hand side, the display position of the attention area is set to the right of the display area. The right shift will now be described with reference to FIG. 26. Area AR550 in FIG. 25 is the entire display area of ​​the display. AR551 is the attention area. Dashed line 552 indicates a line segment that divides area AR550 horizontally into equal parts. Dashed line 543 indicates a line segment that divides area AR551 horizontally into equal parts.

[0215] In light of this, "rightward" means that line segment 553 is located to the right of line segment 552. In other words, the area where region AR551 intersects with the right half of region AR550 is larger than the area where region AR551 intersects with the left half of region AR550. In this way, the visibility for the operator can be improved.

[0216] When the screen is long in landscape orientation and the POS cabinet is on the customer's right hand side, the display position of the attention area is set to the left of the display area. Leftward means that line segment 553 is to the left of line segment 552. In other words, the area where area AR551 intersects with the right half of area AR550 is smaller than the area where area AR551 intersects with the left half of area AR550. This can improve the operator's visibility. Specifically, by using a camera (described later), the customer's position in relation to the screen is identified, and if the customer is standing to the right of the screen, the area of ​​interest is displayed on the right, and if the customer is standing to the left of the screen, the area of ​​interest is displayed on the left. Furthermore, it is possible to follow the customer's movements and always display the area of ​​interest in front of the customer.

[0217] The above-mentioned upper, lower, left, and right positions may be combined. For example, if the POS cabinet is on the customer's right hand side and the display is located at a relatively high position, the attention area may be displayed lower and to the left. If the POS cabinet is on the customer's left hand side and the display is located at a relatively low position, the attention area may be displayed upper and to the right.

[0218] Next, a control for setting the display position of a part of the area according to the height of the operator will be described. In recent years, POS devices are sometimes equipped with a camera for recording customer operations and change. The customer's approximate height is measured from the image of the customer captured by this camera. Then, in proportion to the measured height, the display position of the attention area is set to a higher position if the customer is tall, and to a lower position if the customer is short. In this way, the display position is set according to the height of any operator, thereby improving the visibility of the operator.

[0219] The display itself may be moved. That is, the display is moved to a higher position if the customer is taller, and moved to a lower position if the customer is shorter, in proportion to the customer's height measured from the image of the customer captured by the camera. In this way, the display position is set according to the height of the operator, regardless of the operator's height, and visibility for the operator can be improved.

[0220] In the embodiment described above, a portion of the area and an area different from the portion of the area are displayed so as to have different visibility, but the portion of the area and an area surrounding the portion of the area may be displayed so as to have different visibility.

[0221] FIG. 27 is a diagram showing a display example in which a partial area and an area surrounding the partial area have different visibility.

[0222] 27A, an area AR560 is the entire display area of ​​the display. An area AR562 is a focus area. An area AR561 is a periphery of a portion of the area. In this manner, the visibility of the area AR562 and the surrounding area AR561 may be made different. This also improves the visibility of the focus area compared to a case in which the visibility is not made different.

[0223] An area AR560 in FIG. 27B is the entire display area of ​​the display. An area AR563 is the frame of the attention area. In other words, the area AR563 is the periphery of a portion of the area. In this manner, the visibility of the attention area and its frame may be made different. This also improves the visibility of the attention area compared to a case where the visibility is not made different.

[0224] The display manner of the area AR561 and the area AR563 in Fig. 27 may be changed. For example, by changing the color or the pattern, it is possible to naturally draw attention to the attention area surrounded by the area AR561 and the area AR563.

[0225] FIG. 28 is a diagram showing a display example of a darkened area. In FIG. 28, area AR700 indicates the attention area. Area AR701 indicates the darkened area. FIG. 28(A) shows an example of darkening using diagonal lines. FIG. 28(B) shows an example of darkening using dots. FIG. 28(C) shows an example of darkening by filling in. In either case, the information displayed before the darkening may be made slightly transparent or may not be made transparent, which masks information in areas different from some areas, thereby improving the visibility of the attention area.

[0226] Note that the areas other than the area of ​​interest may be displayed in monochrome. Since color display is the standard for displays, by displaying the areas other than the area of ​​interest in monochrome, the color display of the area of ​​interest stands out, improving the visibility of the area of ​​interest. It is not necessary to display the area in a darkened state like this.

[0227] Alternatively, instead of blacking out the image, blurring or mosaic processing may be performed on the areas other than the area of ​​interest. The blurring or mosaic processing makes it impossible to recognize images or characters drawn in the areas other than the area of ​​interest, thereby improving the visibility of the area of ​​interest.

[0228] Also, the luminance of the area other than the attention area may be reduced. By reducing the luminance, images and characters drawn in the area other than the attention area become unrecognizable, thereby improving the visibility of the attention area.

[0229] In general, the human eye is likely to focus on moving objects. This property may be utilized to dynamically change the area of ​​interest. FIG. 29 is a diagram showing an example of dynamically changing the area of ​​interest. Area AR711 in FIG. 29 is the entire display area of ​​the display. Area AR710 is the frame of the area of ​​interest.

[0230] 29, the size of the area AR710 is gradually increased, which naturally draws attention to the area AR710, thereby improving the visibility of the area of ​​interest.

[0231] As another example, the visibility of the area of ​​interest can be improved by dynamically changing the area of ​​interest, such as by sliding the area of ​​interest from above, below, left, or right, or by rotating the area of ​​interest.

[0232] In addition, a video may be displayed to draw attention to the attention area. For example, a video in which the word "attention" and an arrow flow into the attention area can be displayed to improve the visibility of the attention area.

[0233] The operator may be prompted to touch the screen, and the attention area may be displayed at the touched location. In this case, the operator is looking at the location where the attention area is displayed from the beginning, which improves the visibility of the attention area.

[0234] The area in which the attention area is displayed may be fixed. In this case, the position of the attention area is notified to the operator in advance, or the operator recognizes that the position of the attention area is fixed while using the device several times. In this way, the operator first sees the attention area, and as a result, the visibility of the attention area can be improved. The operator here may be a store clerk or a customer as described above, but this is particularly effective in operations aimed at customers who are unfamiliar with operations. For example, this is effective in full self-service devices and ticket vending machines that leave registration and settlement to the customer, and self-service settlement devices that leave only settlement to the customer.

[0235] As described above, according to this embodiment, the visibility of the attention area can be improved. This allows the operator to immediately understand where to look on the display, thereby suppressing the occurrence of erroneous operations. In addition, compared to when the visibility of the attention area is not improved, the time required to think about where to look and what to do is reduced, which reduces the time required to handle customers and the operation time of store clerks, thereby enabling optimal store management.

[0236] In the above embodiment, an embodiment using the settlement device 40 has been described, but the present invention is not limited to settlement devices. This embodiment can be applied to any device equipped with a display unit. Examples of devices equipped with a display unit include label printers, label attachment devices, packaging devices, various inspection devices, and weighing devices, to which this embodiment can be applied. By applying this embodiment to a device equipped with a display unit, the device can also obtain the effects obtained from this embodiment.

[0237] <Overview of the embodiment> [Technical field] The present invention relates to a sales processing device, a program, and a control method. [Background technology] There is a POS (Point Of Sales) provided with a display (see, for example, Patent Document 1). [Prior art document] [Patent documents] [Patent Document 1] Patent No. 5559252 [Summary of the Invention] [Problem to be solved by the invention]

[0238] Displays have become larger in recent years. The larger the display, the greater the amount of information that can be displayed, which has created a new problem: it is difficult for customers and store clerks to know where to look on the display.

[0239] The present invention has been made in consideration of the above circumstances, and has an object to improve visibility for an operator.

[0240] [Means for solving the problem] (1) As described above, one aspect of this embodiment includes a display means for displaying information related to a transaction, and a display control means for displaying a portion of the display area of ​​the display means (e.g., AR502 in FIG. 21) and an area other than the portion (e.g., AR501 in FIG. 21) so as to have different visibility.

[0241] According to the above configuration, by displaying with different visibility, the operator can narrow down the area to which he or she should pay attention, thereby improving the operator's visibility.

[0242] (2) One aspect of this embodiment is the sales processing device described in (1), wherein the display control means displays the visibility of an auxiliary area (e.g., area AR510, area AR512 in FIG. 22(B)) that displays information auxiliary to the information displayed in the partial area (e.g., area AR512 in FIG. 22(B)) with the same visibility as the partial area.

[0243] According to the above configuration, the auxiliary information allows the operator to more accurately grasp the display contents, compared to a case where the areas other than the area of ​​interest are uniformly darkened.

[0244] (3) One aspect of this embodiment is a sales processing device as described in (1) or (2), which is provided with a position setting means for setting a display position of the portion of the area in the display area, and the position setting means sets the display position of the portion of the area to a position near the bottom or top of the display area, or according to the height of the operator.

[0245] According to the above configuration, the visibility of the operator can be improved according to the tendency of the operator's height and the usage mode of the store. By setting the display position lower, the visibility of the operator can be improved when there are many operators who are relatively short. By setting the display position higher, the visibility of the operator can be improved when there are many operators who are relatively tall. Furthermore, by setting the display position according to the height of the operator, the display position is set according to the height of the operator regardless of the operator's height, so the visibility of the operator can be improved.

[0246] (4) One aspect of this embodiment is a sales processing device as described in any of (1) to (3), wherein the portion of the area displays warning information that alerts the operator or reception information that accepts operations from the operator.

[0247] According to the above configuration, it is possible to improve the visibility of information that is important to the operator.

[0248] (5) One aspect of this embodiment is a sales processing device as described in any of (1) to (4), wherein the display control means displays areas that are different from the certain area by graying out the areas so that the visibility is different.

[0249] According to the above configuration, by graying out the area, information of an area different from the part of the area is blocked, so that the visibility of the area of ​​interest can be improved.

[0250] (6) One aspect of this embodiment is a program for causing a computer to function as a sales processing device having a display means for displaying information regarding a transaction, the program causing the computer to function as a display control means for displaying a portion of a display area on the display means and an area other than the portion so as to have different visibility.

[0251] According to the above configuration, by displaying with different visibility, the operator can narrow down the area to which he or she should pay attention, thereby improving the operator's visibility.

[0252] (7) One aspect of this embodiment is a control method for a sales processing device equipped with a display means for displaying information regarding a transaction, the control method including a display control step for displaying a portion of a display area on the display means and an area other than the portion so as to have different visibility.

[0253] According to the above configuration, by displaying with different visibility, the operator can narrow down the area to which he or she should pay attention, thereby improving the operator's visibility.

[0254] In addition, a program for realizing the functions of the settlement device 40 in each of the above embodiments may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform the processing as the settlement device 40. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer system" may include multiple computer devices connected via a network including the Internet, a WAN, a LAN, a dedicated line, and other communication lines. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. In addition, the recording medium also includes a recording medium provided inside or outside the computer system that can be accessed from a distribution server in order to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format executable by a terminal device. That is, as long as it can be downloaded from a distribution server and installed in a form executable by a terminal device, the format in which it is stored in the distribution server does not matter. The program may be divided into multiple parts, downloaded at different times, and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a program that holds a program for a certain period of time, such as a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a network. The program may also be a program for realizing part of the above-mentioned functions. Furthermore, the program may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0255] [Summary of other embodiments] (1) As described above, one aspect of this embodiment is a product sales data processing system in which a customer uses a terminal device (e.g., customer terminal device 50) to register a product, the product sales data processing system including: an imaging means (e.g., imaging unit 506) that captures an image of the product to be registered; a product identification means that identifies the product to be registered based on the captured image obtained by the imaging means; and a storage means (step S203) that, when an unidentified result in which a product cannot be identified by the product identification means occurs, stores an captured image corresponding to the product first identified by the product identification means after the unidentified result occurs as an auxiliary image (e.g., a next captured image) corresponding to the unidentified result.

[0256] According to the above configuration, in response to the registration of a reserved product as an unspecified product, an image corresponding to the registration result in which the product is first identified after the unspecified product is registered can be acquired as a next captured image. By displaying the next captured image at a predetermined opportunity, the unspecified state of the product can be resolved efficiently (for example, the reservation of an unspecified reserved product can be released).

[0257] (2) One aspect of this embodiment is a product sales data processing system as described in (1), which displays a registration result information list (e.g., a registered product list area AR3 on a pending product processing screen) showing specific registration information (e.g., a product identification record) for the product identified by the product identification means and unspecified information (e.g., an unspecified record) showing the unspecified result as registration result information showing the result of a product registration made by a customer using a terminal device, and is equipped with a display control means for displaying, in response to a designation of unspecified information from the registration result information shown in the registration result information list, an image (e.g., a target image in a target image area AR43-1) when an unspecified result corresponding to the designated unspecified information occurs and an auxiliary image (e.g., a next image in a next image area AR43-2) corresponding to the unspecified result shown by the designated unspecified information.

[0258] According to the above configuration, it is possible to display a target captured image obtained by capturing an image of the status of the product registration operation corresponding to the unspecified product to be released from the hold, and a next captured image corresponding to the unspecified product. This makes it easier for a store clerk, for example, to determine whether or not a customer has registered the same product twice by looking at the target captured image and the next captured image.

[0259] (3) One aspect of this embodiment is the product sales data processing system according to (2), wherein the display control means displays the unspecified information in a contiguous order in the registration result information list.

[0260] According to the above configuration, it is possible to form a reserved item list area AR31 in which the list items of unspecified items are arranged in order from the top, for example, in the registered item list area AR3 of the reserved item processing screen. This makes it easier for the store clerk to release the reserved items from the unspecified items.

[0261] (4) One aspect of this embodiment is a product sales data processing system as described in (2), in which the unidentified information is displayed in the registration result information list in a chronological order relative to the specific registration information for the product that was first identified by the product identification means after the time when the unidentified result corresponding to the unidentified information occurred.

[0262] According to the above configuration, in the registered product list area AR3 of the reserved product processing screen, the list item of an unspecified product can be arranged in sequence with the list item of the product that was first specified after the product registration corresponding to the unspecified product was performed. This allows, for example, the list item of an unspecified product and the list item of a product that may have been registered in correspondence with the unspecified product to be arranged side by side in the registered product list area AR3, thereby improving the efficiency of the work of the store clerk in releasing the reserved unspecified product.

[0263] (5) One aspect of this embodiment is a product sales data processing system described in any one of (2) to (4), wherein the display control means displays, together with the auxiliary image, the registered quantity of the product corresponding to the auxiliary image (e.g., displays a quantity display section NM).

[0264] According to the above configuration, the quantity display section NM can be displayed in correspondence with the next captured image, which allows the store clerk to grasp the quantity of the unspecified product to be released and the normally registered product at the stage of performing the hold release operation, and allows the store clerk to perform the work and operation related to the hold release appropriately.

[0265] (6) One aspect of this embodiment is a product sales data processing system as described in any one of (2) to (5), comprising an editing means (e.g., product information input screen P40) for editing the unspecified information, and the display control means, in response to editing by the editing means to change the unspecified information to specific registration information, arranges the specific registration information changed from the unspecified information in the registration result information list in an order that conforms to a predetermined arrangement rule for registration result information.

[0266] According to the above configuration, in the registered product list area AR3 in which list items of reserved products are arranged in a consolidated order, list items corresponding to products that have been released from reservation can be rearranged in a prescribed arrangement order as list items of normally registered products.

[0267] (7) One aspect of this embodiment is a settlement device (40) in a product sales data processing system in which a customer uses a terminal device (e.g., a customer terminal device 50) to register products, the settlement device including: an information acquisition means for acquiring registration result information relating to the result of the product registration performed using the terminal device; and a display control means for displaying a registration result information list (e.g., a registered product list area AR3 on a pending product processing screen) showing specific registration information for a product identified in the product registration performed by the customer using the terminal device based on the registration result information acquired by the information acquisition means, and unspecified information showing an unspecified result in which the product was not identified, and, in response to the unspecified information being designated from the registration result information shown in the registration result information list, displaying the designated unspecified information (e.g., a target image in a target image area AR43-1) and information relating to the product that was first identified after the unspecified result corresponding to the designated unspecified information occurred (e.g., a next captured image in a next captured image area AR43-2).

[0268] (8) One aspect of this embodiment is a program for causing a computer serving as a settlement device in a product sales data processing system in which a customer uses a terminal device to register products to function as display control means for acquiring registration result information relating to the results of the product registration performed using the terminal device, displaying a registration result information list showing specific registration information for a product identified in the product registration performed by the customer using the terminal device based on the registration result information acquired by the information acquiring means, and unspecified information showing an unspecified result in which the product was not identified, and, in response to the unspecified information being designated from the registration result information shown in the registration result information list, displaying the designated unspecified information and information relating to the product that was first identified after the unspecified result corresponding to the designated unspecified information occurred. [Explanation of symbols]

[0269] 10 Management device 40 Payment device 50 Customer terminal equipment 60 Transaction Management Device 70 Credit card payment server 80 Transaction status management device AR502 Areas of Interest AR501 Darkened area AR501 auxiliary area

Claims

1. A sales processing device operated by a customer, comprising: A display means for displaying information regarding the transaction; a display control means for displaying, in a display area of ​​the display means, a specific area displaying information requiring a response by a store clerk and an auxiliary area displaying information supplementing the information displayed in the specific area with the same visibility, and displaying an area different from both the specific area and the auxiliary area with a visibility different from the specific area and the auxiliary area; Equipped with A sales processing device in which at least total amount information of the transaction is displayed in the auxiliary area.

2. A position setting means for setting display positions of the specific area and the auxiliary area in the display area, 2. The sales processing device according to claim 1, wherein the position setting means sets the display position of the specific area to a lower part of the display area, and sets the display position of the auxiliary area to an upper part of the display area.

3. a position setting means for setting a display position of the specific area in the display area, The sales processing device according to claim 1 , wherein the position setting means sets the display position of the specific area to a lower part or an upper part of the display area, or to a position according to a height of an operator.

4. A sales processing device as described in any one of claims 1 to 3, in which a list of products registered in the transaction is displayed in an area different from both the specific area and the auxiliary area.

5. A program for causing a computer to function as a sales processing device operated by a customer, the program comprising: The computer, a display control means for displaying, in a display area of ​​said display means, a specific area displaying information requiring a response by a store clerk and an auxiliary area displaying information supplementing the information displayed in said specific area with the same visibility, and displaying an area different from both said specific area and said auxiliary area with a visibility different from said specific area and said auxiliary area; A program that functions as a

6. A method for controlling a sales processing device operated by a customer, the method comprising: a display control step of displaying, in a display area of ​​the display means, a specific area displaying information requiring a response by a store clerk and an auxiliary area displaying information supplementing the information displayed in the specific area with the same visibility, and displaying an area different from both the specific area and the auxiliary area with a visibility different from the specific area and the auxiliary area; A control method comprising:

Images