Commodity sales data processing system and portable terminal device

Abstract

To provide a commodity sales data processing system allowing a customer to register a commodity by using a terminal device, in which the customer can properly perform a procedure for cancelling registration of the commodity.SOLUTION: A commodity sales data processing system allowing a customer to register a commodity by using a terminal device, includes: detection means for detecting cancellation of the commodity from a commodity input unit into which the commodity secured by the customer is input; and notification means for providing the customer with notification related to cancellation of registration of the registered commodity on the basis of the detection result by the detection means.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to a product sales data processing system and a mobile terminal device. [Background technology]

[0002] 2. Description of the Related Art A system is known in which a customer registers an item using a mobile terminal device (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-204551 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned product sales data processing system, a customer may register a certain product and reserve it by putting it in a shopping cart, but may wish to cancel the registration, for example, because they have selected the wrong product or have decided not to purchase it. In such a case, the customer can operate a mobile terminal to cancel the registration of the target product and return the cancelled product to its original display location, such as a display shelf. However, there are cases in which the customer is unable to properly carry out the procedure for canceling the registration of such a product. If the procedure for canceling the registration of a product is not carried out properly, the product registration result will not match the details of the product reserved by the customer, which will hinder smooth checkout.

[0005] The present invention has been made in consideration of the above circumstances, and has as its object to enable customers to properly carry out procedures for canceling product registration in a product sales data processing system in which the customers register products using a terminal device. [Means for solving the problem]

[0006] One aspect of the present invention for solving the above-mentioned problems is a product sales data processing system in which a customer registers a product using a mobile terminal device, the product sales data processing system comprising: a detection means for detecting the removal of a product from a product insertion section where the product reserved by the customer is inserted; and a notification means for notifying the customer regarding the cancellation of the registration of a registered product based on the detection result by the detection means.

[0007] Another aspect of the present invention is a portable terminal device used by a customer to register a product, the portable terminal device comprising: a detection means for detecting the removal of a product from a product insertion section where the product secured by the customer is inserted; and a notification means for notifying the customer of information regarding a registration cancellation operation for a registered product based on the detection result by the detection means. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a shopping system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of installation of a settlement device in this embodiment. [Diagram 3] 1 is a diagram showing an example of the appearance of a settlement device according to an embodiment of the present invention; [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a settlement device in the present embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of the configuration of a customer terminal device in the present embodiment. [Figure 6] 1 is a diagram illustrating an example of the configuration of a transaction management device in this embodiment. [Figure 7] FIG. 1 is a diagram illustrating an example of the configuration of a transaction status management device according to an embodiment of the present invention. [Figure 8] 3A to 3C are diagrams illustrating an example of customer information, store information, and cart information according to the present embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a definition of a reserved product in the present embodiment. [Figure 10]A sequence diagram showing an example of processing procedures executed by a customer terminal device, a settlement device, and a transaction management device in this embodiment in response to the accounting of one transaction in self-service operation. [Figure 11] FIG. 2 is a diagram showing an example of a screen of a shopping application in the present embodiment. [Figure 12] 13A to 13C are diagrams showing an example of a registered product presentation screen and a screen displayed on a settlement device in this embodiment. [Figure 13] 13A and 13B are diagrams illustrating an example of a product registration change screen and a cancellation confirmation screen according to the present embodiment. [Figure 14] 13A and 13B are diagrams illustrating a specific example of a situation in which a non-allocated re-guidance screen is displayed in the embodiment. [Figure 15] 11A and 11B are diagrams illustrating a specific example of a situation in which a registration cancellation operation guide screen is displayed in the embodiment. [Figure 16] 11 is a flowchart showing an example of a processing procedure executed by the customer terminal device and the transaction management device in this embodiment in relation to canceling the registration of a product. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <Embodiment> [Example of shopping system configuration] Fig. 1 shows an example of the configuration of a shopping system (an example of a product sales data processing system) according to this embodiment. The shopping system in Fig. 1 corresponds to accounting (customer terminal-compatible accounting) in which, for example, a customer takes out a product to be purchased from a display shelf, registers the taken-out product using a customer terminal device 50, and the same customer also uses a settlement device 40 to settle the payment for the registered product. The accounting here refers to a procedure carried out in response to one transaction, including the registration of the product and the settlement of the registered product. The shopping system of this embodiment includes a store controller 10, a transaction management device 60, a settlement device 40, a customer terminal device 50, a transaction status management device 80, and a credit card settlement server 70.

[0010] The store controller 10, the settlement device 40, and the transaction status management device 80 are installed in a store and are communicatively connected via a LAN 19 (which may be wired or wireless). The store controller 10 is a business terminal used by store staff for store management and the like, and is capable of communicating with the transaction management device 60.

[0011] The transaction status management device 80 is a business terminal that manages (including monitoring the transaction status) the status of transactions performed by customers using the customer terminal device 50 in the store, and is capable of communicating with the transaction management device 60. Although four settlement devices 40 are illustrated in FIG. 1, the number of settlement devices 40 within one store is not particularly limited.

[0012] The customer terminal device 50 is operated by customers (such as shoppers who are members of the store, hereinafter also referred to simply as customers). In addition to typical communication and imaging functions (camera), the customer terminal device 50 has a recognition function that scans barcodes attached to products to read product codes, that is, recognizes barcodes attached to products. Note that the recognition function of the customer terminal device 50 only needs to be able to read product codes, and does not need to be able to recognize which product the read product code belongs to. In other words, the customer terminal device 50 only needs to be able to read a product code from a barcode that exists as an object in an image captured by the imaging function (an image acquired as a through image).

[0013] Such a customer terminal device 50 may be a smartphone, a tablet terminal, or the like owned by a customer. A shopping application having a function of registering products in accordance with the shopping system of this embodiment is installed in the smartphone, tablet terminal, or the like serving as the customer terminal device 50.

[0014] Furthermore, the customer terminal device 50 may be, for example, a terminal loaned from a store to a customer. An application having a function of registering products in accordance with the shopping system of this embodiment is also installed in the customer terminal device 50 loaned from the store to a customer. The customer terminal device 50 loaned from the store to a customer may be one that is held by the customer or may be one that is attached to a shopping cart and used by the customer.

[0015] The customer terminal device 50 also captures an image (e.g., releases the shutter) of the product (e.g., the peripheral portion with a barcode attached) and generates a captured image (image data). The customer terminal device 50 may capture an image by releasing the shutter in response to an operation by the customer who is the operator, but in this embodiment, it is preferable that the customer terminal device 50 captures an image by releasing the shutter at its own discretion.

[0016] In addition, the customer terminal device 50 may be equipped with an image recognition technology that extracts feature points from an image (through image, captured image) and recognizes an imaged subject (object, etc.). For example, the customer terminal device 50 may use the image recognition technology to identify (estimate) an imaged product.

[0017] The settlement device 40 is a settlement device that can at least accept cash as a settlement method. The settlement device 40 is configured to be able to communicate with the transaction management device 60. The settlement device 40 may be configured to be able to communicate with the transaction management device 60, for example, via the store controller 10.

[0018] The transaction management device 60 manages transactions carried out in accordance with the shopping system of this embodiment.

[0019] The credit card payment server 70 executes payment processing corresponding to the credit card in response to a credit card payment request transmitted from the settlement device 40, the customer terminal device 50, etc.

[0020] [About the payment device] Fig. 2 is a diagram showing an example of the installation of the settlement device 40. Fig. 2(A) is a perspective view of the settlement device 40 etc. as seen from the customer side. Fig. 2(B) is a perspective view of the settlement device 40 etc. as seen from the store clerk side. As shown in Fig. 2(A), a counter is placed on the right side of the settlement device 40 as seen from the customer side.

[0021] Fig. 3 is a diagram showing an example of the appearance of the settlement device 40. Fig. 3(A) is a perspective view of the settlement device 40 as seen from the customer side. Fig. 3(B) is a perspective view of the settlement device 40 as seen from the store clerk side. Fig. 4 is a diagram showing an example of the configuration of the settlement device 40. In Fig. 3 and Fig. 4, the same parts are given the same reference numerals.

[0022] An example of the configuration of settlement device 40 shown in Fig. 4 will be described below with reference to Fig. 3. Settlement device 40 includes a CPU 401, a ROM 402, a RAM 403, a hard disk 404, a customer display unit 405, a customer scanner unit 406, a card payment unit 408, a change machine 409, a store clerk display unit 410, a key operation unit 411, a store clerk scanner unit 412, a printing unit 413, an audio output unit 414, and a communication unit 415. These can communicate with each other via a bus.

[0023] The CPU 401 is a central processing unit, and controls the operation of the settlement device 40 by reading and executing programs stored in the ROM 402 . The ROM 402 is a read-only memory, and stores various types of information used by the CPU 401, including programs.

[0024] The RAM 403 is a read / write memory and stores various information, such as information read from the ROM 402 or the hard disk 404, information acquired from the outside, and information generated during processing.

[0025] The hard disk 404 stores various information. For example, the hard disk 404 may store programs executed by the CPU 401 instead of the ROM 402. Also, instead of the RAM 403, the hard disk 404 may store information read from the ROM 402, information acquired from the outside, information generated during processing, and the like.

[0026] The customer-side display unit 405 is a touch display for customers, which displays various information to customers and accepts various inputs from customers. Customer-side scanner unit 406 is a scanner unit for customers, and for example, scans a code (for example, a code other than a barcode, such as a barcode or a two-dimensional code) attached to a product and reads the product code. Customer-side scanner unit 406 may also scan a code (barcode, two-dimensional code, etc.) printed on a bill (registered trademark) and read information required for settlement. Customer-side scanner unit 406 may also scan a code (two-dimensional code, barcode, etc.) displayed on the display unit of customer terminal device 50 and read information required for settlement.

[0027] Although the customer-side scanner unit 406 is used when a customer registers a product, the customer may register the product by other methods. For example, if a preset key (a button for ordering the product) corresponding to the product is displayed on the customer-side display unit 405, the customer may operate (press) the preset key to register the product.

[0028] The card payment unit 408 is a payment mechanism that uses various cards (credit cards, prepaid cards such as transportation cards, etc.). The card payment unit 408 of this embodiment includes a card recognition unit (reading unit), a display unit, and an operation unit, but it is sufficient that the card payment unit 408 includes at least the card recognition unit. The card recognition unit may also recognize various cards (e.g., membership cards, point cards, etc.) that are not directly used for payment (settlement).

[0029] The change machine 409 (cash settlement unit) is a cash settlement mechanism, which has an input port for banknotes and coins, and an output port for banknotes and coins, calculates the amount of money inserted into the input port, calculates the change amount, which is the difference between the input amount and the purchase amount, and outputs the change from the output port. The change machine 409 faces the customer and is operated by the customer. When banknotes or coins are inserted into the input port, they are detected by a sensor (detecting that they have been inserted, detecting the number of coins by denomination, etc.).

[0030] That is, the change machine 409 is used when the price of registered products is settled with cash (money) in the settlement device 40. The change machine 409 has a banknote insertion port for inserting banknotes, a coin insertion port for inserting coins, a banknote discharge port for discharging banknotes, a coin discharge port for discharging coins, a counting unit for counting the inserted or discharged money, a money transport mechanism between the insertion port or discharge port and the storage unit, the above-mentioned sensor, etc. The banknote insertion port and the coin insertion port are also referred to as deposit insertion ports. The banknote discharge port and the coin discharge port are also referred to as change discharge ports. The banknote insertion port and the banknote discharge port may be a common port, and the coin insertion port and the coin discharge port may be a common port.

[0031] The change machine 409 also counts the coins replenished during the closing process and stores them in a storage unit. The change machine 409 also counts the coins to be dispensed during the closing process and discharges them from a change discharge port. The closing process is a process of adjusting the amount of money stored in the change machine 409 (cash on hand / cash on hand) to a reference amount after the store is closed or before the store is opened.

[0032] The store clerk's display unit 410 is a touch display for the store clerk, which displays various information to the store clerk and receives various inputs from the store clerk. The key operation unit 411 is made up of various keys (buttons) and receives various inputs from the store clerk. The store clerk scanner unit 412 is a scanner unit for store clerks, and for example, scans a barcode attached to a product and reads the product code. The store clerk scanner unit 412 also scans a barcode attached to the name tag of the store clerk and reads the store clerk code.

[0033] Although the store clerk's scanner unit 412 is used when the store clerk registers a product, the store clerk may register the product by other methods. For example, if a key corresponding to a product (e.g., a key corresponding to a sports newspaper) is arranged on the key operation unit 411, the store clerk may operate (press) that key to register the product. Also, if a preset key corresponding to a product is displayed on the store clerk's display unit 410, the store clerk may operate that preset key to register the product.

[0034] The printing unit 413 prints and issues various media (receipts, bills, etc.). The printing unit 413 can freely rotate to change its orientation (direction of the media issuing port) from the clerk side to the customer side and from the customer side to the clerk side. The orientation of the printing unit 413 may be changed manually, or may be changed automatically (mechanically controlled, etc.) in response to, for example, a transition (switching) of the operation mode described below. The correctness of the orientation of the printing unit 413 may be detected by a sensor or the like.

[0035] The audio output unit 414 outputs audio. For example, the audio output unit 414 outputs audio guidance and the like. The communication unit 415 communicates with other devices (other settlement devices 40 and the store controller 10).

[0036] In addition, the settlement device 40 of this embodiment can be switched between a total of four operating modes: a customer terminal-responsive settlement mode, which is set to perform settlement processing under customer terminal-responsive accounting, and three accounting modes (first to third accounting modes) corresponding to three different patterns of accounting (product registration, settlement). The first payment mode is an operating mode in which the store clerk performs operations on the payment device 40 corresponding to product registration and operations corresponding to payment. The second payment mode is an operating mode in which the store clerk performs operations on the payment device 40 corresponding to product registration, and the customer performs operations on the same payment device 40 corresponding to payment. The third payment mode is an operating mode in which the customer performs operations on the payment device 40 corresponding to product registration and operations corresponding to payment. As a result, in a store that has introduced the shopping system of this embodiment, some of the multiple payment devices 40 can be set to customer terminal compatible payment mode to support customer terminal compatible payment, and the other payment devices 40 can be set to, for example, the first or second payment mode so that a store clerk can handle payments in consultation with customers. Note that the other payment devices 40 may also include those set to the third payment mode.

[0037] [Example of customer terminal device configuration] 5 shows an example of the configuration of the customer terminal device 50. The customer terminal device 50 in the figure includes a CPU 501, a storage unit 502, a RAM 503, a display unit 504, an operation unit 505, an imaging unit 506, and a network-compatible communication unit 507. The CPU 501 is a central processing unit, and controls the operation of the customer terminal device 50 by reading and executing programs stored in the storage unit 502 . The storage unit 502 is an auxiliary storage device for the CPU 501, and stores various types of information used by the CPU 501, including programs. The RAM 503 is a main storage device for the CPU 501 . The display unit 504 displays an image according to the display control of the CPU 501. The display unit 504 may be configured as a touch panel that allows operations to be performed on the screen. The operation unit 505 collectively indicates various operators provided on the customer terminal device 50 and various input devices connected to the customer terminal device 50 . The imaging unit 506 captures images under the control of the CPU 401. Furthermore, the imaging unit 506 of this embodiment can also function as a part that reads two-dimensional codes in a code reader that processes the two-dimensional codes. The network-compatible communication unit 507 is a unit that wirelessly communicates via a network. The network-compatible communication unit 507 enables the customer terminal device 50 to communicate with the transaction management device 60.

[0038] [Example of transaction management device configuration] 6 shows an example of the configuration of transaction management device 60. Transaction management device 60 in the figure comprises a CPU 601, a storage unit 602, a RAM 603, and a network-compatible communication unit 604. The CPU 601 is a central processing unit, and controls the operation of the transaction management device 60 by reading and executing programs stored in the memory unit 602 . The storage unit 602 is an auxiliary storage device for the CPU 601, and stores various types of information used by the CPU 501, including programs. The RAM 603 is a main storage device for the CPU 601 . The network compatible communication unit 604 is a part that performs communication via a network.

[0039] The memory unit 602 of the transaction management device 60 stores customer information, store information, product master (product information), and cart information.

[0040] [Example of transaction status management device configuration] 7 shows an example of the configuration of a transaction status management device 80. The transaction status management device 80 in the figure includes a CPU 801, a storage unit 802, a RAM 803, an in-store compatible communication unit 804, a network compatible communication unit 805, a display unit 806, and an operation unit 807. The CPU 801 is a central processing unit, and controls the operation of the transaction status management device 80 by reading and executing programs stored in the storage unit 802 . The storage unit 602 is an auxiliary storage device for the CPU 801, and stores various types of information used by the CPU 801, including programs. The RAM 803 is the main storage device of the CPU 801 . The in-store communication unit 804 communicates with terminals within the store, such as the settlement device 40 and the store controller 10, via the LAN 19. The network-compatible communication unit 805 is a part that communicates with the outside via a network. In the case of the transaction status management device 80, the network-compatible communication unit 805 can communicate with the transaction management device 60 on the cloud. The display unit 806 displays images under the control of the CPU 801. The display unit 806 may be configured as a touch panel. The operation unit 807 collectively indicates the operators provided in the transaction status management device 80 and the input devices connected to the transaction status management device 80 .

[0041] [Information stored in the transaction management device] The customer information stored in the memory unit 602 is information for managing individual customers. The transaction management device 60 generates customer information when a customer is registered (a customer's registration may be considered to have been completed when the customer information of that customer is stored). The transaction management device 60 also updates the customer information as appropriate based on cart information, etc. The transaction management device 60 may update the customer information, for example, by referring to the cart information at a specified time every day.

[0042] 8(A) shows an example of customer information stored in the memory unit 602 of the transaction management device 60. The customer information in this figure includes customer identification information, customer name, customer registration date, cancellation information, customer rank, number of points, etc. The customer identification information is identification information that uniquely identifies a customer. The customer identification information may be application identification information that is uniquely assigned to each installed shopping application when the shopping application of this embodiment is installed. The application identification information may be, for example, a number that is uniquely (for example, sequentially) assigned to each installed shopping application. Alternatively, the application identification information may be terminal identification information that is unique to the customer terminal device 50 in which the shopping application is installed. The terminal identification information may be, for example, a telephone number assigned to a customer who is the user of the customer terminal device 50, or hardware identification information that is uniquely assigned to the hardware of the customer terminal device 50, such as a MAC (Media Access Control) address. The customer name is the customer's name or nickname, etc. The customer registration date is the date and time when the customer was registered. Cancellation information is information regarding the cancellation of registered products after registration. The customer rank is a rank according to the customer's purchasing history. When generating customer information for a new customer, the customer identification information, customer name, and customer registration date are generated, but since this is before the start of actual trading (product registration), other information (cancellation information, etc.) is not generated.

[0043] The transaction management device 60 generates and stores customer identification information, for example, when registering a customer (for example, when the customer terminal device 50 downloads or installs a shopping application to use a shopping service provided by the transaction management device 60 from an external device (for example, a specified server that provides all applications, the transaction management device 60 itself)). In addition, for example, when registering a customer, the transaction management device 60 acquires information entered into the name field of a registration form (input form) using the customer terminal device 50, and stores it as the customer name. In addition, the transaction management device 60 acquires, for example, the current date and time when registering the customer, and stores it as the customer registration date.

[0044] In addition, instead of or in addition to storing the customer information in its own memory unit, the transaction management device 60 may store some or all of the customer information in another device (such as a file server accessible to the transaction management device 60).

[0045] 8(B) shows an example of store information stored in the memory unit 602 of the transaction management device 60. The store information in the figure includes store identification information, store name (branch name), store identification information 1, and store identification information 2. The store identification information is identification information that uniquely identifies a store. The store identification information shown in Fig. 8(B) is composed of a store (trade name) or company code and a branch code. The store name is the name of the store. The store name shown in Fig. 8(B) is composed of the store (trade name) or company and the branch name. The store identification information 1 is information on a two-dimensional code (QR code (registered trademark) or the like) for identifying a store where a transaction takes place (a store where a product is bought and sold). The store identification information 2 is store location information (GPS information) for identifying the store where the transaction is to be carried out. In the example shown in FIG. 8(B), the store identification information and the store specifying information 1 are different, but the store identification information and the store specifying information 1 may be the same.

[0046] The transaction management device 60 may obtain and store store information from an external source (such as a server (not shown) at a headquarters that manages each store). The transaction management device 60 may store some or all of the store information in another device (such as a file server accessible by the transaction management device 60) instead of or in addition to storing the store information in its own memory.

[0047] Cart information is information for managing individual transactions. The transaction management device 60 generates the cart information at the start of a transaction. In addition, the transaction management device 60 updates the cart information as the transaction progresses (each time a product is registered) (the storage of a product in the cart information may be interpreted as the registration of that product).

[0048] FIG. 8(C) is an example of cart information stored in the memory unit 602 of the transaction management device 60. The cart information in the figure includes cart identification information, transaction start date and time, transaction end date and time, customer identification information, product information in the cart (registered product information, reserved product information), cancellation information, and the like.

[0049] The cart identification information is identification information that uniquely identifies the cart information. For example, the cart identification information includes store identification information, a date, and a serial number (for example, a serial number for each store and each date). The transaction start date and time is the date and time when the transaction starts. The transaction start date and time may be, for example, the date and time when the cart information is generated. The transaction start date and time may be the date and time when the first item of product is registered (the date and time when the registered product information (registered product 1) in FIG. 8(C) is stored). The date and time when the cart information is generated and the date and time when the first item of product is registered may both be stored separately.

[0050] The transaction end date and time is the end date and time of the transaction. For example, the transaction end date and time may be the settlement date and time. The customer identification information is customer identification information that identifies the customer of the transaction. When the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are generated, but since this is before the start of the actual transaction (product registration), other information (transaction end date and time, etc.) is not generated. The settlement date and time may be the settlement start date and time or the settlement end date and time. Both the settlement start date and time and the settlement end date and time may be stored separately as the transaction end date and time.

[0051] Registered product information (total) is information that is updated each time a product is registered. Registered product information (total) includes the number of products (number of products), estimated subtotal amount (e.g., the subtotal amount before discounts, etc. are reflected), subtotal amount, etc. Registered product information (registered product 1) is the registration information for one product item. Registered product information (registered product 2) is the registration information for two products. In the example shown in Figure 8 (C), registered product information (registered product 3) to registered product information (registered product 5) are omitted from the illustration. Registered product information (N; N is an integer) includes the product code, product name (product name), price, etc.

[0052] The registered product information (N) may include the registration date and time of the Nth item of product. In other words, the transaction management device 60 may store the registration date and time of the registered product as the registered product information. The registration date and time of each product may also be used as a basis for determining whether or not a service such as a time service should be applied and the effect of the application.

[0053] The reserved product information (total) is updated each time a reserved product is registered. A reserved product is a product that was not registered properly due to a certain reason and is registered in a temporarily reserved state. The reserved product information (total) includes the number of reserved products (number of products), the number of non-file reserved products, the number of unreadable reserved products (requiring confirmation of unauthorized operation), and the like.

[0054] NON-FILE refers to a product whose barcode or product code was successfully scanned in the store (the product code was read), but the product code is not stored as product information in the product master (a product whose product information is not registered).

[0055] A read NG refers to a failure to scan the product code at the store (the product code could not be read), or a product whose product code has not been scanned at the store. In other words, a read NG refers to a timeout process when it is possible to determine that a product has been imaged for a certain period of time using image recognition technology but has not been able to be recognized as a barcode, or a product that has been timed out. For example, if the barcode cannot be correctly acquired (recognized) despite the customer performing an operation to have the barcode read due to wrinkles in the package, smudges or dirt on the barcode print, or the barcode being printed on a transparent film, it will be determined that the barcode cannot be read.

[0056] Reserved Product Information (Reserved Product 1) is information on reserved product 1. Reserved Product Information (Reserved Product 2) is information on reserved product 2. Reserved Product Information (Reserved Product 3) is information on reserved product 3.

[0057] The reserved product information (reserved product N; N is an integer) includes the reserved product type (information indicating whether the reserved product is NON-FILE or cannot be read) and image data (image data captured when reading was NG). For example, if the Nth item is a reserved product due to NON-FILE, the reserved product information (reserved product N) includes the reserved product type "1 (NON-FILE)", product code, and image data. Also, if the Nth item is a reserved product due to reading being NG, the reserved product information (reserved product N) includes the reserved product type "2 (cannot be read)" and image data.

[0058] 9 shows an example of reserved product definition corresponding to cart information in this embodiment. In the figure, an example is shown in which reserved products are classified into eight categories. Classification numbers "1" to "8" are assigned to each of the eight reserved product classifications. The classification of the reserved product with classification number 1 is “NON-FILE” as explained in FIG. The classification of the reserved product with classification number 2 is “reading not allowed” as explained in FIG.

[0059] The classification of the reserved product with classification number 3 is "pseudo scan input." A reserved product with "pseudo scan input" is a product for which the customer has attempted to have the customer terminal device 50 read the product's code, but the customer terminal device 50 has not been able to read the code properly.

[0060] As a specific example, an example of a "pseudo scan input" corresponds to a situation in which the product is brought into the imaging range of the imaging unit 506, but the product code is not brought into the imaging range of the imaging unit 506. Additionally, a "pseudo scan insertion" situation corresponds to a situation in which the product is brought into the imaging range of the imaging unit 506, but an image of a part that does not have a code is captured. Additionally, "pseudo scan insertion" refers to a situation in which a product is brought into the imaging range of imaging unit 506 with the code portion hidden by a finger or the like. In addition, a "pseudo scan input" situation occurs when the code on a product is captured by the imaging unit 506, but the product is passed through the imaging range of the imaging unit 506 at a speed that prevents the customer terminal device 50 from reading the code.

[0061] The classification of the reserved product with classification number 4 is “non-scan-inserted.” A reserved product with the “non-scan-inserted” classification is a product that was added to the shopping cart without performing an operation to position the product within the imaging range of imaging unit 506. This non-scanning insertion can occur, for example, when the customer terminal device 50 is fixedly attached to a shopping cart and is configured to be able to capture an image of the inside of a shopping basket placed in the shopping cart. For example, when the customer terminal device 50 determines based on the captured image that a product has been placed in the shopping cart even though the product's code has not been scanned, it can determine that a non-scanning insertion has occurred.

[0062] The classification of reserved items with classification number 5 is "age verification items." "Age verification items" are items such as alcohol and tobacco that require a store clerk to verify the customer's age before selling them.

[0063] The classification of reserved goods with classification number 6 is "pharmaceuticals." "Pharmaceuticals" are goods that require a qualified person, such as a pharmacist, to confirm with the purchaser.

[0064] The classification of reserved items with classification number 7 is "anti-theft tag items." "Anti-theft tag items" are items that have anti-theft tags attached and require a store clerk to remove the anti-theft tag from the item when it is sold.

[0065] The classification of the reserved product with classification number 8 is a "cancelled product." A "cancelled product" is a product that was once registered but was cancelled by a customer's cancellation operation on the customer terminal device 50. A "cancelled product" is taken from the customer by a store clerk at the time of payment, for example.

[0066] In this embodiment, a reserved product that falls into any of the four classifications of "NON-FILE", "Read NG", "Pseudo-scan input", and "Non-scan input" of classification numbers 1 to 4 among classification numbers 1 to 7 is an unspecified product. An unspecified product is, for example, a product whose registration result is incomplete because it has not been specified which of the products registered in the product master it is.

[0067] The product master is stored, for example, by the store controller 10. The product master stores product information for each product sold in the store. Product information for one product includes a number of predetermined information items, including a product code and a product name. The transaction management device 60 stores the product master data acquired from the store controller 10. The transaction management device 60 uses the stored product master data for processes related to product registration, etc.

[0068] For the reserved item corresponding to the classification number 1 "NON-FILE", the item code is acquired by code scanning in response to the item registration operation, but the item information corresponding to the acquired item code is not registered in the item master. Therefore, it is not specified which of the items registered in the item master the reserved item corresponding to the classification number 1 "NON-FILE" is. Specifically, the customer terminal device 50 may transmit the item code acquired by code scanning to the transaction management device 60, and the transaction management device 60 may compare the transmitted item code with the item code of the item registered in the item master stored in the transaction management device 60. If it is determined as a result of the comparison that the same item code as the transmitted item code does not exist in the item master, the transaction management device 60 may treat the item as a reserved item corresponding to the classification number 1 "NON-FILE" and notify the customer terminal device 50 of the fact. For pending products that fall into one of the three categories of classification numbers 2 to 4, "Read NG", "Pseudo scan input", and "Non-scan input", the product code has failed to be acquired as a result of the code scan in response to the product registration operation. Therefore, it is not possible to identify which of the products registered in the product master the pending products that fall into one of the three categories of classification numbers 2 to 4, "Read NG", "Pseudo scan input", and "Non-scan input", are. Such unspecified products can be released from their reserved status (reserved status) by a store clerk operating the transaction status management device 80 as described below, and can be registered normally.

[0069] In addition, instead of or in addition to storing the cart information in its own memory, the transaction management device 60 may store some or all of the cart information in another device (such as a file server accessible by the transaction management device 60).

[0070] [Example of basic processing procedure for self-operation] The sequence diagram in Fig. 10 shows an example of a processing procedure executed by the customer terminal device 50, the settlement device 40, and the transaction management device 60 in response to the accounting of one transaction in a self-service operation. Specifically, the processing in Fig. 10 shows an example of the processing of the customer terminal device 50, the settlement device 40, and the transaction management device 60 from when a customer registers a product to be purchased from among products displayed in the store, until the settlement of the registered product is completed.

[0071] Step S1: The customer terminal device 50 acquires information for identifying a store (store identification information). For example, a two-dimensional code for identifying the store may be displayed near the entrance of the store (the two-dimensional code may be output on a display screen, a medium on which the two-dimensional code is printed may be attached, etc.), and a customer who visits the store may scan (read) the two-dimensional code with the customer terminal device 50, thereby allowing the customer terminal device 50 to acquire the store identification information. When a customer who visits the store starts a shopping application, a screen instructing the customer to scan the two-dimensional code may be displayed as a start-up screen, or when a customer who visits the store scans the two-dimensional code with the customer terminal device 50, the shopping application may be started, and a screen informing the customer that the transaction management device 60 is currently connected may be displayed as a start-up screen.

[0072] Also, for example, since a store is identified by its location, a customer who visits the store may obtain location information (GPS information) at the store using the customer terminal device 50 (i.e., the store's location information may be obtained as store identification information). When a customer who visits the store starts a shopping application, location information may be obtained and a screen may be displayed as a start-up screen informing the customer that the application is currently connected to the transaction management device 60. If multiple stores are detected from the location information and one cannot be identified, a selection screen may be displayed to allow the customer to select. Alternatively, a mode may be switched to forcefully obtain a two-dimensional code.

[0073] 11(A) shows an example of a start-up screen of a shopping application displayed on the display unit 504 of the customer terminal device 50. The start-up screen in the figure displays the store name. This store name is displayed based on the store-specific information acquired in step S1. When a customer wishes to start registering a product for shopping, the customer operates a transaction start button BT10 arranged on the start-up screen. In response to the operation of the transaction start instruction button BT10, the customer terminal device 50 transmits a transaction start request including the acquired store specific information and customer identification information to the transaction management device 60. The transaction start request may be transmitted in response to the customer's operation on the customer terminal device 50 as described above, or may be transmitted in response to the acquisition of store specific information without receiving any particular operation. Regarding the customer identification information, the information entered in the name field of the registration form using the customer terminal device 50 during customer registration (when downloading or installing a shopping application on the customer terminal device 50) may be stored in the customer information of the transaction management device 60, but may also be stored in the memory of the customer terminal device 50 in addition to the transaction management device 60. When a store is identified (when the product registration initial screen described below is acquired), the store name, the services being offered (flyer information distributed on that day), and available coupon information may be displayed on a screen (the product registration initial screen or a screen other than the product registration initial screen). The service and coupon information may be acquired from the transaction management device 60 as screen information, for example.

[0074] In addition, the destination information (address of the transaction management device 60) may also be obtained when registering the customer (when downloading or installing the shopping application to the customer terminal device 50) and stored in the memory of the customer terminal device 50. In addition to the store-specific information, the destination information may also be converted into a two-dimensional code, and by scanning the two-dimensional code with the customer terminal device 50, the customer terminal device 50 may obtain the destination information along with the store-specific information.

[0075] Step S2: The transaction management device 60, which has received the customer identification information and store identification information as a transaction start request from the customer terminal device 50, generates cart information for the transaction. Note that in the initial state in which the cart information is generated, the cart identification information, transaction start date and time, and customer identification information are stored, but since this is before the start of the actual transaction (product registration), other information (transaction end date and time, product information (registered product information, reserved product information), etc.) is not stored.

[0076] As described above, the transaction management device 60 stores store information such as that shown in Figure 8 (B), so when it receives store identification information (it also receives customer identification information) as a transaction start request from the customer terminal device 50, if the received store identification information is a two-dimensional code, it acquires the store identification information by referring to store identification information 1, and if the received store identification information is location information (GPS information), it acquires the store identification information by referring to store identification information 2. Note that if the store identification information received from the customer terminal device 50 is store identification information converted into a two-dimensional code, the transaction management device 60 need only acquire it as is.

[0077] In other words, when the transaction management device 60 receives customer identification information and store identification information from the customer terminal device 50 as a transaction start request, it acquires the store identification information from the store identification information received from the customer terminal device 50, and further acquires the current date, issues (assigns) a serial number, combines the store identification information with the current date and serial number, and stores the combined information as cart identification information in the cart information. Also, when the transaction management device 60 receives store identification information and customer identification information from the customer terminal device 50 as a transaction start request, it acquires the current date and time, and stores the current date and time as the transaction start date and time (generation date and time) in the cart information. Also, when the transaction management device 60 receives store identification information and customer identification information from the customer terminal device 50 as a transaction start request, it stores the customer identification information received from the customer terminal device 50 as customer identification information in the cart information.

[0078] Step S3: The transaction management device 60, which has generated the cart information for the transaction, generates product registration initial screen information (screen information of the product registration screen, which is the initial screen), and transmits it to the customer terminal device 50. Specifically, the transaction management device 60, for example, generates product registration initial screen information, and transmits the generated product registration initial screen information to the customer terminal device 50 together with the cart identification information.

[0079] Step S4: The customer terminal device 50, which has received the cart identification information and the product registration initial screen information from the transaction management device 60, stores the cart identification information and displays the registration screen on the display unit. Specifically, the customer terminal device 50 displays the product registration initial screen as shown in FIG. 11(B), for example.

[0080] The product registration initial screen is a product registration screen that is displayed first when product registration is started. The product registration screen has a captured image area AR1 and a product registration result area AR2. The captured image area AR1 is an area in which a captured image obtained by imaging by the imaging unit 506 is displayed. A product code (e.g., a barcode) captured so as to fit within the range of the captured image displayed in the captured image area AR1 is the target for reading. The product registration result area AR2 shows the number of registered products (registered points) and the total price for the products registered so far, and is an area where list items corresponding to each registered product are arranged. In the case of the product registration initial screen of the same figure, since no products have been registered yet, the product registration result area AR2 is blank with no list items arranged.

[0081] Step S5: The customer terminal device 50 scans the barcode attached to the product by the customer's operation, and reads the product code. In the figure, it is assumed that the barcode has been successfully scanned. Steps S5 to S9 are repeatedly executed every time the barcode attached to the product is scanned.

[0082] The customer terminal device 50 that has acquired the barcode transmits to the transaction management device 60 the cart identification information and product registration information including the product code obtained by scanning.

[0083] Step S6: The transaction management device 60 receives the product registration information (cart identification information and product code) from the customer terminal device 50 and identifies the cart information of the transaction from the cart identification information included in the received product registration information.

[0084] Step S7: The transaction management device 60 updates the product data in the identified cart information. Specifically, if the received product registration information contains a product code for the Nth item, the transaction management device 60 stores the product code as the product code of the registered product information (registered product N) in the identified cart information, obtains the product name and price corresponding to the product code from the product information, and stores them as the product and price of the registered product information (registered product N). The transaction management device 60 also updates the registered product information (total) in the identified cart information.

[0085] Step S8: After updating the product data in the cart information, the transaction management device 60 generates product registration update screen information (screen information of the product registration screen, which is an update screen to which the registered product has been added) and transmits it to the customer terminal device 50. Specifically, the transaction management device 60 generates product registration update screen information such that the product registration update screen, to which the list item Itm of the currently registered product has been added, is displayed in the product registration result area AR2 on the customer terminal device 50, as shown in Fig. 11(C), and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information.

[0086] The product registration screen (product registration update screen) shown in FIG. 11(C) is displayed on the customer terminal device 50 after "XX bread" is registered as the third product item. In other words, when "○○ yogurt" is stored in the cart information as the first item, the transaction management device 60 generates product registration update screen information such that "○○ yogurt" is displayed as the list item Itm on the customer terminal device 50 and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information; when "○○ chocolate" is stored in the cart as the second item, the transaction management device 60 generates product registration update screen information such that "○○ yogurt" and "○○ chocolate" are displayed as list items Itm on the customer terminal device 50 and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information; and when "○○ bread" is stored in the cart information as the third item, the transaction management device 60 generates product registration update screen information such that "○○ yogurt", "○○ chocolate", and "○○ bread" are displayed as list items Itm on the customer terminal device 50 as shown in FIG. 11(C), and transmits the generated product registration update screen information to the customer terminal device 50 together with the cart identification information.

[0087] In response to the registration of one product, a registered product display screen WD1 may be displayed to display the currently registered product, as shown in Fig. 12(A), for example. In addition, in the registered product display screen WD1 shown in the same figure, in response to the target product being available for eating in, a "Yes" button BT1 and a "No" button BT2 are displayed to confirm with the customer whether or not they would like to eat in. In this case, the price of the product is calculated so that the consumption tax rate for the target product is different when the "Yes" button BT1 is operated and when the "No" button BT2 is operated.

[0088] Step S9: The customer terminal device 50, which has received the cart identification information and product registration update screen information from the transaction management device 60, adds the product to the registration screen. Specifically, the customer terminal device 50 displays a product registration update screen such as that shown in Fig. 11(C). As described above, the product registration screen (product registration update screen) shown in Fig. 11(C) is displayed on the customer terminal device 50 after "XX bread" is registered as the third product item.

[0089] Step S10: The customer terminal device 50 accepts a payment instruction operation as an operation by the customer, for example, a touch of the "Proceed to Checkout" button shown in FIG.

[0090] Step S11: The customer terminal device 50 that has accepted the settlement transition instruction operation generates a two-dimensional code. That is, the customer terminal device 50 converts information required for executing the settlement process for the items purchased by the customer terminal device 50 (e.g., cart identification information) into a two-dimensional code. The customer terminal device 50 that has generated the two-dimensional code displays the generated two-dimensional code on the display unit. For example, a code display screen with a two-dimensional code arranged as shown in FIG. 11(D) is displayed on the display unit.

[0091] Step S12: The settlement device 40 scans (reads) the two-dimensional code displayed on the display unit of the customer terminal device 50. For example, the settlement device 40 scans the two-dimensional code displayed on the display unit of the customer terminal device 50 that is directed by the customer within the recognition range of the customer-side scanner unit 406.

[0092] Step S13: The settlement device 40, which has read the two-dimensional code displayed on the display unit of the customer terminal device 50, requests the calculation of the subtotal amount from the transaction management device 60. For example, the settlement device 40 transmits a calculation request (subtotal calculation request information) requesting the calculation of the subtotal amount to the transaction management device 60 together with the cart identification information obtained from the two-dimensional code.

[0093] Step S14: The transaction management device 60, which has received the cart identification information and the sub-calculation output request information from the customer terminal device 50, identifies the cart information of the transaction from the cart identification information.

[0094] Step S15: After identifying the cart, the transaction management device 60 calculates the subtotal amount using information such as the price in the registered product information included in the cart information corresponding to the identified cart information.

[0095] Step S16: After calculating the subtotal, the transaction management device 60 updates the cart information (storing the subtotal (calculated subtotal)), and transmits the subtotal information indicating the calculated subtotal to the settlement device 40 together with the cart identification information.

[0096] Step S17: The settlement device 40, which has received the cart identification information and the subtotal information from the transaction management device 60, displays the subtotal amount on the customer side display unit 405, for example, as shown in FIG. 12(B).

[0097] Step S18: The settlement device 40, which has displayed the subtotal on the customer display unit 405, executes the payment (settlement). Specifically, the settlement device 40 accepts the selection of the payment type. In the case of cash, it accepts the deposit, calculates the amount of change, and, if there is change, dispenses the change and issues a receipt. In addition, when the settlement is completed, the settlement device 40 sends settlement completion information together with the cart information to the transaction management device 60, and the transaction management device 60 stores the transaction end date and time (settlement date and time) for that cart.

[0098] After scanning a product, the customer terminal device 50 transmits the product code to the transaction management device 60 (S5). However, the customer terminal device 50 may be configured not to transmit product codes for products (e.g., products scanned after moving to another store) other than those scanned in the store (the store when the customer visited the store and displayed the product registration initial screen). For example, the customer terminal device 50 acquires and stores location information (GPS information) when the customer visits the store (or when the product registration initial screen is displayed). The customer terminal device 50 also acquires location information when scanning each product, and compares the location information acquired when the customer visits the store (or when the product registration initial screen is displayed). If the two match (or approximately match), the customer terminal device 50 may permit the transmission of the product code of the product to the transaction management device 60, and if they do not match (or approximately match), the customer terminal device 50 may prohibit the transmission of the product code of the product to the transaction management device 60. This makes it possible to prevent inappropriate product registration (for example, registering a product to a cart created in another store, etc.).

[0099] When the settlement device 40 prohibits the transmission of the product code as described above, after scanning the product, the settlement device 40 may display an error message (e.g., "Registration is not possible because it is not in the XX store") on the customer side display unit 405. Furthermore, the settlement device 40 may display the above message on the clerk side display unit 410 instead of or in addition to the message on the customer side display unit 405.

[0100] The settlement device 40 reads the two-dimensional code displayed on the display of the customer terminal device 50 and then requests the transaction management device 60 to calculate the subtotal amount (S13), but may not request calculation of the subtotal amount for products other than the product scanned in the store (for example, products scanned in other stores). For example, the settlement device 40 refers to the store identification information of the store (it may store and refer to the store identification information of the store in its own settlement device 40, or may refer to the store identification information stored in another accessible device), and when it reads the two-dimensional code displayed on the display of the customer terminal device 50, it compares the cart identification information obtained from the two-dimensional code with the store identification information of the store. The settlement device 40 may permit the request to calculate the subtotal amount if the store identification information included in the cart identification information includes the store identification information of the store, and may prohibit the request to calculate the subtotal amount if the store identification information does not include the store identification information of the store. This makes it possible to prevent inappropriate payments (for example, payment for products registered at other stores, etc.).

[0101] When the settlement device 40 prohibits the request for the subtotal amount as described above, after reading the two-dimensional code, the settlement device 40 may display an error message (e.g., "Settlement cannot be completed because the purchase includes products from a store other than XX store") on the customer-side display unit 405. The settlement device 40 may also display the above message on the clerk-side display unit 410 instead of or in addition to the message on the customer-side display unit 405.

[0102] In the example of the processing procedure in Fig. 10, the transaction management device 60 calculates the subtotal amount. However, for example, the settlement device 40 may acquire cart information from the transaction management device 60 and calculate the subtotal amount based on the acquired cart information. Alternatively, the transaction management device 60 may transmit cart information to the store controller 10 and request calculation of the subtotal amount, the store controller 10 may calculate the subtotal amount using the received cart information, and the calculated subtotal amount may be transmitted to the settlement device 40 via the transaction management device 60.

[0103] During the settlement process, the transaction management device 60 may check the cancellation status of the customer's products based on the customer information and cart information of the customer, and determine whether or not to issue a warning against fraud. If it is determined that a warning should be issued, the transaction management device 60 may, for example, cause the settlement device 40 to issue a warning notification.

[0104] Furthermore, during checkout processing, the transaction management device 60 may determine whether or not the product information included in the cart information includes any reserved products, and if so, may request the checkout device 40 to provide instructions to correct the reserved products. Upon receiving a request to provide instructions to correct the reserved products, the checkout device 40 displays a message to notify the user that there are reserved products, as shown in Fig. 12(C). Upon confirming the message, the store clerk may perform an operation to release (correct) the reserved products, such as entering a price corresponding to the reserved products or canceling the reserved products.

[0105] [Example of procedure for canceling registration of registered products] There are cases where a customer has once registered a product, but wishes to cancel the registration, for example because the customer has registered the wrong product or has decided not to purchase. In the shopping system of this embodiment, therefore, while the customer terminal device 50 is in the process of registering a product, the customer can cancel the registration of the registered product by operating the customer terminal device 50. An example of the procedure a customer follows to cancel the registration of a product will be described below. In the following description, the customer terminal device 50 is taken as an example to be attached to a shopping cart so as to capture an image of the inside of the shopping cart. In this case, an image captured of the inside of the shopping cart is displayed in the captured image area AR1 of the product registration screen.

[0106] When the customer terminal device 50 is in the process of registering a product and it becomes necessary to cancel a certain product among the products already registered, the customer performs an operation to select the list item Itm of the product to be cancelled from among the list items Itm in the product registration result area AR2 of the product registration screen. The operation of selecting the list item Itm may be an operation of tapping the list item Itm. In response to the operation of selecting the list item Itm, a product registration change screen is displayed.

[0107] The product registration change screen is a screen on which the user can change the registered details (change the quantity or cancel the registration) of the product corresponding to the selected list item Itm. 13A shows an example of a product registration change screen P30. The product registration change screen P30 in the figure shows an example in the form of a pop-up screen that is displayed superimposed on the product registration screen. The product registration change screen P30 in the figure is displayed in response to selecting the list item Itm for the product "XX Chocolate" from among the three list items Itm in the product registration result area AR2 in the figure. In this case, in the product registration result area AR2, the two list items Itm for products other than "XX Chocolate" are grayed out to indicate that they are not available for selection, and the list item Itm for "XX Chocolate" is not grayed out to indicate that it is available for selection.

[0108] The product registration change screen P30 is arranged with a cancel button BT31, a quantity / price area AR30, a plus button BT32-1, and a minus button BT32-2. The cancel button BT31 is a button that is operated to cancel the registration of the target product. When the cancel button is operated, if the registered quantity of the target product is two or more, the entire registered quantity is cancelled. The quantity / price area AR30 indicates the registered quantity of the target product corresponding to the current time and the price according to the registered quantity. The plus button BT32-1 is a button that is operated when making a change to increase the registered quantity of a target product. Each time the plus button BT32-1 is operated, the registered quantity is changed to increase by one. The minus button BT32-2 is a button that is operated when making a change to decrease the registered quantity of the target product. Each time the minus button BT32-2 is operated, the registered quantity is changed to decrease by one. In response to a change in the registered quantity by operating the plus button BT32-1 or the minus button BT32-2, the registered quantity and price in the quantity-price area AR30 are changed.

[0109] In this embodiment, decreasing the registered quantity in response to the operation of the minus button BT32-2 may be treated as canceling the product. In other words, when the registered quantity of a registered product is decreased, the product corresponding to the decreased quantity may be treated as being canceled.

[0110] When the product registration change screen P30 is displayed, for example, if a list item Itm corresponding to a target product is operated, the product registration change screen P30 is cleared and the product registration screen is returned to its displayed state.

[0111] When canceling the entire quantity of the target product, the customer operates the cancel button BT31. When the cancel button BT31 is operated, the product registration change screen P30 is cleared and a cancellation confirmation screen is displayed successively.

[0112] 13B shows an example of a cancellation confirmation screen WD40. The cancellation confirmation screen WD40 in the drawing shows an example in the form of a pop-up screen that is displayed superimposed on the product registration screen. When a customer is performing an operation to cancel a target product, the target product is in a state of being reserved by the customer, for example, by being placed in the customer's shopping cart. The cancellation confirmation screen WD40 is a screen that notifies the customer that the registration cancellation has been successfully completed in response to the operation to cancel the product. The cancellation confirmation screen WD40 in the figure also guides the customer to return the target product to the place where the target product was displayed, such as a display shelf (to return it to an unreserved state). The cancellation confirmation screen WD40 displays a message notifying the user that the product has been cancelled and instructing the user to return the product to an unsecured state, and also has a confirmation button BT40 arranged thereon.

[0113] In response to the cancellation of the registration, the list item Itm of the cancelled product in the product registration result area AR2 indicates that the product has been cancelled by, for example, erasing the quantity and striking through the product name and price, as shown in the figure. Such cancelled products are managed as "cancelled products" that are reserved products with classification number 8 in the cart information. The strikethrough through the product name and price of the list item Itm also indicates that the corresponding product is a reserved product classified as a "cancelled product". Also, in the figure, although it is hidden by the cancellation confirmation screen WD40, the display of the number of registered items and the total amount in the product registration result area AR2 has been changed to reflect the cancellation of the product registration.

[0114] In response to the cancellation of registration, the list item Itm of the targeted product may be deleted.

[0115] If the customer views the cancellation confirmation screen WD40 and has not yet returned the product whose registration has been cancelled to an unsecured state, the customer may, for example, return the product to a display shelf to return it to an unsecured state, and then operate the cancel button BT31. When the cancel button BT31 is operated, the cancellation confirmation screen WD40 is cleared, and the display returns to the product registration screen. Even when the display returns to the product registration screen, the list item Itm of the cancelled product still shows that it has been cancelled.

[0116] [Notification of non-reserved status when cancelling the registration of a registered product] As described above, when a customer wishes to cancel the registration of a product that has already been registered, the customer must display the cancellation confirmation screen WD40, perform the registration cancellation operation (operate the cancel button BT31), and return the previously reserved product to its unreserved state, for example by returning it to the shelf. However, there are cases where a customer performs a registration cancellation operation but forgets to return the target product to a non-reserved state. In this case, the number of products actually in the shopping cart may be greater than the number of products registered in the cart information, resulting in a discrepancy between the breakdown of registered products and the breakdown of products actually reserved by the customer. Alternatively, a malicious customer may perform a cancellation operation but intentionally leave the item in the shopping cart without returning it to an unsecured state, and then go ahead and check out, attempting to complete the purchase without paying for the item.

[0117] Therefore, the customer terminal device 50 determines whether the target product has been returned to a non-secured state in response to the registration cancellation operation, and if it is determined that the product has not been returned to a non-secured state, it provides guidance to the customer to have the target product returned to a non-secured state.

[0118] When determining whether the target product has been returned to an unsecured state, the customer terminal device 50 is configured to detect the removal of a product from the shopping cart by using a captured image obtained by capturing an image of the inside of the shopping cart. For such detection, for example, the customer terminal device 50 is configured to be capable of recognizing an object as a product from the captured image. In addition, the customer terminal device 50 may be configured to detect that a product has been removed from the shopping cart when any of the objects as products that existed in the captured image transitions to a state where they are not recognized in the captured image. At this time, the customer terminal device 50 may be configured to recognize a human hand from the captured image, and may use the recognition result of the human hand to detect the removal of a product from the shopping cart. For example, the customer terminal device 50 may detect that a product has been removed from the shopping cart when, after a human hand is recognized in the captured image, a product recognized in the captured image transitions to a state where the human hand is no longer recognized and the product is no longer recognized in the captured image. In this case, the mobile terminal 300 may execute a process of identifying the type of product for each product object. Such detection of the removal of an item from a shopping basket may be achieved by a learning model that uses images of a person removing an item from a basket as learning data. In addition, for example, the customer terminal device 50 may transmit an image to the transaction management device 60, and the transaction management device 60 may use the transmitted image to detect the removal of an item from the shopping cart through the above-mentioned processing.

[0119] The customer terminal device 50 is then configured to detect whether or not a product has been removed from the shopping cart during a determination period, for example, from the time the cancellation confirmation screen WD40 is erased until a certain period of time has elapsed. If a removal of an item from the shopping cart is detected within a certain period of time after the cancellation confirmation screen WD40 is deleted, the customer terminal device 50 determines that the target item has been returned to an unsecured state in response to the current registration cancellation operation. If no removal of an item from the shopping cart is detected within a certain period of time from when the cancellation confirmation screen WD40 is deleted, the customer terminal device 50 determines that the target item has not been returned to an unsecured state in response to the current registration cancellation operation.

[0120] It should be noted that the customer may reset the target product to an unsecured state while the product registration change screen P30 is being displayed, or before the product registration change screen is displayed. Therefore, the customer terminal device 50 may be configured to determine whether or not a product has been detected being removed from the shopping cart during a determination period that begins when the product registration change screen P30 begins to be displayed and ends when a certain amount of time has elapsed since the cancellation confirmation screen WD40 was erased. Alternatively, the customer terminal device 50 may be configured to determine whether or not removal of a product from the shopping cart is detected during a determination period that begins a predetermined time prior to the start of displaying the product registration change screen P30 and ends a certain time period that has elapsed since the cancellation confirmation screen WD40 was erased. In this case, if the customer terminal device 50 stores captured images taken up to a predetermined time prior to the time the cancellation confirmation screen WD40 was erased, the captured images taken up to a predetermined time prior to the time the cancellation confirmation screen WD40 was erased can be used for detection.

[0121] Then, when the customer terminal device 50 determines that the target product has not been returned to a non-secured state in response to the registration cancellation operation, it displays a non-secured re-guidance screen. The non-secured re-guidance screen is a screen that again guides the customer to return the product corresponding to the registration cancellation operation to a non-secured state.

[0122] A specific example of a situation in which a non-reserved re-guidance screen is displayed will be described with reference to Fig. 14. Fig. 14(A) to Fig. 14(D) show an example of screen transition on the display unit 504 of the customer terminal device 50 in response to an action taken by a customer in relation to the cancellation of the registration of a registered product. As shown in the product registration result area AR2 of the product registration screen in Fig. 14(A), when the customer has completed the registration of three products, "XX Potatoes," "XX Tea," and "XX Chocolate," he operates the list item Itm for "XX Chocolate" to display the product registration change screen P30 in order to cancel the registration of the product "XX Chocolate." The customer operates the cancel button BT31 on the displayed product registration change screen P30 to cancel the registration of the product "XX Chocolate."

[0123] In response to the operation of the cancel button BT31, the customer terminal device 50 erased the product registration change screen P30 and displayed a cancel confirmation screen WD40 as shown in Fig. 14(B). The customer operated the confirmation button BT40 arranged on the cancel confirmation screen WD40.

[0124] In response to the operation of the Confirm button BT40, the Cancellation Confirmation screen WD40 is cleared, and the product registration screen is displayed as shown in Fig. 14(C). In the product registration result area AR2 of the product registration screen in the same figure, in the list item Itm of the product "XX Chocolate", the quantity display is cleared, and the product name and price are struck through to indicate that the registration has been cancelled.

[0125] Also, the captured image area AR1 of the product registration screen in the figure shows that the product "XX Chocolate" that was the subject of the cancellation has not been removed from the shopping cart and remains in it. In other words, at this stage, the customer still has the product "XX Chocolate" that was the subject of the cancellation reserved. For example, the customer in this case performed the operation to cancel the registration of the product "XX Chocolate", but forgot to return the product "XX Chocolate" to the display shelf and return it to an unreserved state.

[0126] The customer left the "XX Chocolate" product in the shopping cart, forgetting to return it to an unsecured state even after a certain amount of time had passed since the cancellation confirmation screen WD40 was deleted. Therefore, the customer terminal device 50 determined that the target product had not been returned to an unsecured state in response to the registration cancellation operation, when a certain amount of time had passed since the cancellation confirmation screen WD40 was deleted.

[0127] Therefore, the customer terminal device 50 displays a non-reserved re-guidance screen WD50 as shown in Fig. 14(D). The non-reserved re-guidance screen WD50 in the figure shows an example in the form of a pop-up screen that is displayed superimposed on the product registration screen. The non-reserved re-guidance screen WD50 in the figure displays a message for the customer to confirm that although the registration of a product has been cancelled, the product has not yet been returned to a non-reserved state, and displays a confirmation button BT50.

[0128] The customer realizes by looking at the displayed non-reserved re-notification screen WD50 that he / she performed the registration cancellation operation but did not return the target product to a non-reserved state. The customer then returns the "XX Chocolate" product to the display shelf to make it non-reserved, and operates the Confirm button BT50. It does not matter which comes first: returning the "XX Chocolate" product to the display shelf to make it non-reserved, or operating the Confirm button BT50. In response to the operation of the Confirm button BT50, the non-reserved re-notification screen WD50 is erased, and the product registration screen is displayed.

[0129] In this manner, in this embodiment, if the target product has not been returned to the non-secured state even after a certain time has elapsed since the cancellation confirmation screen WD40 was erased, the non-secured re-notification screen WD50 is displayed. This allows the customer to realize that the target product has not yet been returned to the non-secured state, and to return the target product to, for example, a display shelf. In other words, the customer can properly carry out the procedure for canceling the registration of the product. In addition, even if a malicious customer performs a registration cancellation operation but intentionally leaves the target item in the shopping cart without returning it to an unsecured state, the unsecured re-guidance screen WD50 is displayed, increasing the likelihood that the customer will return the target item to an unsecured state, thereby deterring fraudulent behavior.

[0130] If a certain period of time has passed since the non-reserved re-guidance screen WD50 was erased without the target product being returned to a non-reserved state, the customer terminal device 50 may display the non-reserved re-guidance screen WD50 again. This makes it possible to more effectively prevent and deter fraudulent behavior, such as a customer forgetting to return the target product to a non-reserved state or a customer not returning a product that has been cancelled from registration to a non-reserved state.

[0131] [Registration cancellation instructions for canceling registered products] Conversely, there are also cases where a customer who wishes to cancel the registration of a registered product returns the product to the shelf or other such state, but forgets to cancel the registration. In such cases, the number of products registered in the cart information may be greater than the number of products actually in the shopping cart, resulting in a discrepancy between the breakdown of products actually reserved by the customer and the breakdown of products registered.

[0132] Therefore, when the customer terminal device 50 determines that a product that was once reserved by a customer has been returned to an unreserved state, it determines whether or not a registration cancellation operation has been performed to cancel the registration of the product that has been returned to an unreserved state, and if it determines that a registration cancellation operation has not been performed, it provides guidance to the customer to perform a registration cancellation operation for the product in question.

[0133] In this case, the customer terminal device 50 is configured to constantly detect removal of an item from the shopping cart using an image obtained by capturing an image of the inside of the shopping cart during item registration. The customer terminal device 50 then determines whether or not a registration cancellation operation has been performed during a predetermined determination period including the time when it was detected that the item had been removed. Specifically, the customer terminal device 50 may determine whether or not a registration cancellation operation has been performed during a certain period of time since it was detected that the item had been removed. The customer terminal device 50 may also include a certain period of time from the time when it was detected that the item had been removed in the period for determining whether or not a registration cancellation operation has been performed.

[0134] When the customer terminal device 50 determines that the registration cancellation operation has not been performed during the above-mentioned determination period, the customer terminal device 50 displays a registration cancellation operation guide screen. The registration cancellation operation guide screen is a screen that guides the customer to perform a registration cancellation operation corresponding to the product that has been set to the non-secured state.

[0135] A specific example of a situation in which the registration cancellation operation guide screen is displayed will be described with reference to Fig. 15. Fig. 15(A) to Fig. 15(C) show an example of screen transition on the display unit 504 of the customer terminal device 50 in response to an action taken by a customer in relation to the cancellation of the registration of a registered product.

[0136] The product registration screen in Fig. 15(A) corresponds to the stage where three products, "XX Potatoes", "XX Tea", and "XX Chocolate", have been registered. In this case, the captured image area AR1 displays a captured image of the three products, "XX Potatoes", "XX Tea", and "XX Chocolate", in a shopping cart. The product registration result area AR2 displays list items Itm corresponding to each of the products, "XX Potatoes", "XX Tea", and "XX Chocolate".

[0137] In this state, the customer decides to cancel the registration of the "XX Chocolate" product. Therefore, in this case, the customer first removes the "XX Chocolate" product from the shopping cart and returns it to an unsecured state (for example, by returning it to the display shelf). In this way, when the customer removes the "XX Chocolate" product from the shopping cart and returns it to an unsecured state, the image displayed in the captured image area AR1 changes to a state where the "XX Chocolate" product is not present, as shown in Figure 15(B).

[0138] Furthermore, the customer terminal device 50 detects that a product in the shopping cart has been removed based on the change in the captured image from a state in which the "XX Chocolate" product was present to a state in which it is no longer present, as described above. When the customer terminal device 50 detects the removal of a product in this manner, it sets a determination period and determines whether or not a registration cancellation operation has been performed between the start and end of the determination period. During the determination period, the customer terminal device 50 may be configured not to accept new product registrations. In this case, the customer put the "XX Chocolate" product back on the shelf, but forgot to perform the corresponding registration cancellation operation. Therefore, a situation occurred in which a certain amount of time had passed since the customer terminal device 50 detected the removal of the product, and the judgment period had ended without the registration cancellation operation being performed. In this case, the customer terminal device 50 determines that the registration cancellation operation was not performed within the judgment period.

[0139] In response to determining that the registration cancellation operation has not been performed within the determination period, the customer terminal device 50 displays a registration cancellation operation guide screen. Fig. 15(C) shows an example of the registration cancellation operation guide screen WD60. The registration cancellation operation guide screen WD60 in the figure shows an example in the form of a pop-up screen that is displayed superimposed on the product registration screen. On the registration cancellation operation guidance screen WD60, a message informing the user of the registration cancellation operation corresponding to the product that has just been returned to the non-reserved state, and a confirmation button BT60 are arranged.

[0140] The customer realizes by looking at the displayed registration cancellation operation guide screen WD60 that although he has returned the product to be cancelled to an unsecured state, he has not performed the corresponding registration cancellation operation. The customer then operates the confirmation button BT60 to close the registration cancellation operation guide screen WD60 and display the product registration screen. The customer operates the displayed product registration screen to perform the registration cancellation operation for the "XX Chocolate" product, and cancels the registration of the "XX Chocolate" product.

[0141] In this manner, in this embodiment, if it is determined that the customer has not performed the registration cancellation operation even after a certain period of time has elapsed since the customer returned the product to the non-reserved state, the registration cancellation operation guide screen WD60 is displayed. This allows the customer to realize that the registration cancellation operation has not yet been performed, and to perform the registration cancellation operation. In other words, even in this case, the customer can properly perform the procedure for canceling the registration of the product.

[0142] The customer terminal device 50 may set a judgment period until a certain time has elapsed in response to the cancellation of registration operation screen WD60 being erased once, and if the judgment period has elapsed without the cancellation of registration operation being performed, may display the cancellation of registration operation screen WD60 again.

[0143] [Example of processing procedure] 16 is a flowchart showing an example of a processing procedure executed by the customer terminal device 50 and the transaction management device 60 in relation to canceling the registration of a product. Step S101: While the product registration screen is displayed on the display unit 504, the customer terminal device 50 determines whether or not a product code has been read (a product code has been acquired) by code scanning in response to a product registration operation.

[0144] Step S102: In response to reading the product code, customer terminal device 50 transmits product registration information to transaction management device 60. The product registration information includes the product code read in step S101 and cart identification information.

[0145] Step S103: The customer terminal device 50 receives the registration completion notification transmitted from the transaction management device 60 in response to the transmission of the product registration information in step S102. The customer terminal device 50 uses the information on the product registration update screen contained in the received registration completion notification to update the product registration screen, for example, as illustrated in FIG. 11(C) and the like.

[0146] Step S104: After the process of step S103, or if it is determined in step S101 that the product code has not been read, the customer terminal device 50 determines whether or not a registration cancellation operation has been performed. The registration cancellation operation here is a series of operations in which the list item Itm of the product to be cancelled on the product registration screen is tapped to display the product registration change screen P30 (FIG. 13(A)), and the cancel button BT31 on the displayed product registration change screen P30 is tapped.

[0147] Step S105: If it is determined that a registration cancellation operation has been performed, the customer terminal device 50 transmits registration cancellation information to the transaction management device 60. The registration cancellation information is information that notifies the transaction management device 60 that a registration cancellation operation has been performed at the customer terminal device 50. The registration cancellation information includes a product code indicating the product designated as the target for registration cancellation and cart identification information.

[0148] Step S106: The customer terminal device 50 receives the cancellation completion notification transmitted from the transaction management device 60 in response to the transmission of the product registration information in step S105. The customer terminal device 50 uses the information on the product registration update screen contained in the received cancellation completion notification to update the product registration screen so that it reflects that the target product has been canceled in the product registration result area AR2, for example, as illustrated in Fig. 12(B). Step S107: The customer terminal device 50 also displays a cancellation confirmation screen WD40 as exemplified in FIG. 13(B) and the like. Step S108: In response to an operation on the confirmation button BT40 on the cancellation confirmation screen WD40, the customer terminal device 50 erases the cancellation confirmation screen WD40. As a result, the product registration screen in which the cancellation of the target product is reflected in the product registration result area AR2 by the update in step S106 is displayed as the top screen.

[0149] Step S109: When the cancellation confirmation screen is cleared, the customer terminal device 50 sets a determination period, and determines whether or not removal of a product from the shopping cart has been detected based on the captured images during the set determination period. If removal of the product is detected, steps S110 to S112 described below are skipped and the process proceeds to step S113. In this case, the non-reserved re-guidance screen WD50 is not displayed.

[0150] Step S110: If it is determined in step S109 that removal of a product has not been detected, the customer terminal device 50 determines whether or not the determination period set in response to the detection of removal of a product in step S109 has elapsed (ended). If it is determined that the determination period has not elapsed, the process returns to step S109. When returning the process from step S110 to step S109 in this manner, the process may be configured such that reading of a product code for new product registration and registration cancellation operations for other registered products cannot be accepted until the cancellation confirmation screen is erased and the determination period has elapsed.

[0151] Step S111: If it is determined in step S110 that the determination period has elapsed without the removal of the product being detected in step S109, the customer terminal device 50 displays the non-reserved re-guidance screen WD50 (FIG. 14(D)). Step S112: In response to an operation on the confirmation button BT50 on the non-reserved re-guidance screen WD50, the customer terminal device 50 erases the non-reserved re-guidance screen WD50 and displays the product registration screen as the top screen.

[0152] Step S113: If it is determined in step S104 that the registration cancellation operation has not been performed, or if the removal of a product within the determination period is detected in step S109, or after the processing of step S112, the customer terminal device 50 determines whether or not the removal of a product from the shopping cart has been detected based on the captured image while in a state in which product registration can be accepted. The determination in step S109 above of whether removal has been detected corresponds to a determination of whether the target product has been returned to a non-secured state in response to a registration cancellation operation being performed. In contrast, the determination in step S113 of whether removal has been detected is a process for determining whether the target product has been returned to a non-secured state prior to the registration cancellation operation when canceling the registration of the product.

[0153] Step S114: If removal of a product from the shopping cart is detected in step S113, the customer terminal device 50 sets a determination period, and determines whether or not a registration cancellation operation has been performed during the set determination period. Step S115: If a registration cancellation operation is not detected in step S114, the customer terminal device 50 judges whether or not the determination period set in step S114 has elapsed. If the determination period has not elapsed, the process returns to step S114. When the process returns from step S115 to step S114 in this manner, reading of a product code for new product registration and registration cancellation operations for other registered products may not be accepted until the determination period has elapsed.

[0154] Step S116: If it is determined in step S114 that a registration cancellation operation has been performed during the determination period, registration cancellation information is sent to the transaction management device 60. The registration cancellation information includes a product code indicating the product designated as the target for registration cancellation and cart identification information. Step S117: The customer terminal device 50 receives the cancellation completion notification transmitted from the transaction management device 60 in response to the transmission of the product registration information in step S116. The customer terminal device 50 uses the information on the product registration update screen contained in the received cancellation completion notification to update the product registration screen in the product registration result area AR2 to reflect that the target product has been canceled. In this case, since the target product has already been returned to the unsecured state, the customer terminal device 50 does not need to execute the process related to displaying the cancellation confirmation screen WD40 as in steps S107 and S108 above.

[0155] Step S118: If the determination period has elapsed without the registration cancellation operation being performed in step S115, the customer terminal device 50 displays the registration cancellation operation guide screen WD60 (FIG. 15(C)). Step S119: In response to an operation on the confirmation button BT60 on the registration cancellation operation guide screen WD60, the customer terminal device 50 erases the registration cancellation operation guide screen WD60 and displays the product registration screen as the top screen.

[0156] Step S120: If removal of the product is not detected in step S113, or after the processing of step S117 or S119, the customer terminal device 50 judges whether or not a payment instruction operation has been performed while the product registration screen is displayed. If a payment instruction operation has not been performed, the process returns to step S101.

[0157] Step S121: When a payment transition instruction operation is performed, the customer terminal device 50 converts the cart identification information into a two-dimensional code, and displays on the display unit 504 a code display screen (FIG. 11(D)) on which the generated two-dimensional code is displayed.

[0158] Next, an example of a processing procedure executed by the transaction management device 60 will be described. Step S201: The transaction management device 60 determines whether or not the product registration information transmitted from the customer terminal device 50 in step S102 has been received.

[0159] Step S202: When the transaction management device 60 receives the product registration information, it identifies the cart information indicated by the cart identification information included in the received product registration information. Step S203: The transaction management device 60 updates the cart information identified in step S202 by registering a new product. That is, the transaction management device 60 adds product information based on the product code included in the product registration information received in response to step S201 to the identified cart information. This registers the product in the cart information. Step S204: When the transaction management device 60 completes the process of step S203, it generates a product registration update screen that reflects the results of updating the cart information in step S203 (addition of the newly registered product). The transaction management device 60 transmits a registration completion notice including the generated product registration update screen to the customer terminal device 50.

[0160] Step S205: After processing of step S204, or if it is determined in step S201 that product registration information has not been received, the transaction management device 60 determines whether registration cancellation information sent from the customer terminal device 50 by step S105 or step S116 has been received.

[0161] Step S206: When the transaction management device 60 receives the registration cancellation information, it identifies the cart information indicated by the cart identification information included in the received registration cancellation information. Step S207: The transaction management device 60 updates the cart information identified in step S206 so that the product indicated by the product code included in the received registration cancellation information is managed as a canceled product (a pending product classified as "cancelled product" under classification number 8). Step S208: When the transaction management device 60 completes the process of step S207, it generates a product registration update screen that reflects the update results of the cart information in response to the cancellation of the product registration in step S207. The transaction management device 60 transmits a cancellation completion notice including the generated product registration update screen to the customer terminal device 50.

[0162] <Modification> Modifications of the present embodiment will be described below. The modifications described below may be combined with the above embodiment and other modifications as appropriate.

[0163] [First Modification] Examples of ways to "return a product that is the subject of deregistration to an unsecured state" include returning the product to its original display location, such as the display shelf where it was displayed, as well as handing the product over to the nearest store clerk or taking the product to a predetermined storage location for cancelled products in the store.

[0164] [Second modified example] In the above embodiment, the detection of removal of an item from the shopping basket is performed using a captured image. However, the detection of removal of an item from the shopping basket may be performed, for example, by providing a weighing scale capable of measuring the weight of the shopping basket placed on the shopping cart, and detecting removal of an item from the shopping basket based on the change in weight measured by the weighing scale. Alternatively, instead of a shopping cart, a weighing scale may be provided on the bottom surface of the shopping basket. Furthermore, removal of an item from the shopping basket may be detected based on the detection output of an optical sensor, a vibration sensor, or the like.

[0165] [Third Modification] In the above embodiment, an example has been given in which the customer terminal device 50 is attached to a shopping cart so as to be able to capture an image of the inside of the shopping cart and executes processing corresponding to product registration or the like. However, for example, there may be cases where a customer removes the customer terminal device 50 from the shopping cart and holds it in his / her hand to read the product code of the product to be registered. In this case, there is a possibility that a registration cancellation operation will be performed on the product that was registered by reading the product code while the customer is holding the customer terminal device 50 in his / her hand. As described above, if a customer holds the customer terminal device 50 in his / her hand and registers a product, and then immediately performs a registration cancellation operation, the customer is likely to return the target product to the display shelf without placing it in the shopping cart, and thus to make the product unsecured. Therefore, the customer terminal device 50 does not need to display the unsecured re-guidance screen WD50 if a registration cancellation operation is performed within a predetermined time (e.g., about several tens of seconds) after the customer holds the customer terminal device 50 in his / her hand and registers the product. That is, in this case, the customer side display unit 405 does not need to determine whether or not a product has been removed from the shopping cart within the determination period. In this modified example, it is necessary to determine whether or not the customer is holding the customer terminal device 50. This determination may be made, for example, by the customer terminal device 50 based on the detection output of a gyro sensor provided therein.

[0166] [Fourth Variation] In addition, when the customer returns the target product to an unsecured state before performing the registration cancellation procedure, for example, if the item of the target product detected as being removed from the shopping cart can be identified based on the captured image, a process of canceling the registration of the identified product may be performed. The process of canceling the registration of the product according to this modified example may be performed, for example, by the customer terminal device 50 and the transaction management device 60 in cooperation with each other, without involving any operation of the customer on the customer terminal device 50. Specifically, for example, the transaction management device 60 may identify the item of the product removed from the shopping cart based on the captured image sent from the customer terminal device 50, and manage the product as a pending product classified as a "cancelled product." In the configuration of this modified example, if the target product is returned to a non-secured state before the registration cancellation operation is performed, the registration of the target product can be properly cancelled without performing the registration cancellation operation after the product is returned to the non-secured state. In this case, for example, the registration cancellation operation guide screen WD60 does not need to be displayed.

[0167] In addition, if the customer returns the target product to an unsecured state before performing the registration cancellation procedure, and the item name of the target product detected as being removed from the shopping cart cannot be identified, the registration cancellation operation can be made unnecessary and the registration cancellation operation guide screen WD60 can be prevented from being displayed as follows. That is, in this case, in response to detection of removal of an item from the shopping cart, at least one of the currently registered items may be treated as a reserved item. Alternatively, a virtual (dummy) reserved item may be added to the cart information. Since the reserved item is thus included in the cart information, a store clerk will be present to release the reserved item before the checkout. When the store clerk attends, he or she will notice that the contents of the cart information do not match the breakdown of items in the shopping cart, and will be able to appropriately correct the cart information and proceed with the checkout.

[0168] In addition, if, as a procedure for canceling registration, the customer returns the target product to a non-reserved state before performing the registration cancellation operation, in response to detection of the product being removed from the shopping cart, the list item Itm of the reserved product may be added to the product registration result area AR2 on the product registration screen. In addition, if the registration cancellation operation is not performed within a certain period of time after the removal of a product from the shopping cart is detected, for example, a registration cancellation operation guide screen WD60 may be displayed. Alternatively, instead of displaying the registration cancellation operation guidance screen WD60, a message, icon, etc., guiding the user to perform the registration cancellation operation may be displayed in the list item Itm of the added pending item, for example, immediately after the pending item is added to the list item Itm, or when a certain amount of time has passed since the removal of the item from the shopping cart is detected and no registration cancellation operation has been performed.

[0169] [Fifth Modification] In the above embodiment, a product whose registration has been cancelled by a cancellation operation is managed as a "cancelled product" that is a reserved product with classification number 8. Also, on the product registration screen, the list item Itm of a product whose registration has been cancelled by a cancellation operation has a strikethrough drawn through the product name and price, thereby indicating that the corresponding product is a reserved product classified as a "cancelled product". When a product treated as a reserved product is included in the cart information, for example, a store clerk performs an operation to release the reserved product's status (reserved state) before checkout. In other words, when a reserved product is included in the products managed in the cart information, the reserved state must be released by the intervention of a store clerk. Considering the aspect that the shopping system of this embodiment is originally intended to enable customers to perform the process from product registration to checkout by themselves, it is preferable that reserved products not be included in the cart information. Therefore, for example, the transaction management device 60 may be configured not to manage as a reserved item in the cart information an item (properly canceled item) for which it has been confirmed that both the registration cancellation operation and the item have been returned to a non-reserved state. In this case, in response to the registration cancellation operation being performed, the list item Itm of the product that is the subject of the registration cancellation may be deleted from the product registration screen of the customer terminal device 50. The transaction management device 60 may delete the product information corresponding to the appropriately canceled product from the cart information, but may add information (flag) indicating that the product information corresponding to the appropriately canceled product has been cancelled, and store the information in the cart information, or store the information as part of the registered transaction history, so that a store clerk, for example, can check the cancellation status of the registration later.

[0170] [Sixth Modification] In the above embodiment, a part of the processing executed by the customer terminal device 50 may be executed by, for example, the transaction management device 60, the transaction status management device 80, the settlement device 40, the store controller 10, other higher-level devices in the store, or a server device on the cloud. Similarly, a part of the processing executed by each of the transaction management device 60, the transaction status management device 80, the settlement device 40, the store controller 10, etc. may be executed by other devices. As one of the above-mentioned aspects, the shopping system of the present embodiment may be provided with a thin client system configuration. As an example, the transaction management device 60 or other servers may be configured as a thin client-compatible server device (e.g., a cloud server), and the customer terminal device 50, the transaction status management device 80, etc. may function as thin clients. In this case, for example, the cloud server may determine whether a product has been removed from a shopping cart, or whether a registration cancellation operation has been performed within a determination period, by acquiring as detection information the captured image sent from the customer terminal device 50, and the detection output of the weighing scale, optical sensor, vibration sensor, etc. described in the second variant, and using the acquired detection information. In addition, the transaction management device 60, the transaction status management device 80, the store controller 10, etc. may be configured to be distributed across multiple computer devices or servers.

[0171] <Overview of the embodiment> [Technical field] The present invention relates to a product sales data processing system and a mobile terminal device. [Background technology] 2. Description of the Related Art A system is known in which a customer registers an item using a mobile terminal device (see, for example, Patent Document 1). [Prior art document] [Patent documents] [Patent Document 1] JP 2019-204551 A [Summary of the invention] [Problem to be solved by the invention] In the above-mentioned product sales data processing system, a customer may register a certain product and reserve it by putting it in a shopping cart, but may wish to cancel the registration, for example, because they have selected the wrong product or have decided not to purchase it. In such a case, the customer can operate a mobile terminal to cancel the registration of the target product and return the cancelled product to its original display location, such as a display shelf. However, there are cases in which the customer is unable to properly carry out the procedure for canceling the registration of such a product. If the procedure for canceling the registration of a product is not carried out properly, the product registration result will not match the details of the product reserved by the customer, which will hinder smooth checkout. The present invention has been made in consideration of the above circumstances, and has as its object to enable customers to properly carry out procedures for canceling product registration in a product sales data processing system in which the customers register products using a terminal device.

[0172] [Means for solving the problem] (1) As described above, one aspect of this embodiment is a product sales data processing system (e.g., a shopping system) in which a customer registers products using a mobile terminal device (e.g., customer terminal device 50), and the product sales data processing system is equipped with a detection means for detecting the removal of a product from a product insertion section (e.g., a shopping cart) into which products reserved by the customer are inserted, and a notification means for notifying the customer of the cancellation of the registration of a registered product (e.g., a non-reservation re-guidance screen WD50, a registration cancellation operation guidance screen WD60) based on the detection result by the detection means.

[0173] According to the above configuration, a notification is given depending on whether or not it is detected that a product has been removed from the shopping cart, and the customer can be guided to take appropriate action to cancel the registration of a registered product. This allows the customer to appropriately carry out the procedure for canceling the registration of a product in a shopping system in which the customer registers products using the customer terminal device 50.

[0174] (2) One aspect of this embodiment is the product sales data processing system described in (1), wherein the notification means issues a notification to guide the customer to return the cancelled product to an unsecured state when a certain period of time has passed since the customer cancelled the registration of the registered product by performing a cancellation operation without the detection means detecting the product being removed.

[0175] According to the above configuration, if the customer does not return the target product to a non-secured state even after a certain amount of time has passed since the registration cancellation operation was performed, the customer is guided to return the target product to a non-secured state, and the customer can properly carry out the procedure for canceling the registration of the product.

[0176] (3) One aspect of this embodiment is a product sales data processing system as described in (1) or (2), wherein the notification means issues a notification to guide the user to perform a registration cancellation operation corresponding to the product detected as being removed from the product insertion section if a registration cancellation operation is not performed within a certain time after the detection means detects the removal of the product.

[0177] According to the above configuration, if a customer does not carry out the registration cancellation operation even after a certain amount of time has passed since the customer returned the target product to a non-secured state, the customer is guided to carry out the registration cancellation operation, and the customer can properly carry out the procedure for canceling the product registration.

[0178] (4) One aspect of this embodiment is a product sales data processing system as described in any one of (1) to (3), comprising a server device equipped with the detection means, wherein the detection means acquires predetermined detection information indicating a change in response to the removal of a product from the product insertion section, and detects the removal of a product from the product insertion section based on the acquired detection information.

[0179] According to the above configuration, the server device in the shopping system detects the removal of an item from a shopping cart, so that highly accurate detection results can be obtained through advanced processing. Also, since the customer terminal device 50 does not need to perform processing to detect the removal of an item, the processing load of the customer terminal device 50 can be reduced, and the operation of the customer terminal device 50 during product registration can be made smoother.

[0180] (5) One aspect of this embodiment is a portable terminal device used by a customer to register a product, the portable terminal device comprising: a detection means for detecting the removal of a product from a product insertion section where the product reserved by the customer is inserted; and a notification means for notifying the customer regarding the cancellation of the registration of a registered product based on the detection result by the detection means.

[0181] In addition, a program for realizing the functions of the store controller 10, the settlement device 40, the customer terminal device 50, the transaction management device 60, the transaction status management device 80, etc., in each of the above embodiments may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing as the above store controller 10, the settlement device 40, the customer terminal device 50, the transaction management device 60, the transaction status management device 80, etc. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" here includes hardware such as an OS and peripheral devices. In addition, the "computer system" may include multiple computer devices connected via a network including a communication line such as the Internet, a WAN, a LAN, or a dedicated line. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, etc., and a storage device such as a hard disk built into a computer system. In this way, the recording medium storing the program may be a non-transient recording medium such as a CD-ROM. The recording medium also includes a recording medium provided inside or outside the distribution server and accessible to distribute the program. The code of the program stored in the distribution server's recording medium may be different from the code of the program in a format executable by the terminal device. In other words, the format in which the program is stored in the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a format executable by the terminal device. The program may be divided into multiple parts, downloaded at different times, and then combined in the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a memory that holds the program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or a client when the program is transmitted via a network. The program may be for realizing part of the above-mentioned functions.Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program). [Explanation of symbols]

[0182] 10 store controller, 40 settlement device, 50 customer terminal device, 60 transaction management device, 70 credit card settlement server, 80 transaction status management device

Claims

1. A product sales data processing system in which a customer uses a mobile terminal device to register a product, a detection means for detecting removal of a product from a product insertion section into which the product secured by a customer is inserted; a notification means for notifying a customer of the cancellation of the registration of a registered product based on a result of the detection by the detection means; A product sales data processing system comprising:

2. The notification means issues a notification to guide the customer to return the registered product to a non-secured state when a certain period of time has elapsed since the registration of the registered product was cancelled by a customer's cancellation operation without the detection means detecting the removal of the product.

2. The product sales data processing system according to claim 1.

3. The notification means issues a notification to guide a user to perform a registration cancellation operation corresponding to the product detected to be removed from the product insertion section when the registration cancellation operation is not performed within a certain time after the detection means detects the removal of the product.

3. A product sales data processing system according to claim 1 or 2.

4. A server device including the detection means, The detection means acquires predetermined detection information indicating a change in response to the removal of a product from the product insertion section, and detects the removal of a product from the product insertion section based on the acquired detection information.

4. A product sales data processing system according to claim 1.

5. A mobile terminal device used by a customer to register a product, a detection means for detecting removal of a product from a product insertion section into which the product secured by a customer is inserted; a notification means for notifying a customer of the cancellation of the registration of a registered product based on a result of the detection by the detection means; A mobile terminal device comprising:

Images