Program, information processor, system, server, and method for recognizing article replenishment and removal

Abstract

To operate a shopping cart more effectively.SOLUTION: A program enables an information processor used with an article placement device including a plurality of placement units to function as a computer. The program allows the computer to function as article replenishment / removal recognition means that recognizes the replenishment of articles to the placement units or the removal of the articles from the placement units. The article replenishment / removal recognition means recognizes which placement unit was replenished with the articles or from which placement unit the articles were removed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing device, a system, a server, and a method for recognizing whether a commodity is inserted or removed. [Background technology]

[0002] BACKGROUND ART A shopping cart is known in which one weighing unit collectively weighs the weight of products and the like placed on two placement units (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-127262 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is room for improvement in the operation of the shopping cart.

[0005] The present invention has been made in view of the above circumstances, and has an object to provide a technology for more optimally operating a shopping cart. [Means for solving the problem]

[0006] One aspect of the program for solving the above-mentioned problems is a program for causing an information processing device used in conjunction with a product placement device having multiple placement sections to function as a computer, and for causing the computer to function as a product insertion / removal recognition means that recognizes the insertion of a product into a placement section or the removal of a product from a placement section, the product insertion / removal recognition means being characterized in that it recognizes which placement section a product has been inserted into or which placement section a product has been removed from. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a conceptual diagram of a network for explaining a sales system. [Figure 2] FIG. 2 is an explanatory diagram illustrating a cart. [Figure 3] 10 is a display example of a mobile terminal. [Figure 4] 10 is a display example of a mobile terminal. [Figure 5] 10 is a display example of a mobile terminal. [Figure 6] 10 is a display example of a mobile terminal. [Figure 7] 10 is a flowchart illustrating an example of an operation of the mobile terminal. [Figure 8] FIG. 10 is an explanatory diagram for explaining fraud and the like. [Figure 9] 10 is a display example of a payment device. DETAILED DESCRIPTION OF THE INVENTION

[0008] Fig. 1 is a conceptual diagram of a network for explaining a sales system (which may be referred to as a "sales data processing system") 1 of this embodiment. The sales system 1 shown in Fig. 1 includes a headquarters server 10, a cloud server 20, a store controller 30, a transaction status management device 40, a settlement device 50, and a mobile terminal (which may be referred to as an "information processing terminal (device)," an "information communication terminal (device)," a "mobile communication terminal (device)," a "mobile registration terminal (device)," a "portable registration terminal (device)," etc.) 60.

[0009] The sales system 1 can be introduced into various business types (supermarkets, convenience stores, drug stores, home centers, electronics retailers, etc.). The store controller 30, transaction status management device 40, and payment device 50 are installed within the store (product display area, checkout area, backroom, office, etc.). These can communicate via a LAN 19 (which may be wired or wireless). The headquarters server 10 and cloud server 20 are installed outside the store (for example, at a headquarters or data center, etc.). The devices installed within the store (store controller 30, transaction status management device 40, payment device 50) can communicate (directly or via the store controller 30) with the devices installed outside the store (headquarters server 10, cloud server 20).

[0010] Also, within the store, there is a shopping cart (hereinafter simply referred to as a "cart") 70 prepared by the store. Also within the store, there is a mobile terminal 60 that is owned by a customer who visits the store. The mobile terminal 60 can be attached (set) to the cart 70. The mobile terminal 60 and the cart 70 can communicate with each other via wireless communication (for example, Bluetooth (registered trademark; the same applies below)). In FIG. 1, the mobile terminal 60 shown inside the cart 70 represents the mobile terminal 60 attached to the cart 70. The mobile terminal 60 and the cloud server 20 can communicate with each other (directly or via the store controller 30).

[0011] Sales system 1 provides a self-service (hereinafter, sometimes referred to as service A) in which a customer registers a product using a mobile terminal 60 and settles the payment using a payment settlement device 50. In other words, service A is a self-service in which the customer registers the product and settles the payment themselves, using mobile terminal 60 for product registration and settlement using payment settlement device 50.

[0012] Note that the number of each device (terminal) in FIG. 1 is an example. For example, while "..." is illustrated to the right of the payment device 50 in FIG. 1, the number of payment devices 50 installed in a store may be one or more. Also, for example, while two mobile terminals 60 are illustrated in FIG. 1, the number of mobile terminals 60 present in the store is the number of customers who visit the store with their mobile terminals 60, and is not necessarily two. Also, for example, while two carts 70 are illustrated in FIG. 1, the number of carts 70 present in the store is the number prepared by the store based on the number of mobile terminals 60 expected to be present in the store at the same time, and may be, for example, 10 or more.

[0013] (Mobile terminal 60) The mobile terminal 60 is used by customers themselves when registering products in a store. Specifically, the mobile terminal 60 registers products based on the customer's operation (motion) while in a state where it can communicate with one cart 70. Note that, as long as the mobile terminal 60 is in a state where it can communicate with one cart 70, it can register products based on the customer's operation whether it is attached to the cart 70 or detached from the cart 70 (for example, while the customer is holding it and operating it).

[0014] Although not shown, the mobile terminal 60 includes, for example, a CPU, a storage unit, a display unit, an operation unit, an audio output unit (speaker), an audio input unit (microphone), an imaging unit (camera), a detection unit (sensor), and a communication unit. The mobile terminal 60 is, for example, a smartphone.

[0015] The CPU is a central processing unit that controls the entire mobile terminal 60 by reading and executing programs stored in the storage unit. The storage unit stores various information (for example, information referenced or generated by the CPU, information obtained from the outside, etc.). The storage unit is a general term for one or more various storage components (for example, memory or storage, such as RAM, ROM, hard disk, SSD (Solid State Drive), etc.).

[0016] For example, the mobile terminal 60 may acquire (receive) an application program (hereinafter, App A) for using Service A from an external source (which may be a predetermined server, such as the cloud server 20) and store (install) it in a storage unit. The mobile terminal 60 may execute various processes related to Service A and display screens using App A. As described above, Service A is a self-service service in which the mobile terminal 60 is used for product registration and the settlement device 50 is used for payment, and App A is an application program that handles the mobile terminal 60 (product registration) portion of Service A.

[0017] The display unit is, for example, a liquid crystal display, and displays various information (including image information). The display unit is configured as a touch panel that can accept operations on the display screen. The operation unit accepts operations from an operator (customer). The operation unit is a collective term for one or more various operation members (various operators (hard buttons) provided on the housing. Note that, since the display unit is configured as a touch panel, it may be considered as part of the operation unit. The audio output unit outputs audio. The audio input unit inputs audio. The imaging unit acquires (generates) captured images (still images, moving images). The detection unit (measurement unit) detects or measures various events, etc. The detection unit is a collective term for, for example, one or more various detection members (GPS sensor, acceleration sensor, gyro sensor, illuminance sensor, etc.). Note that the operation unit, audio input unit, and imaging unit may also be considered as part of the detection unit. The communication unit transmits and receives various information. The communication unit is a collective term for one or more various communication members (various communication members such as 5G, wireless LAN, NFC (Near Field Communication), Bluetooth, etc.).

[0018] Needless to say, the mobile terminal 60 includes a timekeeping unit (timekeeping function). The timekeeping unit enables the mobile terminal 60 to manage the current time, the time elapsed from a certain point in time, and the time remaining until a certain point in time.

[0019] For example, the mobile terminal 60 can manage the time that has elapsed since a product was added to a shopping basket or the time that has elapsed since a product was removed from the shopping basket. The mobile terminal 60 can manage the time that has elapsed since a product was registered (scan registration (described later), button registration (described later)). The mobile terminal 60 can manage the time that has elapsed since a product was canceled using the product cancel button (described later). The mobile terminal 60 can manage the time that has elapsed since a quantity change operation (a quantity decrease operation using the minus button (described later), or a quantity increase operation using the plus button (described later)) was performed. The mobile terminal 60 can manage the time that has elapsed since a certain screen (including a small screen (pop-up screen)) was displayed (started to be displayed) or since a certain screen (including a small screen) was erased.

[0020] The mobile terminal 60 connects to one cart 70 via Bluetooth (Bluetooth connection). When connecting to one cart 70, the mobile terminal 60 needs device identification information (e.g., Bluetooth address) of the cart 70 as pairing information. The mobile terminal 60 can acquire the device identification information of each cart 70 from a beacon transmitted by each cart 70. In the following description, the device identification information of the cart 70 may be referred to as connection information with the cart 70 (or simply "connection information").

[0021] The mobile terminal 60 can obtain connection information for each cart 70 from the beacon transmitted by each cart 70, but can also obtain connection information for each cart 70 from a two-dimensional code 78 (described below) affixed to each cart 70.

[0022] The mobile terminal 60 acquires connection information for one cart 70 (the cart 70 that the customer intends to use) based on the customer's actions, connects to the cart 70 using the acquired connection information, and becomes capable of communication (Bluetooth communication) with the cart 70.

[0023] The mobile terminal 60 can display an image captured by the imaging unit as a so-called through image (live view image) on the display unit. When the mobile terminal 60 is attached to the holder unit 76 (described later) of the cart 70 and an image is captured by the imaging unit, the mobile terminal 60 obtains an image captured in a direction looking down on the shopping basket from the position of the holder unit 76 and displays the image on the display unit as a through image.

[0024] The mobile terminal 60 acquires a captured image by the imaging unit whether or not the image is displayed on the display unit as a through image. That is, the mobile terminal 60 acquires a captured image by the imaging unit whether or not the mobile terminal 60 is displaying a scan registration screen (described later) that displays a through image, or a button registration screen (described later) that does not display a through image.

[0025] The mobile terminal 60 may store the captured image in the storage unit instead of or in addition to displaying the captured image as a through image on the display unit. That is, the mobile terminal 60 may store the captured image in the storage unit without displaying it as a through image on the display unit, or may store the captured image as a through image on the display unit without storing it in the storage unit, or may store the captured image in the storage unit while displaying it as a through image on the display unit. The mobile terminal 60 may also transmit the captured image to the cloud server 20.

[0026] The mobile terminal 60 stores in the memory unit, for example, the weighing information received from the cart 70 (the previously received weighing value, the weight of the container (the container weight), etc.). When the mobile terminal 60 receives a weighing value as weighing information from the cart 70, if the weighing value (current weighing value) is greater than the weighing value (previous weighing value) stored in the memory unit, the mobile terminal 60 determines that the weighing value from the cart 70 has increased, and if the weighing value is smaller than the weighing value stored in the memory unit, the mobile terminal 60 determines that the weighing value from the cart 70 has decreased.

[0027] The mobile terminal 60 has various image recognition functions (image processing functions). For example, the mobile terminal 60 has an object recognition function (object determination function) that recognizes objects in a captured image. In addition, the mobile terminal 60 has, for example, the following functions related to the object recognition function.

[0028] The mobile terminal 60 has a barcode reading function that recognizes a barcode in a captured image and reads the recognized barcode. The mobile terminal 60 uses the barcode reading function to read, for example, a JAN code or an in-store code attached (printed) to a product or the like, and acquires a product code (product identification information).

[0029] The mobile terminal 60 has a two-dimensional code reading function that recognizes a two-dimensional code in a captured image and reads the recognized two-dimensional code. The mobile terminal 60 uses the two-dimensional code reading function to read, for example, a two-dimensional code 78 (described later) attached to each cart 70, and acquires connection information.

[0030] The mobile terminal 60 may have a scanning action recognition function that recognizes a scanning action (holding action) by a customer to scan a product (barcode) based on a captured image. The scanning action recognition function may, for example, recognize (by pattern matching) whether something resembling a barcode (for example, an object with a predetermined pattern consisting of multiple line segments) has newly appeared in a predetermined area within the captured image (for example, a code scan area, such as the area displayed with "Please scan the barcode" as shown in FIG. 3(D)).

[0031] (Scan registration, button registration) The mobile terminal 60 has two product registration methods (methods for acquiring product codes): scan registration and button registration. Scan registration is a product registration method based on scanning a barcode. Button registration is a product registration method based on operating (touching) a product button (a preset button for product registration that corresponds to each product displayed on the display).

[0032] (Cart 70) Fig. 2 is an explanatory diagram illustrating the cart 70. Fig. 2(A) is a side view of the cart 70. Fig. 2(B) is a perspective view of the cart 70. The cart 70 has a function of loading and transporting products (products registered by the mobile terminal 60), as well as a function of weighing the products, communicating with the mobile terminal 60, etc.

[0033] As shown in FIGS. 2A and 2B, the cart 70 includes an upper placement section (upper placement platform) 71, a lower placement section (lower placement platform) 72, and a substrate storage section (substrate box) 73. The upper placement section 71 and the lower placement section 72 are each capable of placing a shopping basket or the like thereon. In other words, the cart 70 is capable of placing shopping baskets or the like on two levels, an upper level and an lower level. Although FIGS. 2A and 2B show a state in which no shopping baskets are placed on the upper placement section 71 or the lower placement section 72, shopping baskets or the like can be placed on both the upper placement section 71 and the lower placement section 72. In FIG. 2A, the symbol A indicates the position (space) of the shopping basket placed on the upper placement section 71, and the symbol B indicates the position of the shopping basket placed on the lower placement section 72.

[0034] The board storage section 73 stores one or more boards (details will be described later). A charging terminal 74 is provided on the left surface of the board storage section 73 (the left surface when the direction of travel of the cart 70 is considered the front). A power supply cable (not shown) is detachably connected to the charging terminal 74. Charging terminals 75 are provided on the front and rear surfaces of the board storage section 73 (the front charging terminal 75 is not shown in FIG. 2). The front charging terminal 75 is configured to contact (connect) with the charging terminal 75 on the rear surface of another cart 70 (in other words, the rear charging terminal 75 is configured to contact the charging terminal 75 on the front surface of another cart 70). For example, when multiple carts 70 are stored in a storage area (waiting for return use) lined up in front and behind, the charging terminals 75 of the front and rear carts 70 come into contact with each other, and multiple carts 70 from the first cart 70 to the last cart 70 can be charged by a single power supply.

[0035] The holder section 76 is a member for setting the mobile terminal 60. Note that Fig. 2(A) and Fig. 2(B) show a state in which the mobile terminal 60 is not set in the holder section 76. A customer who comes to the store with the mobile terminal 60 places a shopping basket or the like on the upper placement section 71 or the like, and attaches the mobile terminal 60 to the holder section 76. The holder section 76 has an opening (hole) at the position of the imaging section (lens) of the set mobile terminal 60.

[0036] It has been explained that the mobile terminal 60, when set in the holder unit 76, acquires an image looking down on the shopping basket from the position of the holder unit 76, but when a shopping basket is placed on the upper placement unit 71, it acquires an image looking down on the shopping basket. In other words, the position and angle of the holder unit 76 are adjusted so that the mobile terminal 60, when set in the holder unit 76, acquires an image looking down on the shopping basket placed on the upper placement unit 71. In other words, the cart 70 can place a shopping basket on either the upper placement unit 71 or the lower placement unit 72, but when a shopping basket is placed on the upper placement unit 71, the mobile terminal 60, when set in the holder unit 76, acquires an image of the inside of the shopping basket placed on the upper placement unit 71, regardless of whether a shopping basket is placed on the lower placement unit 72.

[0037] In the following description, the shopping basket placed on the upper loading section 71 may be referred to as the "upper basket" or "upper basket," and the shopping basket placed on the lower loading section 72 may be referred to as the "lower basket" or "lower basket."

[0038] A two-dimensional code 78 is attached to the holder part 76. The two-dimensional code 78 is a two-dimensional code that represents device identification information (connection information) of the cart 70 (the cart 70 to which the two-dimensional code 78 is attached).

[0039] The substrate accommodating unit 72 accommodates substrates (one substrate, separate substrates) that realize weighing and communication functions. In other words, the substrate accommodating unit 72 includes a weighing unit (scale unit) and a communication unit. In the following description, the communication unit included in the substrate accommodating unit 72 may be referred to as a cart communication unit.

[0040] The weighing unit weighs the weight of an item placed on the upper placement portion 71 or the lower placement portion 72. More specifically, the weighing unit weighs the weight of all items placed on the upper placement portion 71 or the lower placement portion 72, without distinguishing between the weight of an item placed on the upper placement portion 71 and the weight of an item placed on the lower placement portion 72. The common support member that supports the upper placement portion 71 and the lower placement portion 72 is configured so that the total weight, including the weight of the items placed on each portion, is applied to the weighing unit (that is, it is configured so that the total weight is not applied to other components of the cart 70 (e.g., the bottom component, the back component, etc.)). Note that the support member that supports the upper placement portion 71 and the support member that supports the lower placement portion 72 may be separate components, rather than a common support member, as long as the total weight, including the weight of the items placed on each portion, is applied to the weighing unit.

[0041] The cart communication unit transmits, for example, a beacon. The cart communication unit also transmits and receives various information to and from the mobile terminal 60. For example, the cart communication unit transmits weighing information to the mobile terminal 60.

[0042] Furthermore, the board housing section 72 may house a board that realizes a memory function (which may be the same board as the board that realizes the weighing function and the communication function, or may be a different board). In other words, the board housing section 72 may further include a memory section. The memory section stores, for example, the weight of the container (the container weight) and the previous weighing value (the weighing value before any increase or decrease).

[0043] If the current weight value is greater than the weight value stored in the storage unit (the previous weight value), the cart 70 may transmit a weight increase notification (or the amount of increase in the weight value) to the mobile terminal 60. If the current weight value is less than the weight value stored in the storage unit, the cart 70 may transmit a weight decrease notification (or the amount of decrease in the weight value) to the mobile terminal 60.

[0044] The cart 70 may also include a timing unit (timing function). That is, similar to the mobile terminal 60, the cart 70 may use the timing unit to manage the current time, the elapsed time from a certain point in time, and the remaining time until a certain point in time.

[0045] For example, the cart 70 may manage the time elapsed from when a product is put into the shopping basket or when a product is removed from the shopping basket. The cart 70 may also manage the time elapsed from a timing based on information received from the mobile terminal 60 (for example, the timing of display (start of display) or erasure of a certain screen (including a small screen), or the timing of an operation on the mobile terminal 60).

[0046] (Tare, tare) A container is an item placed on the upper loading section 71 or the lower loading section 72 in the initial state before shopping begins (more specifically, the state before one item is added). A container may include not only a shopping basket but also the customer's luggage (wallet, bag, etc.). Since the weight of a shopping basket is known, the weight of the customer's luggage can be calculated.

[0047] The cart 70 (cart communication unit) receives tare instruction information from the mobile terminal 60. When the cart 70 receives tare instruction information from the mobile terminal 60, it sets the weighed value at the time of reception as the tare weight. The cart 70 may notify the mobile terminal 60 of the weighed value (tare weight) at the time of reception of the tare instruction information, or may always notify the mobile terminal of 0 g regardless of the weighed value at the time of reception of the tare instruction information. For example, if the weighed value at the time of reception of the tare instruction information from the mobile terminal 60 is 3000 g, the cart 70 may notify the mobile terminal 60 of the weighed value, 3000 g, or may notify the mobile terminal 60 of 0 g, which is the weighed value minus the weighed value (i.e., 3000 g is treated as 0 g). In the former case, the mobile terminal 60 proceeds with processing using 3000 g as the reference weight at the start of shopping, and in the latter case, the mobile terminal 60 proceeds with processing using 0 g as the reference weight at the start of shopping.

[0048] Before adding at least one item of merchandise, the mobile terminal 60 transmits tare instruction information to the cart 70. For example, the mobile terminal 60 connects to the cart 70 using the two-dimensional code 78, and then transmits the tare instruction information to the cart 70.

[0049] (Recognition of product input and product removal) The mobile terminal 60 has a product insertion / removal recognition function (product insertion / removal determination function) that recognizes (determines) the insertion of products into the shopping basket (product insertion) and the removal of products from the shopping basket (product removal) based on the weighing information received from the cart 70 and the image captured by the imaging unit. The mobile terminal 60 uses the product insertion / removal recognition function to recognize the insertion of products into the upper basket (shopping basket placed on the upper placement section 71), the insertion of products into the lower basket (shopping basket placed on the lower placement section 72), the removal of products from the upper basket, and the removal of products from the lower basket.

[0050] (Recognition of putting products into the upper basket) The mobile terminal 60 recognizes that a product has been added to the upper basket when the weight value measured by the cart 70 increases and the image captured by the imaging unit changes. In other words, the mobile terminal 60 recognizes that a product has been added to the upper basket or the lower basket because the weight of the product in the upper basket (the product added to the upper basket) or the weight of the product in the lower basket (the product added to the lower basket) has increased, and recognizes (determines) that a product has been added to the upper basket because the image captured by the imaging unit (the image captured inside the upper basket) has changed (see FIG. 7).

[0051] The mobile terminal 60 may recognize that the captured image has changed when there is a change in the number of objects (objects recognized in the captured image) in the captured image (the number of objects increases or decreases).

[0052] When a product is placed in the upper basket (when the mobile terminal 60 recognizes that a product has been placed in the upper basket), the mobile terminal 60 may notify the user (for example, via a screen or by voice) that the product has been placed in the upper basket. For example, the mobile terminal 60 may display a message such as "Item has been placed in the upper basket" (on the small screen SG1 in FIGS. 5(A), 5(G), 8(A), and 8(B)).

[0053] When a product is added to the upper basket, the mobile terminal 60 may transmit information to the cloud server 20 indicating that a product has been added to the upper basket, instead of or in addition to the notification as described above. For example, when a product is added to the upper basket, the mobile terminal 60 may transmit a "product added (upper basket) notification" to the cloud server 20 indicating that a product has been added to the upper basket. In addition, when the cloud server 20 receives the "product added (upper basket) notification" from the mobile terminal 60, it may store the "information indicating that a product has been added to the upper basket" in the transaction information of the mobile terminal.

[0054] If the weight value (current weight value) received as weight information from the cart 70 is greater than the weight value (previous weight value) stored in the memory unit of the mobile terminal 60, the mobile terminal 60 determines that the weight value from the cart 70 has increased. The same applies to "recognizing the insertion of a product into the lower basket."

[0055] The mobile terminal 60 may receive a weight increase notification (or an increase in the weight value) from the cart 70 as weighing information. When the mobile terminal 60 receives a weight increase notification (or an increase in the weight value) from the cart 70, it determines that the weight value of the cart 70 has increased. The same applies to "recognizing the insertion of a product into the lower basket."

[0056] (Recognition of putting products into the lower basket) The mobile terminal 60 recognizes that a product has been added to the lower basket if the weight value measured by the cart 70 has increased and the image captured by the imaging unit has not changed. In other words, the mobile terminal 60 recognizes that a product has been added to the upper basket or the lower basket because the weight of the product in the upper basket or the weight of the product in the lower basket has increased, and recognizes (determines) that a product has been added to the lower basket because the image captured by the imaging unit has not changed (see FIG. 7).

[0057] When a product is put into the lower basket (when the mobile terminal 60 recognizes that a product has been put into the lower basket), the mobile terminal 60 may notify the user that the product has been put into the lower basket. For example, the mobile terminal 60 may display a message such as "The product has been put into the lower basket" (small screen SG2 in FIG. 5(C)).

[0058] When a product is added to the lower basket, the mobile terminal 60 may transmit information to the cloud server 20 indicating that a product has been added to the lower basket, instead of or in addition to the notification as described above. For example, when a product is added to the lower basket, the mobile terminal 60 may transmit a "product added (lower basket) notification" to the cloud server 20 indicating that a product has been added to the lower basket. In addition, when the cloud server 20 receives the "product added (lower basket) notification" from the mobile terminal 60, it may store the "information indicating that a product has been added to the lower basket" in the transaction information of the mobile terminal.

[0059] (Recognizing product removal from the upper basket) The mobile terminal 60 recognizes that a product has been removed from the upper basket when the weight value measured by the cart 70 decreases and the image captured by the imaging unit changes. In other words, the mobile terminal 60 recognizes that a product has been removed from the upper basket or the lower basket because the weight of the product in the upper basket or the weight of the product in the lower basket has decreased, and recognizes (determines) that a product has been removed from the upper basket because the image captured by the imaging unit has changed (see FIG. 7).

[0060] When a product is removed from the upper basket (when the mobile terminal 60 recognizes that a product has been removed from the upper basket), the mobile terminal 60 may notify the user of the removal of the product from the upper basket. For example, the mobile terminal 60 may display a message such as "A product has been removed from the upper basket" (small screen SG3 in FIGS. 6(B), 6(G), 8(C), and 8(D)).

[0061] When a product is removed from the upper basket, the mobile terminal 60 may transmit information to the cloud server 20 indicating that a product has been removed from the upper basket, instead of or in addition to the notification described above. For example, when a product is removed from the upper basket, the mobile terminal 60 may transmit a "product removal (upper basket) notification" to the cloud server 20 indicating that a product has been removed from the upper basket. In addition, when the cloud server 20 receives the "product removal (upper basket) notification" from the mobile terminal 60, it may store the "information indicating that a product has been removed from the upper basket" in the transaction information of the mobile terminal.

[0062] If the weighing value (current weighing value) received as weighing information from the cart 70 is smaller than the weighing value (previous weighing value) stored in the memory unit of the mobile terminal 60, the mobile terminal 60 determines that the weighing value from the cart 70 has decreased. The same applies to "recognizing the removal of a product from the lower basket."

[0063] The mobile terminal 60 may receive a weight loss notification (or the amount of decrease in the weight value) from the cart 70 as weighing information. When the mobile terminal 60 receives a weight loss notification (or the amount of decrease in the weight value) from the cart 70, it determines that the weight value of the cart 70 has decreased. The same applies to "recognizing the removal of a product from the lower basket."

[0064] (Recognition of product removal from the lower basket) The mobile terminal 60 recognizes that a product has been removed from the lower basket if the weight value measured by the cart 70 has decreased and the image captured by the imaging unit has not changed. In other words, the mobile terminal 60 recognizes that a product has been removed from the upper basket or the lower basket because the weight of the product in the upper basket or the weight of the product in the lower basket has decreased, and recognizes (determines) that a product has been removed from the lower basket because the image captured by the imaging unit has not changed (see FIG. 7).

[0065] When a product is removed from the lower basket (when the mobile terminal 60 recognizes that a product has been removed from the lower basket), the mobile terminal 60 may notify the user of the removal of the product from the lower basket. For example, the mobile terminal 60 may display a message such as "A product has been removed from the lower basket" (small screen SG4 in FIG. 6(E)).

[0066] When a product is removed from the lower basket, the mobile terminal 60 may transmit information to the cloud server 20 indicating that a product has been removed from the lower basket, instead of or in addition to the notification described above. For example, when a product is removed from the lower basket, the mobile terminal 60 may transmit a "product removal (lower basket) notification" to the cloud server 20 indicating that a product has been removed from the lower basket. In addition, when the cloud server 20 receives the "product removal (lower basket) notification" from the mobile terminal 60, it may store the "information indicating that a product has been removed from the lower basket" in the transaction information of the mobile terminal.

[0067] (Monitoring fraud, etc.) The mobile terminal 60 monitors for fraudulent acts (fraud or mistakes) by customers based on the status of product registration, product cancellation, etc., and the status of product insertion and removal using the product insertion / removal recognition function. For example, the mobile terminal 60 monitors for unregistered insertion, unregistered insertion, uncancelled removal, and uncancelled removal.

[0068] (Monitoring unregistered entries) An unregistered input is a state in which no product is registered (including an increase in quantity) but a product is input (a state in which a product is input (the input is recognized) but the product is not registered). An unregistered input results in a loss (disadvantage) for the store because the reserved product (the input product) is not subject to settlement. In other words, the mobile terminal 60 monitors whether an unregistered input has occurred, because if an unregistered product is input into the shopping basket, it will result in a loss for the store. An unregistered input is a state in which a product has not been registered, so it can also be called a "forgotten registration."

[0069] The mobile terminal 60 recognizes an unregistered input (recognizes that an unregistered input has occurred) if there is no product registration or quantity increase before or after recognizing the input of a product. As an example, the mobile terminal 60 may determine that an unregistered input has occurred if there is no scan registration, button registration, or quantity increase within a predetermined time (e.g., 8 seconds) before the timing of recognizing the input of a product (putting a product into the upper basket or lower basket) and if there is no button registration or quantity increase within a predetermined time (e.g., 8 seconds) after the timing of recognizing the input of a product (putting a product into the upper basket or lower basket).

[0070] When an unregistered input is made (when the mobile terminal 60 recognizes an unregistered input), it notifies the user of the unregistered input (for example, by notifying via the screen or by voice). For example, the mobile terminal 60 may notify the user of the unregistered input while identifying the shopping basket into which the product was input, and instruct the user to register the product (small screen SG21 in FIG. 8(A)). The mobile terminal 60 may also notify the user that "registration was forgotten."

[0071] When an unregistered input has occurred, the mobile terminal 60 may transmit information to the cloud server 20 indicating that an unregistered input has occurred, instead of or in addition to the notification as described above. For example, when an unregistered input has occurred, the mobile terminal 60 may transmit to the cloud server 20 an "unregistered input notification (upper basket / lower basket)" indicating that an unregistered input has occurred.

[0072] In addition, cloud server 20 that receives an "unregistered input notification" from mobile terminal 60 may store "information indicating unregistered input" in the transaction information of the mobile terminal. Furthermore, cloud server 20 that receives an "unregistered input notification" from mobile terminal 60 may, instead of or in addition to the above, set a confirmation required flag (a confirmation required flag (upper basket / lower basket) for confirming forgotten registration) in the transaction information of the mobile terminal.

[0073] (Monitoring of unregistered entries) "Not registered and added" refers to a state in which a product has been registered (including an increase in quantity) but no product has been added (a state in which a product has been registered but no product has been added (a state in which the product has been added but the product has not been recognized)). "Not registered and added" results in a loss for the customer because the product to be settled (registered product) has not been secured. In other words, the mobile terminal 60 monitors whether "not registered and added" has occurred, because if a registered product is not added to the shopping basket, it results in a loss for the customer. "Not registered and added" refers to a state in which a product has not been added to the shopping basket, so it can also be called "forgotten to add."

[0074] If the mobile terminal 60 does not recognize the insertion of a product before or after the product registration or quantity increase, it recognizes that a registration has not been made (determines that a registration has not been made). As an example, the mobile terminal 60 may determine that a registration has not been made if it does not recognize the insertion of a product (insertion of a product into the upper basket or lower basket) within a predetermined time (e.g., 8 seconds) after the timing of scan registration or button registration or quantity increase, and does not recognize the insertion of a product (insertion of a product into the upper basket or lower basket) within a predetermined time (e.g., 8 seconds) before the timing of button registration or quantity increase.

[0075] When an unregistered item is inserted (when the mobile terminal 60 recognizes that an unregistered item has been inserted), the mobile terminal 60 notifies the user of the unregistered item (for example, by displaying a notification on the screen or by voice). For example, the mobile terminal 60 may notify the user of the unregistered item and instruct the user to insert the item (small screen SG30 in FIG. 8(B)). The mobile terminal 60 may also notify the user that an item has been "forgotten to be inserted."

[0076] When an unregistered input has occurred, the mobile terminal 60 may transmit information to the effect that an unregistered input has occurred, instead of or in addition to the notification as described above. For example, when an unregistered input has occurred, the mobile terminal 60 may transmit a "unregistered input notification" to the cloud server 20 to the effect that an unregistered input has occurred.

[0077] In addition, the cloud server 20 that has received the "not registered / inserted notification" from the mobile terminal 60 may store "information indicating that registration has not been performed" in the transaction information of the mobile terminal. In addition, in place of or in addition to the above, the cloud server 20 that has received the "not registered / inserted notification" from the mobile terminal 60 may set a confirmation required flag (a confirmation required flag for confirming that insertion has been forgotten) in the transaction information of the mobile terminal.

[0078] (Monitoring of uncancelled withdrawals) Uncancelled removal is a state in which there is no product cancellation (including a decrease in quantity) and a product has been removed (a state in which a product has been removed (a product has been recognized as being removed) but the product has not been cancelled). Uncancelled removal results in a loss for the customer because an unsecured product (a removed product) is subject to settlement (it has not been cancelled). In other words, the mobile terminal 60 monitors whether an uncancelled removal has occurred, because if an uncancelled product is removed from the shopping basket, it will result in a loss for the customer. Uncancelled removal may also be called a "forgotten cancellation," because the product has not been cancelled, etc.

[0079] The mobile terminal 60 recognizes an uncancelled removal (recognizes that an uncancelled removal has occurred) if there is no product cancellation or quantity reduction before or after recognizing the product removal. As an example, the mobile terminal 60 may determine that an uncancelled removal has occurred if there is no product cancellation or quantity reduction within a predetermined time (e.g., 8 seconds) before the timing of recognizing the product removal (product removal from the upper basket or lower basket) and if there is no product cancellation or quantity reduction within a predetermined time (e.g., 8 seconds) after the timing of recognizing the product removal (product removal from the upper basket or lower basket).

[0080] When there is an uncancelled removal (when the mobile terminal 60 recognizes an uncancelled removal), it notifies the user of the uncancelled removal (for example, by notifying via the screen or by voice). For example, the mobile terminal 60 may notify the user of the uncancelled removal while identifying the shopping basket from which the product was removed, and instruct the user to cancel the product (small screen SG41 in FIG. 8(C)). The mobile terminal 60 may also notify the user that "cancellation forgotten."

[0081] When an uncancelled retrieval has occurred, the mobile terminal 60 may transmit information to the effect that an uncancelled retrieval has occurred, instead of or in addition to the notification as described above, to the cloud server 20. For example, when an uncancelled retrieval has occurred, the mobile terminal 60 may transmit to the cloud server 20 an "uncancelled retrieval notification (upper basket / lower basket)" to the effect that an uncancelled retrieval has occurred.

[0082] Cloud server 20, upon receiving an "uncancelled and removed notification" from mobile terminal 60, may store "information indicating that the item has not been canceled and removed" in the transaction information of the mobile terminal. Also, cloud server 20, upon receiving an "uncancelled and removed notification" from mobile terminal 60, may set a confirmation required flag (a confirmation required flag (upper basket / lower basket) for confirming whether cancellation has been forgotten) in the transaction information of the mobile terminal, instead of or in addition to the above.

[0083] (Monitoring cancellations and non-retrieval) A "cancelled but not removed" state is a state in which a product has been cancelled (including a reduction in quantity) but has not been removed (a state in which a product has been cancelled but has not been removed (a state in which the product has been removed but not recognized)). A "cancelled but not removed" state results in a loss for the store because a product that is not subject to payment (a cancelled product) has been reserved (not removed). In other words, the mobile terminal 60 monitors whether a "cancelled but not removed" state has occurred, because if a cancelled product is not removed from the shopping basket, it will result in a loss for the store. A "cancelled but not removed" state may also be called a "forgotten to remove" state, because a product has not been removed from the shopping basket.

[0084] If the mobile terminal 60 does not recognize any product removal before or after the product cancellation or quantity reduction, it recognizes a canceled, unremoved item (recognizes that a canceled, unremoved item has occurred). As an example, the mobile terminal 60 may determine that a canceled, unremoved item has occurred if it does not recognize any product removal (product removal from the upper basket or lower basket) within a predetermined time (e.g., 8 seconds) before the product cancellation (or quantity reduction) timing and does not recognize any product removal (product removal from the upper basket or lower basket) within a predetermined time (e.g., 8 seconds) after the product cancellation (or quantity reduction) timing.

[0085] When there is a canceled item that has not been removed (when the mobile terminal 60 recognizes that there is a canceled item that has not been removed), it notifies the user of the canceled item that has not been removed (for example, by notifying via the screen or by voice). For example, the mobile terminal 60 may notify the user of the canceled item that has not been removed and instruct the user to remove the item (small screen SG50 in FIG. 8(D)). The mobile terminal 60 may also notify the user that the item has been "forgotten to be removed."

[0086] When a cancellation has not been performed and the item has not been retrieved, the mobile terminal 60 may transmit information indicating that a cancellation has not been performed and the item has not been retrieved to the cloud server 20, instead of or in addition to the notification as described above. For example, when a cancellation has not been performed and the item has not been retrieved, the mobile terminal 60 may transmit a "cancellation not being performed and the item has not been retrieved" to the cloud server 20.

[0087] Cloud server 20, upon receiving a "cancellation not yet removed" notification from mobile terminal 60, may store "information indicating that the item has been canceled and not yet removed" in the transaction information of the mobile terminal. Furthermore, cloud server 20, upon receiving a "cancellation not yet removed" notification from mobile terminal 60, may set a confirmation required flag (a confirmation required flag for confirming whether the item has been forgotten to be removed) in the transaction information of the mobile terminal instead of or in addition to the above.

[0088] (Payment device 50) The settlement device 50 is used by customers. Specifically, the settlement device 50 settles the purchase of products registered via the mobile terminal 60 based on the customer's operation. Although not shown, the settlement device 50 includes a CPU, a memory unit, a display unit, a scanner unit, a payment unit, a printer unit, an audio output unit, an imaging unit, a communication unit, and a sign pole. For example, the scanner unit scans various codes and reads each code. For example, the payment unit is a collective term for either or both of a cash payment unit (change dispenser) that settles with cash and a non-cash payment unit (card payment unit, code payment unit) that settles with non-cash (e.g., credit, electronic money, etc.). For example, the display unit displays a product list screen ( FIG. 9 ). Note that the main function of the settlement device 50 is settlement, but it also has functions to register products and cancel products. The settlement device 50 may further include a key operation unit consisting of one or more various operation members, a human presence sensor that detects (detects, recognizes) the presence of a person, etc.

[0089] (Headquarters Server 10) The headquarters server 10 is a server that manages the entire sales system 1. The headquarters server 10 stores, for example, various information (for example, a product master, a product button file, etc.). The product master is a file that stores product names, prices, etc. in association with product codes. The product button file stores, for example, in association with product button identification information that identifies the product button, the product code of the product designated (specified) by the product button, category identification information that identifies the category to which the product button belongs (i.e., category identification information that identifies the category to which the product designated by the product button belongs), an image (captured image, illustration, etc.) to be displayed on the product button, etc.

[0090] The headquarters server 10 also supplies various information (e.g., product master data) to, for example, the cloud server 20 and the store controller 30. Although not shown in the figure, the headquarters server 10 includes at least a CPU, a storage unit, and a communication unit. The headquarters server 10 may be configured from multiple servers.

[0091] (Cloud Server 20) Cloud server 20 is a server that mainly supports service A. Cloud server 20 stores various information (for example, product master, product button file, transaction information (cart information), etc.). Cloud server 20 also supplies various information to, for example, mobile terminal 60 and settlement device 50. Although not shown in the figure, cloud server 20 includes at least a CPU, a storage unit, and a communication unit. Cloud server 20 may be composed of multiple servers.

[0092] The transaction information of the mobile terminal 60 is generated, for example, when a transaction is started on the mobile terminal 60, and stores registration information of the product registered by the mobile terminal 60. The transaction information of each mobile terminal 60 may store the status (progress of processing) of each mobile terminal 60. The status stored in the transaction information includes a registration status (registration status) indicating that product registration is being carried out, a settlement declaration status (settlement declaration status) indicating that a declaration has been made to proceed to settlement, and the like.

[0093] (Store Controller 30) The store controller 30 is a server that is mainly responsible for communicating information (e.g., product master, product button file, etc.) between devices installed inside the store (transaction status management device 40, settlement device 50) and devices installed outside the store (headquarters server 10, cloud server 20). Although not shown in the figure, the store controller 30 includes at least a CPU, a memory unit, and a communication unit.

[0094] (Transaction status management device 40) The transaction status management device 40 is, for example, a device (e.g., a personal computer) that manages (monitors, controls, etc.) the settlement device 50 and the mobile terminal 60. For example, the transaction status management device 40 monitors (displays) the processing status and operating status of the settlement device 50. The transaction status management device 40 may also monitor the processing status and operating status of the mobile terminal 60 (e.g., access the cloud server 20 and display or modify the product master, transaction information, etc.). The transaction status management device 40 may also monitor the status of the cart 70 (e.g., access the cloud server 20 and display the status of the cart 70's battery, etc. (battery 76, cart charging unit 73, mobile terminal charging unit 75) and various error information transmitted via the mobile terminal 60 and stored in the cloud server 20). The transaction status management device 40 may also access the headquarters server 10, etc., and perform maintenance (e.g., product settings) on the product master stored in the headquarters server 10, etc. Although not shown in the figures, the transaction status management device 40 includes at least a CPU, a memory unit, a display unit (which may be configured as a touch panel), an operation unit (keyboard, mouse, etc.), and a communication unit. The transaction status management device 40 can communicate with a printing device (not shown), and various information may be output from the printing device.

[0095] (Example of display on mobile terminal 60) 3 to 6 are examples of displays on the mobile terminal 60. It is assumed that the mobile terminal 60 has already installed the application A.

[0096] 3(A) is the initial screen (standby screen, a screen in which application A and other applications are not running) of the mobile terminal 60. The initial screen in FIG. 3(A) displays the icon of application A (denoted by symbol A in the figure), etc.

[0097] Fig. 3(B) is a menu screen (start screen) that is displayed after application A is launched. For example, when the icon for application A on the initial screen (Fig. 3(A)) is operated, mobile terminal 60 displays a menu screen as shown in Fig. 3(B). On the menu screen of Fig. 3(B), various buttons are displayed in an operable manner, such as a start shopping with cart use button BT610A that accepts an operation to start shopping using cart 70, and a start shopping without using cart button BT610B that accepts an operation to start shopping without using cart 70.

[0098] Fig. 3(C) shows the connection code reading instruction screen that is displayed after pressing the shopping cart use start button BT610A. The connection code reading instruction screen in Fig. 3(C) displays a message MS613 that instructs how to read the connection code (two-dimensional code 78) and an explanatory image GA612 that explains how to read the connection code.

[0099] After the connection with the cart 70 is completed by reading the two-dimensional code 78 (e.g., immediately after the connection is completed), the mobile terminal 60 may transmit tare instruction information to the cart 70 and execute the tare. The mobile terminal 60 may display a message instructing the customer to place a shopping basket on the front of the menu screen of FIG. 3(B) based on the operation of the "Shopping with Cart" button BT610A on the menu screen of FIG. 3(B). For example, the mobile terminal 60 may display a small screen (not shown) with an OK button and a message such as "If you are using one basket, please place it on the upper side (upper placement section 71). If you are using two baskets, please also place one on the lower side (lower placement section 72). Once you have finished placing the baskets, press the OK button." The small screen may then be cleared based on the operation of the OK button, and the connection code reading instruction screen of FIG. 3(C) may be displayed. In other words, the mobile terminal 60 may instruct the customer to place a tare before executing the tare (before the two-dimensional code 78 is read).

[0100] When only one shopping basket is used, it is common to set the shopping basket on the upper placement section 71, but just to be safe, customers are instructed to set it on the upper side when only one shopping basket is used. In other words, when a shopping basket is set on the lower placement section 72 without being set on the upper placement section 71, depending on the components of the upper placement section 71, images of the products in the shopping basket on the lower placement section 72 may be captured through the upper placement section 71, which may result in inaccurate recognition of the upper side / lower side when inserting (or removing) products by the mobile terminal 60. Therefore, in order to operate the product insertion / removal recognition function preferably (as when two shopping baskets are used) even when only one shopping basket is used, customers are instructed to set the shopping basket on the upper placement section 71 when only one shopping basket is used. The mobile terminal 60 may display a message such as "Please set the basket on the upper side" when a shopping basket is set on the lower placing section 72 without a shopping basket being set on the upper placing section 71. A specific mark or the like may be affixed to the top surface of the upper placing section 71 (a position that will not be imaged when a shopping basket is placed on it), and the mobile terminal 60 may determine that a shopping basket is not set on the upper placing section 71 when an image of the specific mark or the like is captured.

[0101] The mobile terminal 60 may confirm with the customer the number of times the shopping basket is being used. For example, the mobile terminal 60 may display a small screen on which buttons for accepting the number of times the shopping basket is being used (one button to be operated when the number of times is 1, and two buttons to be operated when the number of times is 2) are arranged based on the operation of the shopping basket use start button BT610A. For example, when the number of times is 1 (when the one button is operated), the mobile terminal 60 displays a message such as "Please place the basket on the top level," and when the number of times is 2 (when the two button is operated), it displays a message such as "Please place a shopping basket on each of the top and bottom levels."

[0102] The mobile terminal 60 may determine the number of times the shopping basket has been used based on the number of times the weight value has increased (for example, the number of times a weight increase notification has been received). When the number of uses is 1 (for example, when the first weight increase notification has been received but the second weight increase notification has not been received) and the shopping basket is set on the lower placement section 72 (for example, when the image of the above-mentioned specific mark or the like is captured), the mobile terminal 60 may display a message such as "Please set the basket on the upper side."

[0103] The mobile terminal 60 may display the small screen in front of the connection code reading instruction screen of Fig. 3(C) based on the operation of the shopping with cart use start button BT610A on the menu screen of Fig. 3(B). In other words, the mobile terminal 60 may display the small screen after switching from the menu screen of Fig. 3(B) to the connection code reading instruction screen of Fig. 3(C).

[0104] The small screen may be displayed after the connection with the cart 70 is completed by reading the two-dimensional code 78, and the small screen may be cleared and the tare weight may be subtracted by pressing the OK button. By prompting the user to set the shopping basket at least before the tare weight is subtracted, it is possible to prevent the problem of the shopping basket being judged to contain unregistered items when it is set after the tare weight is subtracted.

[0105] Furthermore, instead of or in addition to the store-provided shopping basket, customers may be encouraged to use their own bag or basket (for example, a screen display may be displayed). If customers start shopping using a store-provided shopping basket, they will eventually (before leaving the store) (for example, after paying) have to transfer the items (purchased items) in the shopping basket into their own bag or bag. However, if customers start shopping using their own bag or basket, this transfer work does not occur, allowing for hassle-free and efficient shopping.

[0106] In the following description, it is assumed that the customer uses two shopping baskets, that is, the customer places one shopping basket on the upper placement section 71 and one shopping basket on the lower placement section 72.

[0107] FIG. 3(D) is a registration screen (initial registration screen) that is displayed after connection with the cart 70 (connection using device identification information acquired from the two-dimensional code 78) is completed. The registration screen in FIG. 3(D) is provided with a captured image display area HR620, a registration information display area HR630, etc. The captured image display area HR620 is an area that mainly displays a captured image. The registration information display area HR630 is an area that mainly displays product registration information (transaction details). The captured image display area HR620 (above the captured image display area HR620) is provided with a registration method selection button display area HR621, in which a recommendation button BT624 is displayed. The registration method selection button display area HR621 is an area that displays a registration method selection button.

[0108] The registration method selection button is a button that allows selection (switching) of the product registration method (scan registration / button registration). In the registration method selection button display area HR621, the scan registration button BT622 (a button displayed as "Scan" in this example) and the button registration button BT623 (a button displayed as "No barcode" in this example) are displayed as registration method selection buttons. The customer operates (touches) the scan registration button BT622 when switching the product registration method from button registration to scan registration, and operates the button registration button BT623 when switching the product registration method from scan registration to button registration.

[0109] When scan registration is selected, the portable terminal 60 displays the scan registration button BT622 in a display mode indicating a selected state (shown with a thick frame in the figure), and displays the button registration button BT623 in a display mode indicating a non-selected state (shown with a dashed line in the figure). When button registration is selected, the portable terminal 60 displays the button registration button BT623 in a display mode indicating a selected state, and displays the scan registration button BT622 in a display mode indicating a non-selected state. On the registration screen in Figure 3(D), the scan registration button BT622 is displayed with a thick frame, and the button registration button BT623 is displayed with a dashed line (scan registration is selected).

[0110] When the recommendation button BT624 is operated (touched), the mobile terminal 60 displays a recommended product screen (not shown) that presents recommended products. Note that the "12" in "★12" in this example indicates that there are 12 recommended products.

[0111] A registration screen on which scan registration is selected may be referred to as a scan registration screen, and a registration screen on which button registration is selected may be referred to as a button registration screen.

[0112] Fig. 3(E) is a display example of a scene following Fig. 3(D). When the barcode of one product (whitebait) is being imaged by the customer's action, the mobile terminal 60 displays a registration screen (scan registration screen) as shown in Fig. 3(E).

[0113] Fig. 3(F) is a display example of a scene following Fig. 3(E). When the mobile terminal 60 acquires the product code for whitebait from the barcode, it displays a registration screen (scan registration screen) as shown in Fig. 3(F). The registration information display area HR630 on the registration screen in Fig. 3(F) displays the registration information of the product (whitebait) that was scanned and registered. The quantity button BT632 displays the purchase quantity and is operated to change the purchase quantity or cancel the purchase.

[0114] FIG. 3(G) is an example of a display following FIG. 3(F). When the mobile terminal 60 recognizes that a product (whitebait) has been placed in the shopping basket (the upper basket in this example), it displays "Upper Basket" in association with the product (whitebait) and displays small screen SG1 in the foreground, as shown in FIG. 3(G). Small screen SG1 is a small screen that notifies the user that a product has been placed in the upper basket. Small screen SG1 displays a message such as "Added to upper basket." The mobile terminal 60 may output a confirmation sound when small screen SG1 starts to display (similarly when other small screens start to display).

[0115] After registering and inserting a product (one item of product), the mobile terminal 60 displays an operable checkout instruction button BT631 (in this example, a button displaying "Proceed to Checkout") that accepts checkout instruction operations, as shown in Figure 3 (G).

[0116] Fig. 3(H) is a display example of a scene following Fig. 3(G). When a predetermined time (for example, 2 seconds) has elapsed since the start of displaying the small screen SG1, the mobile terminal 60 erases the small screen SG1 as shown in Fig. 3(G).

[0117] Fig. 4(A) is a display example of a scene following Fig. 3(H). When the barcode of the second item (loaf of bread) is being imaged by the customer's action, the mobile terminal 60 displays a registration screen (scan registration screen) as shown in Fig. 4(A).

[0118] Fig. 4(B) is a display example of a scene following Fig. 4(A). When the product code of the bread is acquired from the barcode, the mobile terminal 60 displays a registration screen (scan registration screen) as shown in Fig. 4(B).

[0119] Fig. 4(C) is an example of a display following Fig. 4(B). When the mobile terminal 60 recognizes that a product (bread) has been placed in the shopping basket (in this example, the lower basket), it displays "lower basket" in association with the product (bread) and displays small screen SG2 in the foreground. Small screen SG2 is a small screen that notifies the user that a product has been placed in the lower basket. Small screen SG2 displays a message such as "Added to lower basket."

[0120] Fig. 4(D) is a display example of a scene following Fig. 4(C). When a predetermined time (for example, 2 seconds) has elapsed since the start of displaying the small screen SG2, the mobile terminal 60 erases the small screen SG2 as shown in Fig. 4(D).

[0121] FIG. 4(E) is an example display of a scene following FIG. 4(D). When the checkout instruction button BT631 is operated, the mobile terminal 60 displays a settlement code display screen such as that shown in FIG. 4(E). The settlement code display screen in FIG. 4(E) displays the settlement code GA641 (described below), a message MS642 that instructs the settlement machine (settlement device 50) to read the settlement code GA641, and an explanatory image GA643 that explains how to read the settlement code GA641 on the settlement machine. The settlement code display screen in FIG. 4(E) also displays an operable Return to Shopping button BT649. When the Return to Shopping button BT649 is operated, the registration screen that was displayed before the checkout instruction button BT631 was operated is displayed.

[0122] The settlement code GA641 (indicated as "Accounting Code" in message MS642) is a two-dimensional code of transaction identification information that identifies the transaction information of the transaction. In other words, the settlement code GA641 displayed by a certain mobile terminal 60 is information for identifying the transaction information of the mobile terminal 60 (transaction information of the product registered by the mobile terminal 60) from among multiple pieces of transaction information stored in cloud server 20.

[0123] The settlement device 50, which reads the settlement code GA641 displayed by the mobile terminal 60, transmits transaction information acquisition request information including the transaction identification information (transaction identification information of the mobile terminal 60) acquired from the settlement code GA641 to the cloud server 20. The cloud server 20, which receives the transaction information acquisition request information from the settlement device 50, transmits the transaction information included in the transaction information acquisition request information to the settlement device 50. In other words, when the settlement device 50 reads the settlement code GA641 displayed by the mobile terminal 60, it acquires (receives) the transaction information of the mobile terminal 60 from the cloud server 20.

[0124] The settlement code GA641 may be a two-dimensional code representing location information of the transaction information of the mobile terminal 60 (a storage location within the sales system 1, such as a URI (Uniform Resource Identifier)).

[0125] Figures 4(F) to 5(D) are screen examples of a different scene (different scene 1) from the above-mentioned Figures 3(A) to 4(E). Figure 4(F) is a display example of another scene following Figure 3(H).

[0126] When the quantity button BT632 on the registration screen of Fig. 3(H) is operated, the mobile terminal 60 displays a small screen SG10 on top of the registration screen of Fig. 3(H), as shown in Fig. 4(F). The mobile terminal 60 tone-downs the back (except for the registration information portion of the product) to make the small screen SG10 on the front and the product (whitebait) stand out, but can accept a touch on any part of the back as an operation to clear the small screen SG10. In other words, while the small screen SG10 is displayed, the area behind the small screen SG10 is an area that accepts an operation to clear the small screen SG10.

[0127] The small screen SG10 displays the purchase quantity of the product (whitebait) (displayed as "xn") and the total price for the purchase quantity of the product, and also has a product cancellation button (in this example, a button displayed as "Cancel Product"), a minus button (in this example, a button displayed as "-"), and a plus button (in this example, a button displayed as "+").

[0128] The product cancellation button is a button that is operated when canceling the registration of the registered product (all items). The minus button is a button that is operated when decreasing the purchase quantity of the registered product. Operating the minus button once (quantity decrease operation) decreases the purchase quantity by one. In the example shown in Figure 3, the purchase quantity is "1", so the minus button cannot be operated. The plus button is a button that is operated when increasing the purchase quantity of the registered product. Operating the plus button once (quantity increase operation) increases the purchase quantity by one.

[0129] Fig. 4(G) is a display example of a scene following Fig. 4(F). When the plus button on the small screen SG10 is operated twice (when an operation is performed to increase the purchase quantity from 1 to 3), the mobile terminal 60 displays the small screen SG10 as shown in Fig. 4(G).

[0130] Fig. 4(H) is a display example of a scene following Fig. 4(G). When an operation is performed on an area other than the small screen SG10 (any area on the back), the mobile terminal 60 erases the small screen SG10 and displays a registration screen that reflects the operation to increase the quantity, as shown in Fig. 4(H).

[0131] In addition, if an operation is performed on an area other than the small screen SG10 without any operation being performed on the small screen SG10 (such as canceling a product using the product cancel button, decreasing the quantity using the minus button, or increasing the quantity using the plus button), the mobile terminal 60 will simply erase the small screen SG10 (Figure 4(F) → Figure 3(H)).

[0132] In addition, a decision button to be operated when deciding (confirming) an operation (product cancellation operation, quantity decrease operation, quantity increase operation) or a cancel button to be operated when simply erasing the small screen SG10 may be arranged on the small screen SG10. When the decision button or cancel button is arranged on the small screen SG10, the mobile terminal 60 may erase the small screen SG10 and display a registration screen reflecting the quantity increase operation, as shown in FIG. 4(H), when, for example, the plus button is operated twice and then the decision button is operated. When the decision button or cancel button is arranged on the small screen SG10, the mobile terminal 60 may simply erase the small screen SG10 and display the registration screen of FIG. 3(H) when the cancel button is operated, regardless of whether the product cancellation button, minus button, or plus button is operated.

[0133] Fig. 5(A) is a display example of a scene following Fig. 4(H). When the mobile terminal 60 recognizes that a product (an increase in the quantity of whitebait) has been put into the shopping basket (the upper basket in this example), it displays a small screen SG1 in front of the registration screen of Fig. 4(H), as shown in Fig. 5(A).

[0134] Fig. 5(B) is a display example of a scene following Fig. 5(A). When a predetermined time has elapsed since the start of displaying the small screen SG1, the mobile terminal 60 erases the small screen SG1 as shown in Fig. 5(B).

[0135] 5(A) shows an example in which an increase in the quantity of whitebait is put into a shopping basket (upper basket) that already has whitebait in it, but if an increase in the quantity of whitebait is put into a shopping basket (lower basket) different from the shopping basket in which the whitebait has been put, the mobile terminal 60 displays a small screen SG2 in front of the registration screen in FIG. 4(H) as shown in FIG. 5(C). When a predetermined time has elapsed since the start of displaying the small screen SG2, the mobile terminal 60 erases the small screen SG2 as shown in FIG. 5(D).

[0136] In Figures 5(A) and 5(B), as with the registration screen in Figure 4(H), "upper basket" is displayed for shirasu, and in Figures 5(C) and 5(D), "upper and lower baskets" are displayed, meaning "upper basket" and "lower basket."

[0137] In addition, when the same product is placed in the upper basket and the lower basket separately, the upper basket portion and the lower basket portion may be displayed separately instead of displaying "upper and lower baskets" as in Figures 5(C) and 5(D). For example, in Figures 5(A) and 5(B), the products are displayed together in one line as in Display Mode A below, but they may also be displayed using two lines as in Display Mode B, or using three lines as in Display Mode C (or Display Mode D). (Display mode A) 1st row: Shirasu upper and lower basket × 3 ￥840 (Display mode B) 1st row: Whitebait Upper basket x 1 ￥280 2nd row: Shirasu bottom basket x 2 ￥560 (Display mode C) 1st row: Whitebait Upper basket x 1 2nd row: Shirasu bottom basket x 2 3rd line: Total x3 ￥840 (Display mode D) 1st row: Shirasu x 3 ￥840 2nd row: (Top basket x 2) 3rd row: (lower basket x 3)

[0138] Figures 5(E) to 5(H) are screen examples of yet another scene (another scene 2). Figure 5(E) is a display example of another scene following Figure 3(H).

[0139] When the button registration button BT623 on the registration screen of Fig. 3(H) is operated, the mobile terminal 60 displays a registration screen (button registration screen) as shown in Fig. 5(E). The registration screen of Fig. 5(E) is provided with a registration method selection button display area HR621, a tab display area HR640, a registration information display area HR630, a product button display area HR660, etc. A search button BT625 is displayed to the right of the registration method selection button display area HR621.

[0140] On the registration screen of FIG. 5(E), the scan registration button BT622 is displayed in a dashed line, and the button registration button BT623 is displayed in a thick line (button registration is selected).

[0141] When the search button BT625 is operated, the mobile terminal 60 accepts character input for searching for a product button. For example, when the search button BT625 is operated, the mobile terminal 60 displays a character input area and accepts character input from the customer in the character input area.

[0142] The tab display area HR640 is an area that displays five selectable (switchable) tabs (all tab, fruit tab, vegetable tab, fresh fish tab, prepared food tab). The four tabs other than the all tab (fruit tab, vegetable tab, fresh fish tab, prepared food tab) correspond to the four categories (fruit, vegetable, fresh fish, prepared food) in which products to be registered by button registration exist. The all tab corresponds to all of the above four categories. In the registration screen of Figure 5 (E), the all tab is selected. The underline on the all tab indicates that the all tab is selected.

[0143] As described above, the registration information display area HR630 is an area that mainly displays the registration information of a product. The registration information display area HR630 on the registration screen of Fig. 5(E) displays the registration information of the product (whitebait) that was scanned and registered.

[0144] The product button display area HR660 is an area that displays product buttons for products that belong to the category corresponding to the tab selected (touched) in the tab display area HR640. In the registration screen of Fig. 5(E), the All tab is selected, so all product buttons (product buttons for products belonging to Fruits, Vegetables, Fresh Fish, and Prepared Foods) are displayed in the product button display area HR660.

[0145] If a customer selects the Fruits tab in the tab display area HR640, the mobile terminal 60 displays in the tab display area HR640 an indication that the Fruits tab has been selected (by underlining in this example), and displays product buttons for products belonging to Fruits in the product button display area HR660. For example, if a customer selects the Fruits tab in the tab display area HR640, the mobile terminal 60 identifies the category (fruits) corresponding to the Fruits tab. Next, the mobile terminal 60 references the product button file stored in the cloud server 20, and displays product buttons corresponding to the category identification information that identifies fruits (i.e., product buttons for products belonging to Fruits) in the product button display area HR660. The same applies if a customer selects the Vegetables tab, Fresh Fish tab, or Prepared Food tab.

[0146] The mobile terminal 60 may acquire the product button file together with the app A when acquiring the app A, or may acquire the product button file as part of the app A when acquiring the app A and store it in the storage unit. In this case, the mobile terminal 60 refers to the product button file stored therein and displays the product button in the product button display area HR660.

[0147] Fig. 5(F) is a display example of a scene following Fig. 5(E). When the pear product button (pear button) is operated on the registration screen of Fig. 5(E), the mobile terminal 60 displays a registration screen (button registration screen) as shown in Fig. 5(F). The registration information display area HR630 on the registration screen of Fig. 5(F) displays the registration information of the scan-registered product (whitebait) and the registration information of the button-registered product (pear). In other words, the mobile terminal 60 obtains product button identification information for the pear button based on the operation of the pear button, references the product button file stored by the cloud server 20 (or the product button file stored by the mobile terminal 60 itself) to obtain the product code (pear product code) corresponding to the product button identification information, and registers the pear.

[0148] When the pear button is operated on the registration screen of Fig. 5(E), the mobile terminal 60 may display a small screen (not shown) on top of the registration screen of Fig. 5(F) on which the purchase quantity can be specified, and when an operation to determine the purchase quantity is performed on the small screen, the small screen may be cleared and the registration screen as shown in Fig. 5(F) may be displayed. In other words, in the case of button registration, the purchase quantity may be confirmed with the customer immediately after operating the product button.

[0149] Fig. 5(G) is a display example of a scene following Fig. 5(F). When the mobile terminal 60 recognizes that a product (pear) has been placed in the shopping basket (in this example, the upper basket), it displays the "upper basket" in association with the product (pear) and also displays the small screen SG1 on the front.

[0150] Fig. 5(H) is a display example of a scene following Fig. 5(G). When a predetermined time has elapsed since the start of displaying the small screen SG1, the mobile terminal 60 erases the small screen SG1 as shown in Fig. 5(G).

[0151] 6(A) to 6(C) are screen examples of yet another scene (another scene 3). FIG. 6(A) is a display example of another scene following FIG. 4(D).

[0152] When the whitebait quantity button BT632 on the registration screen of Fig. 4(D) is operated, the mobile terminal 60 displays a small screen SG10 in front of the registration screen of Fig. 4(D), although this is not shown in the figure. The small screen SG10 is a screen that accepts operations for canceling a product or changing the quantity (increasing or decreasing the quantity), as shown in Fig. 4(F). When the product cancellation button on the small screen SG10 is operated and then an operation is performed in an area other than the small screen SG10 (any area on the back), the mobile terminal 60 erases the small screen SG10 and displays a registration screen that reflects the product cancellation operation, as shown in Fig. 6(A).

[0153] In the registration information display area HR630 of the registration screen in FIG. 6(A), a message is displayed indicating that the product (whitebait) subject to the product cancellation operation has been cancelled (the word "Cancelled" and a single line).

[0154] Fig. 6(B) is an example of a display following Fig. 6(A). When the mobile terminal 60 recognizes that a product (whitebait) has been removed from the shopping basket (the upper basket in this example), it erases the "upper basket" that was displayed in association with the product (whitebait) and displays a small screen SG3 in the foreground, as shown in Fig. 6(B). The small screen SG3 is a small screen that notifies the user that a product has been removed from the upper basket. A message such as "Removed from the upper basket" is displayed on the small screen SG3.

[0155] Fig. 6(C) is a display example of a scene following Fig. 6(B). When a predetermined time (for example, 2 seconds) has elapsed since the start of displaying the small screen SG3, the mobile terminal 60 erases the small screen SG3 as shown in Fig. 6(C).

[0156] 6(D) to 6(F) are screen examples of yet another scene (another scene 4). FIG. 6(D) is a display example of another scene following FIG. 4(D).

[0157] When the bread quantity button BT632 on the registration screen of Fig. 4(D) is operated, the mobile terminal 60 displays a small screen SG10 in front of the registration screen of Fig. 4(D), although this is not shown in the figure. When the product cancellation button on the small screen SG10 is operated and then an operation is performed in an area other than the small screen SG10, the mobile terminal 60 erases the small screen SG10 and displays a registration screen that reflects the product cancellation operation, as shown in Fig. 6(D).

[0158] In the registration information display area HR630 of the registration screen in FIG. 6(D), a message is displayed indicating that the product (bread) subject to the product cancellation operation has been cancelled (the word "Cancelled" and a single line).

[0159] Fig. 6(E) is an example of a display following Fig. 6(D). When the mobile terminal 60 recognizes that a product (loaf of bread) has been removed from the shopping basket (the lower basket in this example), it erases the "lower basket" that was displayed in association with the product (loaf of bread) and displays small screen SG4 in the foreground, as shown in Fig. 6(E). Small screen SG4 is a small screen that notifies the user that a product has been removed from the lower basket. Small screen SG4 displays a message such as "Removed from the lower basket."

[0160] Fig. 6(F) is a display example of a scene following Fig. 6(E). When a predetermined time (for example, 2 seconds) has elapsed since the start of displaying the small screen SG4, the mobile terminal 60 erases the small screen SG4 as shown in Fig. 6(F).

[0161] Figures 6(G) and 6(H) are screen examples of yet another scene (another scene 5). Figure 6(G) is a display example of another scene following Figure 5(B).

[0162] When the quantity button BT632 on the registration screen of Fig. 5(B) is operated, the mobile terminal 60 displays a small screen SG10 in front of the registration screen of Fig. 5(B), although illustration is omitted. When the minus button on the small screen SG10 is operated once (an operation to reduce the purchase quantity from 3 to 2) and then an operation is performed in an area other than the small screen SG10, the mobile terminal 60 erases the small screen SG10 and displays a registration screen reflecting the product whose quantity has been reduced, although illustration is omitted. Next, when the mobile terminal 60 recognizes the removal of a product (whitebait) from the shopping basket (the upper basket in this example), it displays a small screen SG3 in front, as shown in Fig. 6(G).

[0163] Fig. 6(H) is a display example of a scene following Fig. 6(G). When a predetermined time has elapsed since the start of displaying the small screen SG3, the mobile terminal 60 erases the small screen SG3 as shown in Fig. 6(H).

[0164] Fig. 7 is a flowchart illustrating an example of the operation of the mobile terminal 60. Specifically, the flowchart in Fig. 7 shows the operation of the product insertion / removal recognition function (recognizing product insertion and product removal). Note that when the flowchart in Fig. 7 is displayed, it is assumed that the mobile terminal 60 is already connected (Bluetooth connection) to the cart 70, and both the upper basket and the lower basket are placed on the cart 70.

[0165] Step S1: The mobile terminal 60 determines whether the weight value has increased. For example, if the weight value (current weight value) received as weight information from the cart 70 is greater than the weight value (previous weight value) stored in the mobile terminal 60's memory, the mobile terminal 60 determines that the weight value has increased. The mobile terminal 60 may determine that the weight value has increased if it receives a weight increase notification (or an increase in the weight value) from the cart 70. If the weight value has increased (step S1: YES), the process proceeds to step S10. If the weight value has not increased (step S1: NO), the process proceeds to step S2.

[0166] Step S2: The mobile terminal 60 determines whether the weight value has decreased. For example, if the weight value (current weight value) received as weight information from the cart 70 is smaller than the weight value (previous weight value) stored in the mobile terminal 60's memory, the mobile terminal 60 determines that the weight value has decreased. The mobile terminal 60 may also determine that the weight value has decreased if it receives a weight decrease notification (or the amount of decrease in the weight value) from the cart 70. If the weight value has decreased (step S2: YES), the process proceeds to step S20. If the weight value has not decreased (step S2: NO), the process returns to step S1.

[0167] Step S10: The mobile terminal 60 determines whether the captured image has changed. If the captured image has changed (Step S10: YES), the process proceeds to Step S11. If the captured image has not changed (Step S10: NO), the process proceeds to Step S13.

[0168] Step S11: The mobile terminal 60 determines (recognizes) that a product has been put into the upper basket. Then, the process proceeds to step S12. Step S12: The mobile terminal 60 notifies the user that the product has been placed in the upper basket. For example, the mobile terminal 60 displays a pop-up small screen SG1. Then, this flowchart ends.

[0169] Step S13: The mobile terminal 60 determines that a product has been put into the lower basket. Then, the process proceeds to step S14. Step S14: The mobile terminal 60 notifies the user that the product has been put into the lower basket. For example, the mobile terminal 60 displays a pop-up small screen SG2. Then, this flowchart ends.

[0170] Step S20: The mobile terminal 60 determines whether the captured image has changed. If the captured image has changed (Step S20: YES), the process proceeds to Step S21. If the captured image has not changed (Step S20: NO), the process proceeds to Step S23.

[0171] Step S21: The mobile terminal 60 determines that a product has been removed from the upper basket, and then proceeds to step S22. Step S22: The mobile terminal 60 notifies the user that the product has been removed from the upper basket. For example, the mobile terminal 60 displays a pop-up small screen SG3. Then, this flowchart ends.

[0172] Step S23: The mobile terminal 60 determines that a product has been removed from the lower basket, and then proceeds to step S24. Step S24: The mobile terminal 60 notifies the user that the product has been removed from the lower basket. For example, the mobile terminal 60 displays a pop-up small screen SG4. Then, this flowchart ends.

[0173] FIG. 8 is an explanatory diagram for explaining fraud (fraud and mistakes). In each of FIG. 8(A) to FIG. 8(D), left to right indicates the flow of time. In FIG. 8(A) to FIG. 8(D), "Registered" indicates that a product has been registered (that is, a scan registration or a button registration has been made). "No Registration" indicates that no product has been registered (that is, neither a scan registration nor a button registration has been made). "Cancelled" indicates that a product has been canceled. "No Cancellation" indicates that no product has been canceled. "Added (Upper Basket)" indicates that a product has been added to the upper basket. "Removal (upper basket)" indicates that a product has been removed from the upper basket.

[0174] FIG. 8(A) is an explanatory diagram illustrating a scene of an unregistered input (forgotten registration). When the mobile terminal 60 recognizes that an item has been input to the upper basket without being registered, it displays the small screen SG1. As described above, the small screen SG1 is a small screen that notifies the user that an item has been input to the upper basket. The mobile terminal 60 erases the small screen SG1 after a predetermined time (for example, two seconds after the small screen SG1 starts to be displayed). When no item is registered within a predetermined time (for example, eight seconds after the mobile terminal 60 recognizes that an item has been input to the upper basket), the mobile terminal 60 determines that an unregistered input (upper basket) has occurred and displays the small screen SG21. The small screen SG21 is a small screen that notifies the user that an unregistered input (upper basket). The small screen SG21 (similar to the small screen SG22 and the small screen SG20) is also a small screen that instructs the user to register an item. The small screen SG21 displays a message such as "≪Unregistered Input (upper basket)≫ Please register the item placed in the upper basket."

[0175] Although not shown in the figures, the mobile terminal 60 displays small screen SG2 when it recognizes that a product has been put into the lower basket without being registered. As described above, small screen SG2 is a small screen that notifies the user that a product has been put into the lower basket. The mobile terminal 60 erases small screen SG2 after a predetermined time has elapsed (for example, two seconds from the start of displaying small screen SG2). If no product is registered within a predetermined time (for example, eight seconds from the recognition of the product being put into the lower basket), the mobile terminal 60 determines that an unregistered put-in (lower basket) has occurred, and displays small screen SG22 (not shown). Small screen SG22 is a small screen that notifies the user of an unregistered put-in (lower basket). Small screen SG22 displays a message such as "≪Unregistered put-in (lower basket)≫ Please register the product placed in the lower basket."

[0176] The mobile terminal 60 may display a common small screen SG20 (not shown) that does not specify the upper basket or the lower basket, instead of the small screen SG21 that specifies the upper basket or the small screen SG22 (not shown) that specifies the lower basket. The small screen SG20 displays a message such as "Please register the product(s) you have added to the basket (not registered)."

[0177] FIG. 8(B) is an explanatory diagram illustrating a scene in which an item has not been registered (forgotten to be inserted). If the mobile terminal 60 does not recognize the insertion of an item within a predetermined time after the item has been registered (for example, 8 seconds after the item has been registered), it determines that an item has not been registered and displays the small screen SG30. The small screen SG30 is a small screen that notifies the user that an item has not been registered. The small screen SG30 also instructs the user to insert an item. The small screen SG30 displays a message such as "Please place the registered item <<Not registered>> in the basket." If the mobile terminal 60 recognizes that an item has been inserted into the upper basket, it displays the small screen SG1 in place of the small screen SG30. If a predetermined time has passed (for example, 2 seconds after the small screen SG1 begins to be displayed), the mobile terminal 60 erases the small screen SG1.

[0178] FIG. 8(C) is an explanatory diagram illustrating a scene of an uncancelled removal (forgotten to cancel). When the mobile terminal 60 recognizes that a product has been removed from the upper basket without canceling the product, it displays the small screen SG3. As described above, the small screen SG3 is a small screen that notifies the user that a product has been removed from the upper basket. The mobile terminal 60 erases the small screen SG3 after a predetermined time (e.g., two seconds after the small screen SG3 starts to be displayed). When no product is canceled within a predetermined time (e.g., eight seconds after the mobile terminal 60 recognizes that a product has been removed from the upper basket), the mobile terminal 60 determines that an uncancelled removal (upper basket) has occurred and displays the small screen SG41. The small screen SG41 is a small screen that notifies the user that a product has been removed (upper basket). The small screen SG41 (similar to the small screen SG42 and the small screen SG40) also instructs the user to cancel the product. The small screen SG41 displays a message such as "≪Uncancelled removal (upper basket)≫ Please cancel the product removed from the upper basket."

[0179] Although not shown in the figures, the mobile terminal 60 displays a small screen SG4 when it recognizes that a product has been removed from the lower basket without canceling the product. As described above, the small screen SG4 is a small screen that notifies the user that a product has been removed from the lower basket. The mobile terminal 60 erases the small screen SG4 after a predetermined time has elapsed (for example, two seconds after the display of the small screen SG4 begins). If no product is canceled within a predetermined time (for example, eight seconds after the mobile terminal 60 recognizes that a product has been removed from the lower basket), the mobile terminal 60 determines that there has been an uncancelled removal (lower basket) and displays a small screen SG42 (not shown). The small screen SG42 is a small screen that notifies the user of an uncancelled removal (lower basket). The small screen SG42 displays a message such as "≪Uncancelled removal (lower basket)≫ Please cancel the product removed from the lower basket."

[0180] The mobile terminal 60 may display a common small screen SG40 (not shown) that does not specify the upper basket or the lower basket, instead of the small screen SG41 that specifies the upper basket or the small screen SG42 (not shown) that specifies the lower basket. The small screen SG40 displays a message such as "Please cancel the items removed from the basket (<Uncancelled removal>)."

[0181] FIG. 8(D) is an explanatory diagram illustrating a scene in which a product is not removed but canceled (forgotten to be removed). If the mobile terminal 60 does not recognize that a product has been removed within a predetermined time after the product cancellation (for example, 8 seconds after the product cancellation), it determines that a product has been removed but canceled, and displays the small screen SG50. The small screen SG50 is a small screen that notifies the user that a product has been removed but canceled. The small screen SG50 also instructs the user to remove the product. The small screen SG50 displays a message such as "≪Not canceled removed≫ Please remove the canceled product from the basket." If the mobile terminal 60 recognizes that a product has been removed from the upper basket, it displays the small screen SG3 instead of the small screen SG50. If a predetermined time (for example, 2 seconds after the small screen SG3 starts to be displayed) has elapsed, the mobile terminal 60 erases the small screen SG3.

[0182] 9 is an example of a display on the settlement device 50. The settlement device 50 displays a product list screen after reading the settlement code GA641.

[0183] Figure 9(A) is an example of the product list screen that appears when proceeding to checkout from the registration screen in Figure 4(D) (a registration screen that has transitioned from whitebait registered → whitebait placed in the upper basket → bread registered → bread placed in the lower basket).

[0184] When the checkout instruction button BT631 is operated on the registration screen of FIG. 4(D), the mobile terminal 60 displays the settlement code display screen (FIG. 4(E)). The settlement device 50 reads the settlement code GA641 from the settlement code display screen displayed by the mobile terminal 60. Having read the settlement code GA641 from the settlement code display screen displayed by the mobile terminal 60, the settlement device 50 sends transaction information acquisition request information to the cloud server 20, including transaction identification information (transaction identification information of the mobile terminal 60) acquired from the settlement code GA641, receives the transaction information of the mobile terminal 60 from the cloud server 20, and displays the product list screen as shown in FIG. 9(A). The flow up to displaying the product list screen is the same in FIGS. 9(B) to 9(D).

[0185] The product list screen in Figure 9(A) displays the registered information for whitebait, the registered information for bread, the total price, etc. In addition, the "upper basket" is displayed in association with whitebait, and the "lower basket" is displayed in association with bread.

[0186] 9(A), the transaction information of the mobile terminal 60 stored by the cloud server 20 includes, for example, "registration information for whitebait" and "information indicating that whitebait has been placed in the upper basket" as information regarding the first product, and "registration information for bread" and "information indicating that whitebait has been placed in the lower basket" as information regarding the second product. The settlement device 50 displays the product list screen of FIG. 9(A) based on the transaction information stored in the cloud server 20.

[0187] Figure 9(B) shows an example of the product list screen that appears when proceeding to checkout from the registration screen of Figure 6(D) (a registration screen that has transitioned from whitebait registered → whitebait placed in upper basket → bread registered → bread placed in lower basket → bread canceled → bread removed from lower basket).

[0188] The product list screen of FIG. 9(B) displays a message indicating that the bread item has been cancelled (the wording "Cancelled" and a single line).

[0189] 9(B), the transaction information of the mobile terminal 60 stored by the cloud server 20 includes, for example, information about the first product, such as "registration information for whitebait" and "information that whitebait has been placed in the upper basket," and information about the second product, such as "registration information for bread," "information that whitebait has been placed in the lower basket," "cancellation information for bread," and "information that whitebait has been removed from the lower basket." The settlement device 50 displays the product list screen of FIG. 9(B) based on the transaction information stored in the cloud server 20.

[0190] Figure 9(C) is another example of the product list screen display when the second product (bread) added in Figure 9(A) has not been registered (in the case of an unregistered input). In other words, it is an example of the product list screen display when proceeding to checkout from a registration screen where it has been determined that an unregistered input has occurred (a registration screen that has transitioned from whitebait registered → whitebait added to the upper basket → whitebait added to the lower basket).

[0191] The product list screen in FIG. 9(C) displays information that the two products have been determined to be unregistered inputs (the message "Unregistered Input") and that the user should check the lower basket where the products were input (the message "Check Lower Basket"), and also includes an image button (the button marked C in the figure). When the image button is operated (basically by a store clerk), a captured image of the product related to the unregistered input (the unregistered product) is displayed (for example, a captured image of the scene in which the unregistered product is being registered is obtained from the cloud server 20 and displayed). When a purchase is made, the settlement device 50 registers the unregistered product (for example, a loaf of bread) based on, for example, the operation of the store clerk (it is not necessary to purchase the product).

[0192] 9(C), the transaction information of the mobile terminal 60 stored by the cloud server 20 includes, for example, "registration information for whitebait" and "information indicating that whitebait has been placed in the upper basket" as information regarding the first product, and "information indicating that whitebait has been placed in the lower basket" and "information indicating that whitebait has been placed unregistered" as information regarding the second product. The settlement device 50 displays the product list screen of FIG. 9(C) based on the transaction information stored in the cloud server 20.

[0193] Figure 9(D) is another example of the product list screen display when the second item (bread) canceled in Figure 9(B) has not been taken out (cancellation not taken out). In other words, it is an example of the product list screen display when proceeding to checkout from a registration screen where it has been determined that there is a cancellation not taken out (a registration screen that has transitioned in the following manner: whitebait registered → whitebait placed in upper basket → bread registered → bread placed in lower basket → bread canceled).

[0194] The product list screen in Figure 9(D) displays that the second item (bread) has been determined to be canceled and not removed (the words "Cancelled and not removed") and that the lower basket that has not been removed should be checked (the words "Check lower basket").

[0195] 9(D), the transaction information of the mobile terminal 60 stored by the cloud server 20 includes, for example, information about the first product, such as "registration information for whitebait" and "information that whitebait has been placed in the upper basket," and information about the second product, such as "registration information for bread," "information that whitebait has been placed in the lower basket," "cancellation information for bread," and "information that whitebait has not been removed." The settlement device 50 displays the product list screen of FIG. 9(D) based on the transaction information stored in the cloud server 20.

[0196] When the checkout button BT90 on the product list screen is operated, the settlement device 50 displays, for example, as the next screen, a payment method selection screen (not shown) on which a payment method (cash, credit card, etc.) can be selected. If fraud or the like is present, staff intervention (confirmation, etc.) is required (to proceed with the process after the staff intervention). Therefore, on the product list screen ( FIG. 9(C) ) on which an unregistered input exists or on the product list screen ( FIG. 9(D) ) on which a canceled and unremoved item exists, the operation of the checkout button BT90 is disabled (indicated by a dashed line in the figure) so that the process does not proceed (the operation of the checkout button BT90 is enabled (indicated by a solid line in the figure) after the staff intervention). Note that even on the product list screen ( FIG. 9(B) ) on which a cancellation exists, the operation of the checkout button BT90 may be disabled so that the process proceeds after the staff intervention.

[0197] Although omitted in Figure 9, the settlement device 50 may display products that have been placed in the upper and lower baskets together as "upper and lower baskets" (display mode A), similar to the mobile terminal 60, or may display them separately as "upper basket" and "lower basket" (display modes B to D).

[0198] In addition, in FIG. 9, the products are arranged in the order in which they were registered, with the destination indicated, but they may also be displayed by destination, as shown below.

[0199] (Tab view) An "All" tab, an "Upper Basket" tab, and a "Lower Basket" tab may be provided to accept a switching operation (selection operation) of the display content, and the checkout device 50 may display products according to the selected tab. The "All" tab is the initial tab (the tab selected to start displaying the product list screen). When the "All" tab is selected, the checkout device 50 displays all products (displaying both products placed in the upper basket and products placed in the lower basket). The "Upper Basket" tab is a tab for displaying products placed in the upper basket. When the "Upper Basket" tab is selected, the checkout device 50 displays products placed in the upper basket (products placed in the lower basket are not displayed). The "Lower Basket" tab is a tab for displaying products placed in the lower basket. When the "Lower Basket" tab is selected, the checkout device 50 displays products placed in the lower basket (products placed in the upper basket are not displayed). In addition, a "Upper and Lower Basket" tab may be provided, and when the "Upper and Lower Basket" tab is selected, the checkout device 50 may display the products that have been placed in the upper and lower baskets (products other than those that have been placed in the upper and lower baskets will not be displayed).

[0200] (Left and right display) The checkout device 50 displays the products placed in the upper basket in the left area (left column) and the products placed in the lower basket in the right area (right column). The checkout device 50 may combine the left and right areas to display products that are placed separately in the upper and lower baskets.

[0201] (displayed top and bottom) While the product list screen in Figure 9 is divided into a left area and a right area, the checkout device 50 may also display a product list screen that is divided into an upper area (upper row) and a lower area (lower row). The checkout device 50 displays the products that have been placed in the upper basket in the upper area, and displays the products that have been placed in the lower basket in the lower area. For products that have been placed in the upper and lower baskets separately, the checkout device 50 may provide a dedicated area between the upper and lower areas for displaying the products that have been placed in the upper and lower baskets, and display the products in that dedicated area.

[0202] (Display of targets for monitoring for fraud, etc.) Instead of or in addition to the above, the settlement device 50 may collectively display the canceled and not removed items among the four events (unregistered deposit, unregistered deposit, uncancelled withdrawal, and canceled and not removed) that are monitored for fraud, etc. For example, the settlement device 50 may display a forgotten removal button for transactions in which a canceled and not removed item exists, and when the forgotten removal button is operated, display the corresponding product (the canceled and not removed product) (displaying the product name, deposit destination, etc.). The same applies to the other three events.

[0203] A price range display area is provided between the list display area and the total display area on the product list screen, and price range buttons indicating the quantity of products in each price range are arranged in the price range display area. For example, the product list screen in FIG. 9(A) includes a button indicating that the quantity of products priced at ¥1,000 or more is zero, a button indicating that the quantity of products priced at ¥500 to ¥999 is zero, a button indicating that the quantity of products priced at ¥200 to ¥499 is two, and a button indicating that the quantity of products priced at less than ¥200 is zero. When a price range button is operated, the checkout device 50 displays products in that price range. By combining the price range buttons with the above-mentioned tab operation and operation of the forgotten item button, the store clerk (and the customer) can quickly (efficiently) check, for example, which luxury items have been added to the shopping basket and whether the canceled luxury items have been removed.

[0204] (Payment function for each purchase) The checkout device 50 may have a function for settling payments separately for each shopping basket (upper basket / lower basket). For example, when the checkout button BT90 is pressed, the checkout device 50 displays a message such as "Do you want to settling the items in the upper and lower baskets separately?" and displays a screen (which may be a small screen) with a Yes button and a No button. When the No button is pressed, the checkout device 50 performs the normal checkout (both the items in the upper basket and the items in the lower basket together). When the Yes button is pressed, the checkout device 50 starts the checkout for the items in the upper basket, and starts the checkout for the items in the lower basket after the checkout for the upper basket is completed (in other words, it controls the two checkouts so that they are performed consecutively, and issues receipts for each basket with a letter or mark printed on it indicating whether it is the upper basket or the lower basket). The checkout device 50 displays the above screen when two shopping baskets are used and at least one item is placed in each shopping basket. Customers may separate their shopping baskets when they are making large purchases, or when they wish to physically separate their own purchases from purchases made by others. In the latter case, the separate shopping basket checkout function is extremely convenient. Based on the customer's declaration (operation of the Yes button), a single transaction can be processed by actually separating it into two transactions (upper basket transaction and lower basket transaction), or it can be processed as a single transaction (internally distinguished, such as child transactions or branch numbers).

[0205] (Display on the transaction status management device 40, etc.) The transaction status management device 40 may also display the same content (information) as the settlement device 50 described above. Furthermore, a terminal (exit terminal) installed on the path from payment to leaving the store may also be configured to operate according to the status of the upper and lower shopping baskets (items inserted, settlement status). For example, an alert may be issued at the exit terminal if there is any fraud remaining (if payment is not yet made), but the alert may be issued in a manner that makes it possible to distinguish which basket has the problem. This allows for quick confirmation and response.

[0206] (Transmission and reception of information between the mobile terminal 60 and the cloud server 20, and storage of information) The transmission and reception of information between the mobile terminal 60 and the cloud server 20, and the storage of information may be realized by any of the following aspects (Aspects A to F), or by appropriately combining part of one aspect with part of another aspect.

[0207] <Aspect A> Each time the mobile terminal 60 acquires a product code (each time the product barcode is read or the product button is operated), it stores the product code in its storage unit and transmits the product code to the cloud server 20. Upon receiving the product code from the mobile terminal 60, the cloud server 20 references the product master, acquires product information (product name, price, etc.) corresponding to the product code, and transmits (responds to) the acquired product information to the mobile terminal 60. As described above, upon receiving each product code from the mobile terminal 60, the cloud server 20 transmits the product information acquired from the product code to the mobile terminal 60, but does not store the product code and the product information acquired from the product code in the transaction information of the mobile terminal 60. The above-mentioned portion of Aspect A also applies to Aspect B described below.

[0208] The mobile terminal 60 transmits the acquired product code to the cloud server 20 and receives product information from the cloud server 20. The mobile terminal 60 stores the product information in its storage unit in association with the product code as registration information for the product. When the mobile terminal 60 completes registration (when the checkout instruction button BT631 is operated), it simultaneously transmits the product codes of one or more products stored in its storage unit to the cloud server 20. When the cloud server 20 receives the product codes of one or more products from the mobile terminal 60, it references the product master, acquires product information corresponding to each product code, associates the product information acquired from each product code with each product code, and stores this as registration information for each product in the transaction information of the mobile terminal 60. In other words, the cloud server 20 does not store the registration information for each product of the mobile terminal 60 in the transaction information of the mobile terminal 60 each time the mobile terminal 60 acquires an individual product code. Instead, when the mobile terminal 60 completes product registration, it collectively stores the registration information for one or more products of the mobile terminal 60 in the transaction information of the mobile terminal 60. According to Aspect A (the same applies to Aspects B, D, and E), the cloud server 20 does not update the transaction information of each mobile terminal 60 (because registration information is not added, etc.) each time the mobile terminal 60 acquires a product code. This eliminates the processing load on the cloud server 20 (e.g., search processing for the target transaction information, writing information, etc.) that would be incurred if the transaction information of each mobile terminal 60 were updated. Furthermore, because there is no processing load as described above each time the mobile terminal 60 acquires a product code, the response (updating the registration screen, etc.) when the mobile terminal 60 acquires a product code is improved. Furthermore, even if the number of product codes acquired by the mobile terminal 60 by the time registration is completed is N (even if N products are registered in one transaction), only one search process (pre-processing before updating) is required to search for transaction information to be updated (i.e., the total lead time for one transaction is reduced to one time), which is efficient. In addition, according to the aspect A, compared to the aspect B, the amount of information transmitted and received between the mobile terminal 60 and the cloud server 20 after the mobile terminal 60 has completed registration can be reduced.

[0209] <Mode B> The mobile terminal 60 transmits the acquired product code to the cloud server 20 and receives product information from the cloud server 20. The mobile terminal 60 stores the product information in a storage unit in association with the product code as registration information for the product. When the mobile terminal 60 completes registration, the mobile terminal 60 simultaneously transmits the registration information for one or more products (one or more product codes, product information corresponding to each product code) stored in the storage unit to the cloud server 20. That is, in Aspect A, the mobile terminal 60 simultaneously transmits the product codes for one or more products to the cloud server 20 when it completes registration, whereas in Aspect B, the mobile terminal 60 simultaneously transmits the registration information for one or more products to the cloud server 20 when it completes registration. The cloud server 20, which receives the registration information for one or more products from the mobile terminal 60, stores the received registration information in the transaction information of the mobile terminal 60. According to Aspect B (similar to Aspect E), the cloud server 20 acquires the product registration information from each mobile terminal 60 when it completes registration, and therefore does not need to refer to the product master when each mobile terminal 60 completes registration. Therefore, in the aspect B, the load of the cloud server 20 referencing the product master and acquiring product information of each product when each mobile terminal 60 has completed registration is eliminated.

[0210] <Aspect C> Each time the mobile terminal 60 acquires a product code, it stores the product code in its storage unit and transmits the product code to the cloud server 20. Upon receiving the product code from the mobile terminal 60, the cloud server 20 references the product master, acquires product information corresponding to the product code, transmits the acquired product information to the mobile terminal 60, and stores registration information for the product in the transaction information of the mobile terminal 60. That is, in Aspects A and B, the cloud server 20 does not update the transaction information of the mobile terminal 60 each time a product code is acquired from the mobile terminal 60, but in Aspect C, the cloud server 20 updates the transaction information of the mobile terminal 60 each time a product code is acquired from the mobile terminal 60. According to Aspect C (and Aspect F as well), the mobile terminal 60 does not need to transmit the information stored in its storage unit (product codes of one or more products, registration information for one or more products) to the cloud server 20 when registration is completed, thereby reducing the amount of communication between the mobile terminal 60 and the cloud server 20 when registration is completed. Furthermore, according to aspect C (similarly to aspect F), when the mobile terminal 60 has completed registration, the cloud server 20 does not incur the load of referencing the product master based on the product code acquired from the mobile terminal 60 to acquire product information and storing the acquired product information in the transaction information (the load in the case of aspect A), nor the load of storing the product information acquired from the mobile terminal 60 in the transaction information (the load in the case of aspect B).

[0211] <Mode D> The mobile terminal 60 receives the product master from another device (for example, the cloud server 20) and stores it in the storage unit. Every time the mobile terminal 60 acquires a product code, it references the product master stored in the storage unit to acquire product information for the product code, and stores the product information in the storage unit in association with the product code as registration information for the product. The above-mentioned part of aspect D also applies to aspect E below.

[0212] When the mobile terminal 60 has completed registration, it transmits the product codes of one or more products stored in the storage unit to the cloud server 20 all at once (hereinafter, the same as in aspect A). That is, in aspect A, the cloud server 20 references the product master, but in aspect D, the mobile terminal 60 references the product master, which is different. According to aspect D (as well as aspect E), the mobile terminal 60 does not need to transmit a product code to the cloud server 20 every time it acquires a product code. Furthermore, according to aspect D (as well as aspect E), the cloud server 20 references the product master to acquire product information every time the mobile terminal 60 acquires a product code, and there is no burden of transmitting the acquired product information to the mobile terminal 60.

[0213] <Aspect E> When the mobile terminal 60 has completed registration, it transmits the registration information of one or more products stored in the storage unit to the cloud server 20 all at once (hereinafter, the same as in aspect B). That is, in aspect B, the cloud server 20 references the product master, but in aspect E, the mobile terminal 60 references the product master. Alternatively, in aspect D, the mobile terminal 60 transmits the product codes of one or more products all at once to the cloud server 20 when it has completed registration, but in aspect E, the mobile terminal 60 transmits the registration information of one or more products all at once to the cloud server 20 when it has completed registration.

[0214] <Aspect F> The mobile terminal 60 receives the product master from another device and stores it in the storage unit. Every time the mobile terminal 60 acquires a product code, it references the product master stored in the storage unit to acquire product information for the product code, stores the product information in the storage unit in association with the product code as registration information for the product, and transmits the product information to the cloud server 20. The cloud server 20 stores the product information received from the mobile terminal 60 as registration information in the transaction information of the mobile terminal 60. In other words, the difference is that in aspect C, the cloud server 20 references the product master, but in aspect F, the mobile terminal 60 references the product master.

[0215] It has been explained that cloud server 20 may be composed of multiple servers, but it may also be composed of, for example, cloud server 20A that stores product master data and cloud server 20B that stores transaction information for each mobile terminal 60.

[0216] In the case of the cloud server 20A and the cloud server 20B, in aspects A to C, each time the mobile terminal 60 acquires a product code, it transmits the product code to the cloud server 20A, and the cloud server 20A, having received the product code, acquires product information by referring to the product master. Note that in aspects A to C, each time the mobile terminal 60 acquires a product code, it may transmit the product code to the cloud server 20B, and the cloud server 20B, having received the product code, may inquire of the cloud server 20A about product information corresponding to the product code.

[0217] In the case of the cloud server 20A and the cloud server 20B, in aspect A (similar to aspect D), the mobile terminal 60 that has completed registration simultaneously transmits the product codes of one or more products stored in the storage unit to the cloud server 20A, the cloud server 20A acquires product information for each product by referring to the product master and transmits (supplies) each product information together with each product code to the cloud server 20B, and the cloud server 20B stores the product information as registration information for each product in the transaction information of the mobile terminal 60. Note that in aspect A (similar to aspect D), the mobile terminal 60 that has completed registration simultaneously transmits the product codes of one or more products stored in the storage unit to the cloud server 20B, and the cloud server 20B may inquire of the cloud server 20A about product information corresponding to each product code, and store the product information for each product acquired by inquiring of the cloud server 20A together with each product code in the transaction information of the mobile terminal 60 as registration information for each product.

[0218] In the case of the cloud server 20A and the cloud server 20B, in aspect B (similar to aspect E), the mobile terminal 60 that has completed registration may simultaneously transmit the registration information of one or more products stored in the storage unit to the cloud server 20B, and the cloud server 20B may store it in the transaction information of the mobile terminal 60. Note that in aspect B (similar to aspect E), the mobile terminal 60 that has completed registration may simultaneously transmit the registration information of one or more products stored in the storage unit to the cloud server 20A, and the cloud server 20A may transmit (transfer) the registration information of the one or more products to the cloud server 20B, and the cloud server 20B may store it in the transaction information of the mobile terminal 60.

[0219] In the case where the system is configured with cloud server 20A and cloud server 20B, in aspects D to F, mobile terminal 60 may receive the product master from cloud server 20A and store it in the storage unit.

[0220] In the case of the cloud server 20A and the cloud server 20B, in aspect F, each time a product code is acquired, the mobile terminal 60 refers to the product master stored in the storage unit to acquire product information for the product code, stores the product information in the storage unit in association with the product code as registration information for the product, and transmits the product information to the cloud server 20B. Note that in aspect F, each time a product code is acquired, the mobile terminal 60 refers to the product master stored in the storage unit to acquire product information for the product code, stores the product information in the storage unit in association with the product code as registration information for the product, and transmits the product information to the cloud server 20A, and the cloud server 20A transmits (transfers) the registration information for the one or more products to the cloud server 20B, and the cloud server 20B may store the registration information in the transaction information of the mobile terminal 60.

[0221] In addition, even in the case of an embodiment (Aspect A, Aspect B, Aspect D, Aspect E) in which the cloud server 20 collectively stores the registration information of one or more products when the mobile terminal 60 finishes registering the products, or an embodiment (Aspect C, Aspect F) in which the cloud server 20 stores the registration information of an individual product each time the mobile terminal 60 acquires the product code of that product, after the mobile terminal 60 finishes registering the products, the cloud server 20 stores the registration information of all products of that mobile terminal 60 in the transaction information of that mobile terminal 60, so the settlement device 50 can receive the registration information of all products from the cloud server 20 by reading the settlement code GA641 displayed after operating the accounting instruction button BT631.

[0222] (Status Update) When the mobile terminal 60 completes the product registration based on the operation of the settlement code GA641 and transitions the screen from the registration screen to the settlement code display screen, it sends information to the cloud server 20 that the screen has transitioned from the registration screen to the settlement code display screen, and the cloud server 20 updates the status in the transaction information of the mobile terminal 60 from the registration state to the settlement declaration state.

[0223] In a mode in which cloud server 20 collectively stores registration information for one or more products at the time when mobile terminal 60 finishes registering products, when mobile terminal 60 transitions the screen from the registration screen to the settlement code display screen, it transmits to cloud server 20 information indicating that the screen has transitioned from the registration screen to the settlement code display screen, along with the product codes of one or more products (or registration information for one or more products). Upon receiving the information, cloud server 20 collectively stores the registration information for one or more products in the transaction information of mobile terminal 60, and updates the status in the transaction information from a registration state to a settlement declaration state.

[0224] In addition, in a case where cloud server 20 collectively stores registration information for one or more products when mobile terminal 60 finishes registering the products, cloud server 20 may update the status by receiving product codes for one or more products (or registration information for one or more products). In other words, in a case where cloud server 20 collectively stores registration information for one or more products when mobile terminal 60 finishes registering the products, when mobile terminal 60 transitions the screen from the registration screen to the settlement code display screen, mobile terminal 60 does not need to transmit information to cloud server 20 that the screen has transitioned from the registration screen to the settlement code display screen.

[0225] In a mode in which cloud server 20 stores the registration information of an individual product each time mobile terminal 60 acquires the product code of that product, when mobile terminal 60 transitions the screen from the registration screen to the settlement code display screen, mobile terminal 60 transmits information to cloud server 20 that the screen has transitioned from the registration screen to the settlement code display screen. Upon receiving the information, cloud server 20 updates the status in the transaction information of mobile terminal 60 from the registration state to the settlement declaration state.

[0226] Whether the cloud server 20 collectively stores the registration information for one or more products when the mobile terminal 60 finishes registering the products, or whether the cloud server 20 stores the registration information for each product each time the mobile terminal 60 acquires the product code of that product, when the settlement device 50 reads the settlement code GA641 of a certain mobile terminal 60, the cloud server 20 may check the status of that mobile terminal 60 and, if it is in a settlement declaration state, send the registration information for all products to the settlement device 50. In order to check the status, problems that can arise, for example, when a captured image of the settlement code GA641 is read (such as sending registration information to the settlement device 50 when the state is in the process of returning from a settlement declaration state to a registration state) will no longer occur.

[0227] When the mobile terminal 60 transitions from the settlement code display screen to the registration screen, for example, by operating the Return to Shopping button BT649, the mobile terminal 60 transmits information to the cloud server 20 that the screen has transitioned from the settlement code display screen to the registration screen, and the cloud server 20, having received the information, updates the status in the transaction information of the mobile terminal 60 from the settlement declaration status to the registration status.

[0228] The above describes the embodiment, but the sales system 1 of the embodiment allows for more optimal operation of the shopping cart. For example, the sales system 1 recognizes and reports the putting of products into the upper basket, putting of products into the lower basket, and taking of products from the upper basket and the lower basket (see, for example, FIG. 7), so that the putting of products into and taking of products into the basket can be efficiently confirmed. Furthermore, in monitoring for fraudulent activity (unregistered putting, unregistered putting, uncancelled taking, uncancelled taking), the sales system 1 identifies and reports the shopping basket for unregistered putting or uncancelled taking (see, for example, FIGS. 8 and 9), so that fraud can be efficiently confirmed and any fraud that occurs can be efficiently dealt with.

[0229] If a shopping cart that can hold two shopping baskets only has one weighing unit common to both baskets, it is difficult to identify the shopping basket into which an item has been added / removed based on the weighing value alone, but sales system 1 uses captured images in addition to the weighing value, so it is possible to identify the shopping basket into which an item has been added / removed, even with one weighing unit. Although it is possible to identify the shopping basket into which an item has been added / removed by providing two weighing units, providing two weighing units increases costs and reduces storage efficiency when storing items stacked together (nesting). Because sales system 1 has only one weighing unit, it is possible to reduce costs and identify the shopping basket into which an item has been added / removed without reducing storage efficiency.

[0230] (Variations, etc.) Although each embodiment has been described above, the device configuration, data configuration, processing flow, display and output modes, etc. can be appropriately changed as shown in the following (1) to (24). In addition, the following (1) to (24) can be appropriately combined with each other.

[0231] (1) In the above embodiment, when an item is added, the mobile terminal 60 notifies the shopping basket into which the item has been added (the shopping basket into which the item has been added) on the small screen (small screen SG1, small screen SG2) and also in the registration information display area HR630 ("upper basket", "lower basket"). However, the shopping basket into which the item has been added may be notified on either the small screen or the registration information display area HR630. Some customers may find the display on the small screen after adding an item annoying, but stopping the display on the small screen after adding an item will prevent this from being annoying. Also, the display of "upper basket" and "lower basket" may make the registration information display area HR630 look cluttered, but stopping the display of "upper basket" and "lower basket" will simplify the registration information display area HR630.

[0232] (2) In the above embodiment, the mobile terminal 60 notifies both the upper basket and the lower basket as the destination shopping basket. However, the mobile terminal 60 may notify only the upper basket or the lower basket as the destination shopping basket. For example, if the usual destination is the upper basket, the mobile terminal 60 may not notify the insertion of products into the upper basket, but may notify the insertion of products into the lower basket. By stopping the notification of one of the two, the amount of information to be notified can be suitably limited.

[0233] (3) In the above embodiment, the mobile terminal 60 does not erase the destination of the product (i.e., the source of the canceled product) when the product is canceled, but erases the destination of the product when the product is removed (i.e., the destination continues to be displayed from the time of cancellation to the time of removal) (FIG. 6(A)). However, the destination of the product may also be erased when the product is canceled. However, it is preferable to erase the destination after removal rather than when canceling, because this allows the user to know from which shopping basket the product is to be removed.

[0234] (4) In the above embodiment, the mobile terminal 60 erases the destination of a product when the product is removed after being cancelled (Figure 6(B)). However, when a product is cancelled, the destination of the product may continue to be displayed regardless of whether the product is removed or not (i.e., regardless of whether the product is in the shopping cart or not).

[0235] (5) In the above embodiment, when the mobile terminal 60 recognizes (determines) that an item has been inserted without registration, it displays a small screen (small screen SG20, SG21, SG22) that notifies the user of the non-registered insertion and instructs the user to register the item. However, a store clerk call button may be displayed so that it can be operated on the small screen. The same applies to the small screen (small screen SG30) that notifies the user of an unregistered insertion and instructs the user to insert the item, the small screen (small screen SG40, SG41, SG42) that notifies the user of an uncancelled removal and instructs the user to cancel the item, and the small screen (small screen SG50) that notifies the user of an uncancelled removal and instructs the user to remove the item.

[0236] When the store clerk call button on the small screen described above is operated, the mobile terminal 60 may send a call notification to a device used by the store clerk (e.g., the transaction status management device 40, a mobile terminal for the store clerk (smartphone, tablet terminal; not shown)), for example, via the cloud server 20.

[0237] (6) In the above embodiment, the mobile terminal 60 reports fraud, etc. on the small screens (small screens SG20, SG21, SG22, SG30, SG40, SG41, SG42, SG50). However, instead of or in addition to the small screens, the report may be made on the registration screen (registered information display area HR630). In other words, the settlement device 50 reports fraud, etc. on the product list screen (FIGS. 9(C) and 9(D)), but the mobile terminal 60 may also report fraud, etc. on the registration screen.

[0238] (7) Although not explained in the above embodiment, the destination of the product may be notified before it is inserted. For example, when an operation to increase the quantity of a certain product is performed, the shopping basket into which the product (the product to be increased in quantity) has been inserted may be notified as the destination of the product, so that multiple identical products are grouped together in the same shopping basket. For example, a small screen (not shown) displaying a message such as "Please insert into the upper basket" may be displayed on the registration screen of FIG. 4(H). The same applies when an identical product to an already registered product is further registered (by scan registration or button registration). In other words, the destination of the product may be notified (suggested, guided) before it is actually inserted. Note that if there is only one destination of the product (if two shopping baskets are not used), notification of the destination of the product is not necessary.

[0239] The destination may be notified even if the product is not the same. For example, if a 2L PET bottle, 5kg of rice, etc. is scanned and registered as a new product (different from a previously registered product) (or if a watermelon, melon, etc. is registered by button), a message such as "We recommend placing the product in the lower basket" may be displayed on a small screen to lower the center of gravity (for stability). In other words, the appropriate destination may be determined based on the weight, size, etc. of the product and suggested to the customer before placing it. Note that destination information (upper basket / lower basket) may be set (stored) in the product master, etc., or the destination may be determined based on the weight, size, etc. set in the product master, etc. (for example, by comparing with a threshold value). By notifying the destination, the risk of falling, etc. is reduced, and shopping becomes safer and more efficient by reducing confusion about where to place the product.

[0240] (8) Although not explained in the above embodiment, the source of removal may be notified before the product is removed. For example, when a cancellation operation or a quantity reduction operation is performed for a certain product, the shopping basket into which the product (the product to be canceled or the product to be reduced in quantity) is placed may be notified as the source of removal. For example, a small screen (not shown) may be displayed on the registration screen of FIG. 6(A) displaying a message such as "Please remove from the upper basket." Note that if there is only one destination (if two shopping baskets are not used), notification of the source of removal is not necessary.

[0241] (9) In the above embodiment, an example was described in which the settlement device 50 displayed a product list screen as the first screen after reading the settlement code. However, the first screen is not limited to the product list screen. For example, the settlement device 50 may display a payment method selection screen as the first screen after reading the settlement code. Note that a list display button that displays the product list screen may be provided on the payment method selection screen.

[0242] Furthermore, just as in the case where a product list screen is displayed as the first screen, even when a payment method selection screen is displayed as the first screen, if fraud or the like is detected, the process may not proceed. For example, if fraud or the like is detected, the settlement device 50 may disable the operation of executing payment on the payment method selection screen (operation of the Confirm button). Furthermore, if fraud or the like is detected, the settlement device 50 may not display the payment method selection screen, but may instead display a product list screen in which operation of the Check Out button BT90 is disabled.

[0243] (10) In the above embodiment, an example was described in which the mobile terminal 60 recognizes product insertion, etc. (insertion of a product into the upper basket, insertion of a product into the lower basket, removal of a product from the upper basket, removal of a product from the lower basket), but another device may recognize product insertion, etc. In other words, another device may be equipped with the product insertion / removal recognition function that the mobile terminal 60 has. In the above embodiment, an example was described in which the mobile terminal 60 monitors fraud, etc. (unregistered insertion, unregistered insertion, uncancelled removal, uncancelled removal), but another device may monitor fraud, etc. In the above embodiment, an example was described in which the device that recognizes product insertion, etc. and the device that monitors fraud, etc. are the same device (mobile terminal 60), but the device that recognizes product insertion, etc. and the device that monitors fraud, etc. may be different devices. Several specific examples are described below.

[0244] (The mobile terminal 60 recognizes the insertion of products, etc., and the cloud server 20 monitors for fraud, etc.) The mobile terminal 60 recognizes the insertion of a product, etc., and the cloud server 20 monitors for fraud, etc. For example, the control may be as follows: The mobile terminal 60 recognizes the insertion of a product, etc., based on the weighing information received from the cart 70 and the image captured by the imaging unit. When the mobile terminal 60 recognizes the insertion of a product, etc., it transmits the recognition result of the insertion of the product, etc. to the cloud server 20. Furthermore, when an operation, etc., has been performed on the mobile terminal 60, it transmits operation information on the mobile terminal (information that a product has been registered (scan registration, button registration), information that a product has been canceled (operation of the product cancel button (described later)), information that a quantity has been changed (an operation to decrease the quantity using the minus button (described later), an operation to increase the quantity using the plus button (described later)), etc.) to the cloud server 20.

[0245] Cloud server 20 monitors for fraud and the like based on information (recognition results of product insertion, etc., operation information) received from mobile terminal 60. Cloud server 20 transmits the recognition results of fraud and the like to mobile terminal 60. For example, when cloud server 20 recognizes that an unregistered insertion has occurred, it transmits information to mobile terminal 60 that an unregistered insertion has occurred.

[0246] (Cloud server 20 recognizes product input, etc., and monitors fraud, etc.) The cloud server 20 recognizes the insertion of goods, etc., and monitors for fraud, etc. For example, the control may be as follows: The mobile terminal 60 transmits weighing information received from the cart 70 to the cloud server 20. The mobile terminal 60 also transmits images captured by the imaging unit to the cloud server 20. The mobile terminal 60 also transmits operation information on its own terminal to the cloud server 20 when an operation, etc., is performed on its own terminal.

[0247] Cloud server 20 recognizes the insertion of merchandise, etc. based on information (weighing information, captured images) received from mobile terminal 60. Cloud server 20 transmits the recognition results of the insertion of merchandise, etc. to mobile terminal 60. Cloud server 20 monitors for fraud, etc. based on the recognition results of the insertion of merchandise, etc. and information (operation information) received from mobile terminal 60. Cloud server 20 transmits the recognition results of fraud, etc. to mobile terminal 60.

[0248] (The cart 70 recognizes the insertion of products, and the mobile terminal 60 monitors for fraud, etc.) The cart 70 recognizes the insertion of goods, etc., and the mobile terminal 60 monitors for fraud, etc. For example, the control may be as follows: The mobile terminal 60 transmits an image captured by the imaging unit to the cart 70. The cart 70 recognizes the insertion of goods, etc., based on the weight value from the weighing unit and the information (captured image) received from the mobile terminal 60. The cart 70 transmits the recognition result of the insertion of goods, etc., to the mobile terminal 60.

[0249] The mobile terminal 60 monitors for fraud and the like based on information received from the cart 70 (recognition results of product insertion, etc.) and the status of product registration, etc. (operation information on the mobile terminal itself).

[0250] (Cart 70 recognizes when products are inserted and monitors for fraud, etc.) The cart 70 recognizes the insertion of products, etc., and monitors for fraud, etc. For example, the control may be as follows: The mobile terminal 60 transmits an image captured by the imaging unit to the cart 70. Furthermore, when an operation, etc. is performed on the mobile terminal 60, the mobile terminal 60 transmits operation information on the terminal to the cart 70.

[0251] The cart 70 recognizes the insertion of goods, etc. based on the weighing value from the weighing unit and information (captured image) received from the mobile terminal 60. The cart 70 transmits the recognition result of the insertion of goods, etc. to the mobile terminal 60. The cart 70 monitors for fraud, etc. based on the recognition result of the insertion of goods, etc. and information (operation information) received from the mobile terminal 60. The cart 70 transmits the recognition result of fraud, etc. to the mobile terminal 60.

[0252] (The cart 70 recognizes the insertion of products, and the cloud server 20 monitors for fraud, etc.) The cart 70 recognizes the insertion of products, etc., and the cloud server 20 monitors for fraud, etc. For example, the control may be as follows: The mobile terminal 60 transmits to the cart 70 an image captured by the imaging unit.

[0253] The cart 70 recognizes the insertion of a product, etc., based on the weight value measured by the weighing unit and the information (captured image) received from the mobile terminal 60. The cart 70 transmits the recognition result of the insertion of a product, etc., to the mobile terminal 60.

[0254] The mobile terminal 60 transmits information received from the cart 70 (recognition results of product insertion, etc.) to the cloud server 20. Furthermore, when an operation, etc. is performed on the mobile terminal 60, the mobile terminal 60 transmits operation information on the mobile terminal to the cloud server 20.

[0255] Cloud server 20 monitors for fraud and the like based on information (recognition results of inserting merchandise, etc., captured images) received from mobile terminal 60. Cloud server 20 transmits the recognition results of fraud and the like to mobile terminal 60.

[0256] (11) Although not explained in the above embodiment, a device used by a store clerk (for example, a transaction status management device 40, a mobile terminal for the store clerk (smartphone, tablet terminal, not shown)) may access a cloud server, and the device may display the recognition results of product insertion and removal, as well as the monitoring results of fraud, etc., on the mobile terminal 60.

[0257] In other words, the cloud server 20 stores, based on notifications from the mobile terminal 60 (the various notifications mentioned above), information such as "information that an item has been placed in the upper basket" or "information that an item has been placed without registration," and the transaction status management device 40, etc. may display the above information (or information based on the above information) stored in the cloud server 20 on a display unit.

[0258] (12) Although not explained in the above embodiment, for example, weight information for each product can be stored in the product master as verification information, and the registered product can be checked by weight (weight check) to see if the inserted product matches the registered product, or if the removed product matches the canceled product.

[0259] (13) As described above, the mobile terminal 60 is equipped with an acceleration sensor and a gyro sensor, and therefore may determine whether the mobile terminal 60 itself (the mobile terminal 60 set in the cart 70, i.e., the cart 70) is moving or stationary. Note that instead of or in addition to the mobile terminal 60, the cart 70 itself may be equipped with an acceleration sensor and a gyro sensor and determine whether it is moving or stationary. In other words, the sensing results of the acceleration sensor and gyro sensor may be used to determine whether the cart 70 is moving or stationary.

[0260] Even if the weight value changes while the cart 70 is moving, the mobile terminal 60 may ignore the change in the weight value, assuming that the change is due to vibration during movement (this is called ignoring control). By ignoring control, a change in the weight value due to movement will not be mistakenly determined to be an item being inserted or removed, thereby improving the accuracy of determining whether an item has been inserted or removed.

[0261] In the ignorance control, the insertion or removal of products during movement (between the start and end of movement) is not monitored, and so the ignorance control may become a breeding ground for fraud, etc. To address this, the mobile terminal 60 may acquire a weighing value from the cart 70 after movement has ended (for example, a weighing value after a predetermined time (3 seconds) has elapsed since the cart came to a standstill), and monitor whether the weighing value after movement has ended has changed from the weighing value before movement started (the previous weighing value stored in the memory) (in other words, whether or not products have been inserted or removed during movement (between the start and end of movement)).

[0262] (14) Related to (13) above, the weight value may also change when the customer is rearranging the contents of the shopping basket (such as changing the position of products) or checking them (such as checking whether they have been purchased). To prevent rearranging the contents of the shopping basket from being judged as adding or removing products, a declaration button (a start shopping basket rearranging button, an end shopping basket rearranging button) for rearranging the shopping basket may be provided on the screen of the mobile terminal 60 (for example, a registration screen), and the customer may be prompted to rearrange the shopping basket during the declaration (between operating the start shopping basket rearranging button and the end shopping basket rearranging button). The mobile terminal 60 executes ignore control during the declaration. As in the case of movement, the mobile terminal 60 may also acquire the weight value after rearranging is completed (after operating the end shopping basket rearranging button) and monitor whether the weight value after rearranging has changed from the weight value before rearranging began (the previous weight value stored in the memory) (i.e., whether products were added or removed during the declaration).

[0263] (15) In the above embodiment, an example has been described in which the device identification information (e.g., Bluetooth address) of each cart 70 is obtained from a beacon, and an example has been described in which the device identification information of each cart 70 is obtained from a two-dimensional code 78 affixed to each cart 70. However, the device identification information of each cart 70 may be obtained by other methods. For example, an NFC tag (not shown) may be affixed to each cart, and the device identification information of each cart 70 may be obtained from the NFC tag. The NFC tag includes a storage unit that stores the device identification information (connection information) of the cart 70 (the cart 70 to which the NFC tag is affixed) and a communication unit that transmits the device identification information stored in the storage unit to an external device (specifically, the mobile terminal 60).

[0264] When connection identification information is acquired from a beacon, multiple pieces of connection identification information (respective connection identification information from multiple carts 70 present in the vicinity) may be acquired. When multiple pieces of connection identification information are acquired, information about each cart 70 for which connection identification information is acquired is displayed on the screen, and the customer is prompted to select one cart 70 to use. The information about the cart 70 may be, for example, one or more of the following (A) to (C). (A) Car number information of each cart 70. The cart number information is affixed to each cart 70. The customer can visually confirm (by checking the cart number information) which cart 70 (available cart 70) displayed on the screen is, and easily select the cart 70 to use on the screen. The cart number information may be numbers such as "Cart No. 1" or "Cart No. 2," letters such as "Cart No. A" or "Cart No. B," or the name of a plant or animal. (B) Characteristic information about each cart 70 (color, shape, etc.). As with the car number information, the customer can visually check which cart 70 is displayed on the screen (by checking the characteristic information) and easily select the cart 70 to use on the screen. (C) Information on distance from the customer. Information such as "closest" or "second closest" may be displayed on the screen for each cart 70. The carts may also be displayed in order of proximity.

[0265] (16) In the above embodiment, the state of establishing communication between the mobile terminal 60 and the cart 70 using Bluetooth is expressed as pairing. However, this is not limited to encrypted communication and may be unencrypted communication. Furthermore, it may be so-called bonding, in which a key exchange protocol for encryption is executed and the key is stored for communication. In other words, in this application, pairing refers to a state in which Bluetooth communication is established, and the detailed form of this communication is not important.

[0266] (17) The cart 70 may be connectable to a network (for example, LAN 19) via, for example, a wireless LAN. The mobile terminal 60 may be connectable to a network (for example, LAN 19) via, for example, a wireless LAN.

[0267] (18) In the above embodiment, the mobile terminal 60 is described as being owned by the customer, but it may also be a terminal (rental terminal) provided by the store. The mobile terminal 60 as a rental terminal is provided as a dedicated terminal in which only application A can be executed. When the mobile terminal 60 is a rental terminal, the menu screen (FIG. 3(B)) is displayed from the beginning.

[0268] (19) This is also related to (18) above, but when the mobile terminal 60 is a rental terminal, the mobile terminal 60 and the cart 70 may be separate entities, or the mobile terminal 60 (or each component of the mobile terminal 60) and the cart 70 may be formed integrally. When the rental terminal, that is, the mobile terminal 60, and the cart 70 are separate entities, the customer borrows the rental terminal after entering the store and sets it in the cart 70.

[0269] The mobile terminal 60 (or each component of the mobile terminal 60) and the cart 70 being integrally formed includes a configuration in which the cart 70 includes the entire mobile terminal 60, as well as a configuration in which the cart 70 includes separate components (display, imaging device, etc.) corresponding to each component of the mobile terminal 60 (display unit, imaging unit, etc.). In other words, one location (part) of the cart 70 may be the entire mobile terminal 60, or two or more locations of the cart 70 may each be a component corresponding to one or more components of the mobile terminal 60. The cart 70 including the mobile terminal 60 (or each component of the mobile terminal 60) may be referred to as an integrated cart (or dedicated cart).

[0270] That is, the type of cart 70 in the sales system 1 and the manner of use of the cart 70 may be either of the following (A) and (B). (A) A cart 70 of a type that requires a separate mobile terminal 60 (a customer's mobile terminal 60 or a mobile terminal 60 that is a rental terminal of the store) to be set in it. Each time a customer uses the cart 70, the customer uses the cart 70 by connecting to the mobile terminal 60 via BT (Bluetooth). (B) A type of cart 70 (integrated cart) that is formed integrally with the mobile terminal 60 (or each component of the mobile terminal 60). Because it is formed integrally, there is no connection equivalent to the connection between separate components, and no special operation is required for connection. For example, it is sufficient that one component of the integrated cart (e.g., a weighing unit) and other components (e.g., a display operation unit, etc.) can automatically communicate when the power is turned on. One component of the integrated cart and other components may communicate via wire or wirelessly.

[0271] (20) In the above embodiment, an example was described in which cart 70 includes upper and lower loading sections 71 and 72 arranged vertically, but cart 70 may also include front and rear loading sections arranged front to back. Just as the items placed on upper and lower loading sections 71 and 72 are weighed by a single common weighing unit, the items placed on the front and rear loading sections are weighed by a single common weighing unit.

[0272] The front placement section is the placement section on the front side (rear side) in the direction of travel. The rear placement section is the placement section on the rear side (near side) in the direction of travel. The holder section 76 is located above the rear placement section on the near side. In other words, the mobile terminal 60 set in the holder section 76 acquires a captured image looking down on the shopping basket (near side basket) placed on the rear placement section.

[0273] In the case of a cart 70 having a front mounting section and a rear mounting section, the portable terminal 60 recognizes that an item has been put into or removed from the front basket (the shopping basket placed in the rear mounting section) if the weight value measured by the cart 70 increases and the image captured by the imaging section changes. The portable terminal 60 recognizes that an item has been put into the rear basket (the shopping basket placed in the front mounting section) if the weight value measured by the cart 70 increases and the image captured by the imaging section does not change. The portable terminal 60 recognizes that an item has been removed from the front basket if the weight value measured by the cart 70 decreases and the image captured by the imaging section changes. The portable terminal 60 recognizes that an item has been removed from the rear basket if the weight value measured by the cart 70 decreases and the image captured by the imaging section does not change.

[0274] (21) In the above embodiment, the cart 70 has two placement sections (upper placement section 71, lower placement section 72), but the cart 70 may have three placement sections. Any items placed on any of the three placement sections are weighed by a single common weighing section. The arrangement of the three placement sections is not particularly limited. For example, the placement sections may be an upper placement section, a middle placement section, and a lower placement section in the vertical direction, or a front placement section (rear placement section), a middle placement section, and a rear placement section (front placement section) in the front-rear direction. Furthermore, the placement sections may have two levels, one on the upper level and two on the lower level (front and rear), or two on the upper level (front and rear) and one on the lower level.

[0275] Two imaging units (an imaging unit for mounting unit A and an imaging unit for mounting unit B) may be provided to capture images of the inside of the shopping basket placed on each of two mounting units (e.g., mounting units A and B) out of the three mounting units (e.g., mounting units A, B, and C). One of the two imaging units may be an imaging unit of the mobile terminal 60, and the other may be an imaging unit provided in, for example, the cart 70 (or may be attached to the cart 70 as a separate unit). The captured images captured by the imaging unit provided in the cart 70 are transmitted to the mobile terminal 60. That is, the mobile terminal 60 acquires, using its own imaging unit, captured images of the inside of the shopping basket placed on one mounting unit (e.g., mounting unit A), and also acquires, from the cart 70, captured images of the inside of the shopping basket placed on another mounting unit (e.g., mounting unit B). As a result, even if a shopping basket is placed on each of the three placement sections, the mobile terminal 60 can recognize which shopping basket an item has been added to and which placement section an item has been removed from.

[0276] The same applies to the case where the cart 70 includes four or more placement units and one common weighing unit. In other words, it is sufficient to have one imaging unit less than the number of placement units.

[0277] (22) Although not described in the above embodiment, particularly in relation to (20) and (21) above, the sizes of the shopping baskets placed on each placement section do not have to be the same. That is, the size of one placement section or the space above one placement section may be different from the size of the space above the other placement sections. As an example, the shopping basket placed on the rear placement section may be smaller than the shopping baskets placed on the other placement sections. That is, the size of the rear placement section or the space above the rear placement section may be smaller than the size of the space above the other placement sections or the other placement sections.

[0278] (23) In the above embodiment, the weighing unit and the loading unit provided on the cart 70 were described, but the same applies to the weighing unit and the loading unit provided on other devices (or the weighing unit and the loading unit installed near other devices).

[0279] For example, in a stationary device where customers register products and make payments (a so-called fully self-service registration and payment device, referred to as registration and payment device 51), product insertion and the like may be recognized and fraud and the like may be monitored.

[0280] In other words, the present invention can also be applied to a configuration in which two loading sections and one weighing section are installed near the registration and settlement device 51 (or in a registration and settlement device 51 that has loading sections and a weighing section), and an imaging section provided in the registration and settlement device 51 (or an imaging section separate from the registration and settlement device 51) captures images of items placed on one of the loading sections.

[0281] For example, if the weight value increases and the image captured by the imaging unit changes, the registered settlement device 51 recognizes that a product has been placed in a shopping basket (basket A) placed on the mounting unit that is the image capture target. If the weight value increases and the image captured by the imaging unit does not change, the registered settlement device 51 recognizes that a product has been placed in a shopping basket (basket B) placed on a mounting unit that is not the image capture target. If the weight value decreases and the image captured by the imaging unit changes, the registered settlement device 51 recognizes that a product has been removed from basket A. If the weight value decreases and the image captured by the imaging unit does not change, the registered settlement device 51 recognizes that a product has been removed from basket B.

[0282] The same can be applied to the monitoring of fraud, etc. by the registration settlement device 51. As an example, monitoring of unregistered insertion will be described. For example, the registration settlement device 51 determines that an unregistered insertion has occurred if there is no scan registration, button registration, or quantity increase within a predetermined time before the timing of recognizing the insertion of a product (insertion of a product into basket A or basket B), and if there is no button registration or quantity increase within a predetermined time after the timing of recognizing the insertion of the product. When an unregistered insertion has occurred (when the registration settlement device 51 recognizes an unregistered insertion), it notifies the user of the unregistered insertion (for example, by screen or voice). For example, the registration settlement device 51 may identify the shopping basket into which the product has been inserted and notify the user of the unregistered insertion, and instruct the user to register the product. The same applies to monitoring of unregistered insertion, uncancelled removal, and uncancelled removal.

[0283] (24) Some or all of the functions (input / output, storage, processing (including judgment)) of each device (e.g., transaction status management device 40, settlement device 50, registration settlement device 51, mobile terminal 60, cart 70, etc.) may be realized by a device other than the device described as the entity that executes the function. For example, each device may function as a so-called thin client specialized for input / output interfaces, etc. As an example, a cloud server (which may be the same as cloud server 20 or another cloud server) may execute a program for controlling the operation of each device, and each device may display a screen using a browsing function, etc.

[0284] As an example, cloud server 20 may control the display on the screen of mobile terminal 60 (FIGS. 3 to 6, etc.), or, as has already been explained in part (determining product insertion, product removal, fraud, etc.), cloud server 20 may also control the operation of mobile terminal 60. In other words, a program (app A) that controls the operation of mobile terminal 60 related to service A may run on the CPU of mobile terminal 60, or may run on the CPU of cloud server 20. In other words, app A may be a web application used using a browser.

[0285] For example, the cloud server 20 may be a server capable of communicating with an information processing device used with a product placement device having multiple placement units, and may include: a first detection means for detecting a change in the placement status of the products on the placement units based on weight information of the placement units acquired from the information processing device; a second detection means for detecting a change in the placement status of the products on the placement units based on captured images of the placement units acquired from the information processing device; and a product insertion / removal recognition means for recognizing the insertion of products into or removal of products from the placement units. The product insertion / removal recognition means may recognize which placement unit the product was inserted into or which placement unit the product was removed from based on the detection results of the first detection means and the second detection means. The cloud server 20 may further include a notification control means for notifying the information processing device that a product has been inserted or removed. The notification control means may identify the placement unit into which the product was inserted or removed based on the recognition result of the product insertion / removal recognition means and notify the information processing device that a product has been inserted or removed.

[0286] <Summary of the embodiment> [Technical field] The present invention relates to a program, an information processing device, a system, a server, and a method for recognizing whether a commodity is inserted or removed. [Background technology] BACKGROUND ART A shopping cart is known in which one weighing unit collectively weighs the weight of products and the like placed on two placement units (see, for example, Patent Document 1). [Prior art document] [Patent Documents] [Patent Document 1] JP 2023-127262 A [Summary of the Invention] [Problem to be solved by the invention] However, there is room for improvement in the operation of shopping carts. The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technology for more optimally operating shopping carts. [Means for solving the problem]

[0287] (1) One aspect for solving the above-mentioned problem is a program for causing an information processing device (e.g., a mobile terminal 60) used in conjunction with a product placement device (e.g., a cart 70) having multiple placement sections (e.g., an upper placement section 71, a lower placement section 72) to function as a computer, the program causing the computer to function as a product insertion / removal recognition means that recognizes the insertion of a product into a placement section or the removal of a product from a placement section (product insertion / removal recognition function), and the product insertion / removal recognition means is characterized in that it recognizes which placement section a product has been inserted into or which placement section a product has been removed from. According to the program (1), the shopping cart can be operated more efficiently, for example, by checking whether or not an item is being put in or taken out.

[0288] (2) The program described in (1) is characterized in that the computer functions as a first detection means (e.g., steps S1 and S2 in Figure 7) that detects changes in the placement status of products on the placement section by weight, and a second detection means (e.g., steps S10 and S20 in Figure 7) that detects changes in the placement status of products on the placement section by captured images, and the product insertion / removal recognition means recognizes which placement section a product has been inserted into or which placement section a product has been removed from based on the detection results by the first detection means and the detection results by the second detection means. The program (2) allows for more efficient operation of the shopping cart, for example, by reducing costs and efficiently checking product insertion and removal without compromising storage efficiency.

[0289] (3) The commodity placement device includes an upper placement section (e.g., upper placement section 71) and a lower placement section (e.g., lower placement section 72) arranged vertically, the first detection means detects a change in placement status of the upper placement section or the lower placement section by weight, the second detection means detects a change in placement status of the upper placement section by an image, and the commodity insertion / removal recognition means recognizes a change in placement status when the first detection means detects a change in placement status and the second detection means detects a change in placement status (e.g., in the case of step S10 (YES) in FIG. 7). The program described in (2) is characterized in that, if the first detection means detects a change in the placement status and the second detection means does not detect a change in the placement status (for example, if step S10 (NO) in Figure 7 or if step S20 (NO) in Figure 7), it recognizes that a product has been placed on or removed from the upper placement section, and if the first detection means detects a change in the placement status and the second detection means does not detect a change in the placement status (for example, if step S10 (NO) in Figure 7 or if step S20 (NO) in Figure 7), it recognizes that a product has been placed on or removed from the lower placement section. According to the program (3), the shopping cart can be operated more efficiently.

[0290] (4) A program described in any of (1) to (3) that causes the computer to function as an alarm means for notifying (e.g., displaying or outputting audio) when a product is inserted into or removed from a placement section, and that the alarm means identifies the placement section where the product was inserted or removed and notifies (e.g., displaying small screens SG1 to SG4) that a product has been inserted or removed based on the recognition result by the product insertion / removal recognition means. According to the program (4), the shopping cart can be operated more efficiently.

[0291] (5) A program described in any of (1) to (3) that causes the computer to function as a registration means for registering products to be purchased (e.g., scan registration, button registration), and a display means for displaying information about the products registered by the registration means, wherein the display means, when no product is registered by the registration means and the product insertion / removal recognition means recognizes that a product has been inserted, displays (e.g., displays the small screen SG21 of Figure 8(A)) that a product has been inserted illegally (unregistered insertion) into the placement section where the product insertion / removal recognition means recognizes that a product has been inserted. The program (5) allows for more efficient operation of the shopping cart, for example, by efficiently checking for and dealing with unregistered items.

[0292] (6) A program as described in any one of claims 1 to 3, characterized in that the computer functions as a registration means for registering products to be purchased (e.g., scan registration, button registration), a cancellation means for canceling the purchase of a product, and a display means for displaying information about the product registered by the registration means, and the display means, when a product is canceled by the cancellation means and the product insertion / removal recognition means does not recognize that the product has been removed, displays (e.g., displays the small screen SG41 of Figure 8 (C)) that a product has been fraudulently removed (not canceled but removed) from the placement section where the product insertion / removal recognition means recognized that the product had been removed. The program (6) allows for more efficient operation of the shopping cart, for example, by efficiently checking for and dealing with uncancelled items.

[0293] (7) A program described in any of (1) to (3) that causes the computer to function as a registration means (e.g., scan registration, button registration) for registering products to be purchased, a quantity change means for changing the quantity of products to be purchased, and a display means for displaying information about products registered by the registration means, wherein the display means, when there is no increase in quantity by the quantity change means and the product insertion / removal recognition means recognizes that a product has been inserted, displays that a product has been fraudulently inserted (unregistered insertion) into the placement section where the product insertion / removal recognition means recognized that a product has been inserted, and when there is a decrease in quantity by the quantity change means and the product insertion / removal recognition means does not recognize that a product has been removed, displays that a product has been fraudulently removed (cancelled but not removed) from the placement section where the product insertion / removal recognition means recognized that a product has been removed. The program (7) allows for more efficient operation of the shopping cart. For example, fraudulent attempts such as quantity changes can be dealt with efficiently.

[0294] (8) A program described in any of (1) to (3) that causes the computer to function as a registration means (e.g., scan registration, button registration) for registering products to be purchased, a quantity change means for changing the quantity of products to be purchased, and a display means for displaying information about products registered by the registration means, wherein the display means displays a message that a product has been forgotten to be inserted (forgotten to be inserted) for a product that has been registered or increased in quantity when the registration means has registered a product but the product insertion / removal recognition means has not recognized that the product has been inserted, or when the quantity change means has increased the quantity but the product insertion / removal recognition means has not recognized that the product has been inserted. According to the program (8), the shopping cart can be operated more efficiently. For example, forgetting to insert items, which is disadvantageous to the customer, can be dealt with efficiently.

[0295] (9) A program described in any of (1) to (3) that causes the computer to function as a registration means (e.g., scan registration, button registration) for registering products to be purchased, a cancellation means for canceling the purchase of a product, a quantity change means for changing the quantity of a product to be purchased, and a display means for displaying information about the products registered by the registration means, wherein the display means displays a message indicating that the cancellation or quantity decrease has been forgotten (cancellation forgotten) for the product recognized as having been removed when the cancellation means has not canceled the product and the product input / removal recognition means recognizes that the product has been removed, or when the quantity change means has not decreased the quantity and the product input / removal recognition means recognizes that the product has been removed. The program (9) allows for more efficient operation of the shopping cart. For example, it is possible to deal effectively with cases where customers forget to cancel their purchases, which is disadvantageous to the customer.

[0296] (10) One aspect for solving the above-mentioned problem is an information processing device (e.g., a mobile terminal 60) used together with a product placement device (e.g., a cart 70) having multiple placement sections (e.g., an upper placement section 71, a lower placement section 72), and is equipped with a product insertion / removal recognition means for recognizing the insertion of a product into a placement section or the removal of a product from a placement section, wherein the product insertion / removal recognition means recognizes from which placement section a product has been removed. According to the information processing device of (10), the same effect as that of the program of (1) can be obtained.

[0297] (11) One aspect for solving the above-mentioned problem is a system (e.g., sales system 1) including an information processing device used together with a product placement device capable of placing multiple shopping baskets, and including a product insertion / removal recognition unit that recognizes the insertion of a product into a shopping basket or the removal of a product from a shopping basket, wherein the product insertion / removal recognition unit recognizes which shopping basket a product has been inserted into or which shopping basket a product has been removed from. According to the system (11), the same effect as the program (1) above can be obtained.

[0298] (12) One aspect for solving the above-mentioned problem is a program for causing each of one or more devices (e.g., one or more of the cloud server 20, the checkout device 50, the mobile terminal 60, and the cart 70) that constitute a system (e.g., a sales system 1) including an information processing device used together with a product placement device having multiple placement sections to function as a computer, the program causing the computer to function as a product placement / removal recognition means that recognizes the insertion of a product into a placement section or the removal of a product from a placement section, and the product placement / removal recognition means is characterized in that it recognizes from which placement section a product has been removed. According to the program (12), the same effect as the program (1) can be obtained.

[0299] (13) One aspect for solving the above-mentioned problem is a server (e.g., cloud server 20) capable of communicating with an information processing device used together with a product placement device having multiple placement sections, the server comprising: a first detection means for detecting a change in the placement status of the products on the placement sections based on weight information of the placement sections obtained from the information processing device; a second detection means for detecting a change in the placement status of the products on the placement sections based on an image of the placement sections obtained from the information processing device; and a product insertion / removal recognition means for recognizing the insertion of a product into a placement section or the removal of a product from a placement section, wherein the product insertion / removal recognition means recognizes which placement section a product has been inserted into or which placement section a product has been removed from based on the detection results by the first detection means and the detection results by the second detection means. The server (13) can provide the same effects as the programs (1) and (2) above.

[0300] (14) One aspect for solving the above-mentioned problem is a program for causing a server (e.g., cloud server 20) capable of communicating with an information processing device used together with a product placement device having multiple placement sections to function as a computer, the program causing the computer to function as a first detection means for detecting a change in the placement status of the product on the placement section based on weight information of the placement section obtained from the information processing device, a second detection means for detecting a change in the placement status of the product on the placement section based on an image of the placement section obtained from the information processing device, and a product insertion / removal recognition means for recognizing the insertion of a product into a placement section or the removal of a product from a placement section, the program being characterized in that the product insertion / removal recognition means recognizes which placement section a product has been inserted into or which placement section a product has been removed from based on the detection results by the first detection means and the detection results by the second detection means. According to the program (14), the same effects as those of the programs (1) and (2) above can be obtained.

[0301] (15) One aspect for solving the above-mentioned problem is a product insertion / removal recognition method that uses an information processing device used with a product placement device having multiple placement sections to recognize the insertion of a product into a placement section or the removal of a product from a placement section, the product insertion / removal recognition method comprising a first detection step of detecting a change in the placement status of the product in the placement section by weight, a second detection step of detecting a change in the placement status of the product in the placement section by captured images, and a product insertion / removal recognition step of recognizing the insertion of a product into a placement section or the removal of a product from a placement section, characterized in that the product insertion / removal recognition step recognizes which placement section the product has been inserted into or which placement section the product has been removed from based on the detection results from the first detection step and the detection results from the second detection step. According to the product insertion / removal recognition method (15), the same effects as those of the programs (1) and (2) can be obtained.

[0302] Note that a program for implementing the functions of each of the above-described devices (e.g., cloud server 20, transaction status management device 40, settlement device 50, mobile terminal 60, cart 70, etc.) may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform processing for each of the above devices. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program on a computer system. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" may also include multiple computer devices connected via a network, including communication lines such as the Internet, WAN, LAN, and dedicated line. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into a computer system. Thus, a recording medium storing a program may be a non-transitory recording medium such as a CD-ROM. The term "recording medium" also includes internal or external recording media accessible from a distribution server for distributing the program. The program code stored on the distribution server's recording medium may be different from the program code in a format executable by a terminal device. In other words, the format in which the program is stored on the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that is executable on the terminal device. The program may be divided into multiple parts, each of which may be downloaded at different times and then combined on the terminal device, or each of the divided programs may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a storage medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be one that realizes part of the functions described above.Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program). [Explanation of symbols]

[0303] 1...Sales system 10...Head office server 20...Cloud server 30...Store controller 40...Transaction status management device 50...Payment device 51...Registration and payment device 60...Mobile terminal 70...Cart

Claims

1. A program for causing an information processing device used with a commodity placement device having a plurality of placement units to function as a computer, The computer A product insertion / removal recognition means for recognizing the insertion of a product into the placement section or the removal of a product from the placement section. It functions as The product insertion / removal recognition means A program characterized by recognizing which placement section an item has been put into or which placement section an item has been removed from.

2. The computer a first detection means for detecting a change in the placement state of the commodity on the placement section based on the weight of the commodity; a second detection means for detecting a change in the placement state of the products on the placement section using a captured image; It functions as The product insertion / removal recognition means The program described in claim 1, characterized in that it recognizes which placement section a product has been inserted into or which placement section a product has been removed from based on the detection results by the first detection means and the detection results by the second detection means.

3. The commodity placement device is The apparatus includes an upper placement section and a lower placement section that are laid out vertically, The first detection means Detecting a change in the placement status of the upper placement section or the lower placement section based on weight, The second detection means Changes in the placement status of the upper placement section are detected by images, The product insertion / removal recognition means When the first detection means detects a change in the placement status and the second detection means detects a change in the placement status, it is recognized that a product has been placed on the upper placement section or removed from the upper placement section, When the first detection means detects a change in the placement state and the second detection means does not detect a change in the placement state, it is recognized that an item has been placed on the lower placement section or removed from the lower placement section.

3. The program according to claim 2.

4. The computer A notification means for notifying when a product is put into the placement section or when a product is removed from the placement section. It functions as The notification means Based on the recognition result by the product insertion / removal recognition means, the fact that a product has been inserted or removed is notified by specifying the placement section into which the product has been inserted or removed.

4. The program according to claim 1, wherein the program is a program for executing a program for executing a program for executing a program.

5. An information processing device used together with a commodity placement device having a plurality of placement units, A product insertion / removal recognition means for recognizing the insertion of a product into the placement section or the removal of a product from the placement section. Equipped with The product insertion / removal recognition means An information processing device that recognizes from which placement section a product has been taken out.

6. A system including an information processing device used together with a product placement device capable of placing multiple shopping baskets, A product insertion / removal recognition unit that recognizes the insertion of products into a shopping basket or the removal of products from a shopping basket Equipped with The product insertion / removal recognition unit A system that recognizes which shopping basket an item has been put into or which shopping basket an item has been removed from.

7. A program for causing one or more devices constituting a system including an information processing device used together with a commodity placement device having a plurality of placement units to function as a computer, The computer A product insertion / removal recognition means for recognizing the insertion of a product into the placement section or the removal of a product from the placement section. It functions as The product insertion / removal recognition means A program that recognizes from which placement section a product has been removed.

8. A server capable of communicating with an information processing device used together with a commodity placement device having a plurality of placement units, a first detection means for detecting a change in the placement status of the products on the placement unit based on weight information of the placement unit acquired from the information processing device; a second detection means for detecting a change in the placement state of the products on the placement section based on a captured image of the placement section acquired from the information processing device; a product insertion / removal recognition means for recognizing the insertion of a product into the placement section or the removal of a product from the placement section; Equipped with The product insertion / removal recognition means Based on the detection results from the first detection means and the second detection means, it is recognized which placement section the product was put into or which placement section the product was removed from. A server characterized by:

9. A program for causing a server capable of communicating with an information processing device used together with a commodity placement device having a plurality of placement units to function as a computer, The computer a first detection means for detecting a change in the placement status of the products on the placement unit based on weight information of the placement unit acquired from the information processing device; a second detection means for detecting a change in the placement state of the products on the placement section based on a captured image of the placement section obtained from the information processing device; A product insertion / removal recognition means for recognizing the insertion of a product into the placement section or the removal of a product from the placement section. It functions as The product insertion / removal recognition means Based on the detection results from the first detection means and the second detection means, it is recognized which placement section the product was put into or which placement section the product was removed from. A program characterized by:

10. A method for recognizing whether a commodity is inserted or removed from a commodity placement unit by using an information processing device used with a commodity placement device having a plurality of placement units, the method comprising: a first detection step of detecting a change in the placement state of the product on the placement unit based on its weight; a second detection step of detecting a change in the placement state of the products on the placement section using captured images; a product insertion / removal recognition step for recognizing insertion of a product into the placement section or removal of a product from the placement section; and The product insertion / removal recognition step includes: A product insertion / removal recognition method characterized by recognizing which placement section a product has been inserted into or which placement section a product has been removed from based on the detection results from the first detection step and the detection results from the second detection step.

Images