Information processing terminal and storage medium

Abstract

The invention provides an information processing terminal and a storage medium. An information processing apparatus includes a determination unit, a display control unit, an acquisition unit, and a transmission unit. The determination unit determines whether or not information related to product registration satisfies an imaging condition. The display control unit displays an automatic imaging guide of the product on the display unit in accordance with the determination result of the determination unit. The acquisition unit acquires captured image data of a captured product on the basis of automatic imaging of the product. The transmission unit transmits the captured image data acquired by the acquisition unit.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing terminal and a storage medium. Background Art

[0002] A transaction processing system is well known in which a customer operates a mobile terminal such as a smartphone or a tablet terminal to input data related to the purchased product while shopping in a store where products are displayed.

[0003] This transaction processing system uses a dedicated self-checkout machine to process the checkout of customers who have input data related to purchased goods placed in a shopping cart, etc., so-called self-service customers. In such a system, in order to prevent customers from forgetting to register goods or registering goods improperly, a clerk at the checkout area confirms the correspondence between the purchased goods placed in the shopping cart, etc. by the customer and the registered goods, etc., at the time of checkout.

[0004] However, it is difficult to check all the purchased goods for all the customers. Therefore, it is required to take measures to prevent forgetting to register in advance and improper registration.

[0005] [Prior art literature]

[0006] [Patent Literature]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2020-107122 Summary of the invention

[0008] The technical problem to be solved by the embodiments of the present invention is to provide a technology for preventing forgetting to register a commodity and improper registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a block diagram showing a schematic structure of a transaction processing system according to an embodiment.

[0010] Figure 2 It is a perspective view showing an example of a shopping cart according to an embodiment.

[0011] Figure 3 This is a sequence diagram used to illustrate the actions of a customer before shopping using the transaction processing system.

[0012] Figure 4 This is a sequence diagram for explaining the customer's actions during shopping and before checkout.

[0013] Figure 5 This is a timing diagram for explaining the checkout operation at the customer's dedicated checkout machine.

[0014] Figure 6This is a schematic diagram showing an example of a store entry code reading screen displayed on a smartphone according to an embodiment.

[0015] Figure 7 This is a schematic diagram showing an example of a registration screen displayed on a smartphone according to an embodiment.

[0016] Figure 8 This is a schematic diagram showing an example of a scanning screen displayed on a smartphone according to an embodiment.

[0017] Fig. 9 This is a schematic diagram showing an example of a registration screen displayed on a smartphone according to an embodiment.

[0018] Fig.10 This is a schematic diagram showing an example of a checkout screen displayed on a smartphone according to an embodiment.

[0019] Fig.11 This is a block diagram schematically showing a configuration example of a monitoring server according to an embodiment.

[0020] Fig.12 This is a block diagram schematically showing a configuration example of a smartphone according to an embodiment.

[0021] Fig.13 This is a block diagram schematically showing a configuration example of a monitoring terminal according to an embodiment.

[0022] Fig.14 It is a diagram showing the main data structure of a first index file stored in a transaction data storage area according to one embodiment.

[0023] Fig.15 It is a diagram showing the main data structure of a first transaction file stored in a transaction data storage area according to one embodiment.

[0024] Fig.16 It is a diagram showing the main data structure of captured image management data stored in the captured image storage area according to one embodiment.

[0025] Fig.17 This is a schematic diagram showing an example of a guidance screen displayed on a smartphone according to an embodiment.

[0026] Fig.18 This is a flowchart showing an example of the steps of information processing executed by a smartphone according to one embodiment.

[0027] Fig.19 This is a schematic diagram showing an example of a shooting guide screen displayed on a smartphone according to an embodiment.

[0028] Fig. 20This is a flowchart showing another example of the steps of information processing executed by the smartphone according to one embodiment.

[0029] Fig.21 This is a schematic diagram showing an example of a registered product detail screen displayed on a monitoring terminal according to an embodiment.

[0030] Fig. 22 This is a schematic diagram showing an example of a registered product confirmation screen displayed on a monitoring terminal according to an embodiment.

[0031] Fig.23 This is a schematic diagram showing an example of a registered product confirmation screen displayed on a monitoring terminal according to an embodiment.

[0032] Fig.24 This is a flowchart showing an example of the steps of information processing executed by a monitoring terminal according to one embodiment. DETAILED DESCRIPTION

[0033] In one embodiment, the information processing terminal includes a determination unit, a display control unit, an acquisition unit, and a sending unit. The determination unit determines whether the information related to the registration of the commodity satisfies the shooting condition. The display control unit causes the display unit to display the automatic shooting guide of the commodity according to the determination result of the determination unit. The acquisition unit acquires the photographed image data of the photographed commodity based on the automatic shooting of the commodity. The sending unit sends the photographed image data acquired by the acquisition unit.

[0034] Hereinafter, one embodiment will be described using the drawings.

[0035] It should be noted that this embodiment illustrates a transaction processing system in which when a customer purchases a product in a store where products are displayed, the customer operates a smart phone as a mobile terminal, inputs data related to the purchased product, and checks out using a dedicated checkout machine. The smart phone is an example of an information processing device operated by the customer. It should be noted that in the following description, the customer may also be referred to as a user, consumer, or member.

[0036] (Structure of transaction processing system 100)

[0037] Figure 11 is a block diagram showing the schematic structure of the transaction processing system 100 of this embodiment. The transaction processing system 100 includes a server system 10 using cloud computing and a POS (Point Of Sales) system 20 built in a franchise store. The server system 10 and the POS system 20 can perform two-way data communication through a network 30 using the Internet protocol. The network 30 is a wide area network that uses a public line or a dedicated line as a relay network and a wired LAN (Local Area Network), a wireless LAN, a mobile communication network, a mobile phone communication network, etc. as an access network. Therefore, a smartphone 40 using a mobile phone communication network can access the server system 10 via the network 30.

[0038] A franchise store is a store that has signed a use contract with the management company of the server system 10. Figure 1 , a POS system 20 of one franchise store is shown. Of course, the POS systems of other franchise stores are also connected to the network 30. However, the POS systems of other franchise stores are not necessarily limited to the POS systems of the same franchise stores. Figure 1 The POS systems 20 shown have the same configuration. However, the configuration of the POS system may differ in part depending on the franchise store.

[0039] The server system 10 includes a database server 11, a management server 12, and a virtual POS server 13, and the servers are connected to each other via a communication line 14. In addition, the server system 10 is connected to a router 15 on the communication line 14, and is connected to the network 30 via the router 15.

[0040] The database server 11 is a computer that provides services for managing a franchise store database 111, a member database 112, a first product master file 113 of each franchise store, and the like.

[0041] The franchise store database 111 is a collection of franchise store records created for each franchise store. The franchise store record is a data record that records a unique store ID set for each franchise store and various data related to the franchise store. The data in the franchise store record includes the network address required for data communication with the franchise store via the network 30.

[0042] The member database 112 is a collection of member records created for each consumer who registers as a member to make purchases using the transaction processing system 100. The member record is a data record that records a unique member ID set for each consumer and various data related to the consumer.

[0043] The first product master file 113 of each franchise store is a collection of product records created for each product sold in the corresponding franchise store. The product record is a data record that records a unique product code set for each product and various data related to the product. The data in the product record includes the product name, price, etc. Each first product master file 113 is uploaded to the database server 11 from the POS system 20 of the corresponding franchise store, for example, at a specified time before the store opens.

[0044] In addition, the first product master file 113 may be managed by the virtual POS server 13. In addition, the management server 12 manages the franchise store database 111 and the member database 112, and the database server 11 may be omitted.

[0045] The management server 12 is a computer that provides services for managing information related to transactions of customers who make purchases at affiliated stores using the transaction processing system 100 in cooperation with the database server 11 and the virtual POS server 13. The virtual POS server 13 is a computer that provides services for virtually registering sales data of goods that customers want to purchase based on POS applications. The main functions of the management server 12 and the virtual POS server 13 will be explained in the operation description to be described later.

[0046] The POS system 20 includes a store server 21, a checkout server 22, a monitoring terminal 23, a POS terminal 24, a dedicated checkout machine 25, and a monitoring server 26, which are connected via an in-store network 27. In addition, the POS system 20 connects a router 28 to the in-store network 27, and is connected to a network 30 via the router 28. Furthermore, the POS system 20 connects an access point 29 to the in-store network 27, and can perform data communication with mobile terminals such as a smartphone 40 and a tablet terminal via a wireless LAN through the access point 29.

[0047] The store server 21 is a computer that provides services such as managing sales information, product inventory information, and order information of the affiliated store based on information on transactions settled at the affiliated store that implements the POS system 20 .

[0048] The store server 21 has a second product master file 211. The second product master file 211 is a collection of product records created for each product sold in the franchise store. The product record is a data record that records a unique product code set for each product and various data related to the product. The data in the product record includes the product name, price, etc. That is, the second product master file 211 becomes the first product master file 113 by being uploaded to the server system 10. However, if the price, etc. is changed during the business hours, the change in the second product master file 211 is reflected in the data. Therefore, depending on the product, the price, etc. of the first product master file 113 and the second product master file 211 may be different.

[0049] The store server 21 is a computer that provides a service supporting the member monitoring function of the monitoring terminal 23. The member monitoring function is a function for monitoring the status of customers who make purchases using the transaction processing system 100. Through this member monitoring function, a list of registered products for which data has been input by the customer using the smartphone 40 in the store is displayed on the monitor of the monitoring terminal 23. The registered products are products before settlement for which the customer inputs data using the smartphone 40. The registered products are also referred to as products for short. The monitoring terminal 23 displays a monitoring image on the monitor. The monitoring image is an image for monitoring information of registered products input using the smartphone 40 and the status of the smartphone 40. The monitoring image includes a list of registered products and a photographic image of purchased products. The purchased products are products before settlement that the customer has put in the shopping cart, etc. The member monitoring function includes a function for checking the information of the registered products of the customer and the purchased products for the monitored object. The monitored objects include products or product registrations that have been forgotten to be registered or suspected of being improperly registered. In addition, a warning is displayed on the monitor when an event that should be reported to the store clerk occurs due to the data input. An event that should be reported to the store clerk is, for example, when a customer purchases alcohol or cigarettes with age restrictions, and a registration error occurs because the product code of the scanned barcode is not set in the second product master file 211.

[0050] The transaction server 22 is a computer that provides a service for supporting member transactions using the dedicated transaction machine 25. The functions of the transaction server 22 will be explained in the operation description to be described later.

[0051] The POS terminal 24 is mainly a checkout machine used to settle transactions with customers who do not use the transaction processing system 100. The POS terminal 24 is operated by a store clerk. The POS terminal 24 can also settle transactions with customers who use the transaction processing system 100. It should be noted that the POS terminal can also be separated into a register and a checkout machine, the register is operated by the store clerk, and the checkout machine is a semi-self-service POS terminal operated by the customer. The POS terminal can also be a fully self-service POS terminal in which the customer performs the registration process and the checkout process.

[0052] The dedicated checkout machine 25 is a checkout machine for settling transactions with customers who use the transaction processing system 100. The customer is the operator of the dedicated checkout machine 25. Customers who do not use the transaction processing system 100 cannot use the dedicated checkout machine 25 to settle transactions.

[0053] The dedicated checkout machine 25 has the function of scanning the checkout barcode displayed on the smartphone 40 used by the customer using the transaction processing system 100, the payment processing function of the settlement amount corresponding to various payment methods such as electronic currency, credit card, cash, etc., the electronic currency recharge function, etc.

[0054] The monitoring server 26 is a computer that cooperates with the server system 10 to monitor the information of the registered products whose data is input by the customer using the smartphone 40 in the store. The monitoring server 26 cooperates with the store server 21 to perform information processing for realizing the member monitoring function of the monitoring terminal 23. The function of the monitoring server 26 is explained in the operation description described later. The monitoring server 26 is an example of an information processing device.

[0055] (Composition of the shopping cart)

[0056] Figure 2 1 is a perspective view showing an example of a cart C of an embodiment. The cart C has a caster portion C1 for movement, a handle frame portion C2, and a basket receiving portion C3. The caster portion C1 has four wheels for smooth movement on the ground. The handle frame portion C2 includes a pair of longitudinal frames C21, C21 vertically arranged on the rear wheel side of the caster portion C1, and a handle bar C22 connecting the upper ends of these longitudinal frames C21, C21. The basket receiving portion C3 is located in front of the midway portion of the handle frame portion C2. The cart C can carry a shopping basket BA equipped in the store on the basket receiving portion C3 and the lower portion thereof.

[0057] A rod C4 is mounted on one longitudinal frame C21 of the cart C. A stand 400 for placing a smartphone 40 is provided at the front end of the rod C4. A consumer using the cart C can place his or her smartphone 40 on the stand 400 to shop.

[0058] (Operation of Transaction Processing System 100)

[0059] Next, the main operations of the transaction processing system are described in the order of preparation, before shopping, during shopping, before checkout, and checkout.

[0060] <Preparation>

[0061] A consumer who wants to use the transaction processing system 100 to shop at a franchise store first installs a dedicated application (hereinafter referred to as a shopping application) on a smartphone 40 to register as a member. The member registration is performed on the management server 12 of the server system 10 via the network 30. The management server 12 issues a member ID unique to the consumer who has registered as a member, and registers a member record in which the member ID is recorded in the member database 112. In addition, the management server 12 notifies the smartphone 40 of the consumer who has registered as a member of the member ID via the network 30. The smartphone 40 stores the member ID in association with the shopping application.

[0062] By registering as a member, a consumer becomes a member. A member can register the use of electronic money that can be used in a franchise store. The registration is performed from a smartphone 40 to a management server 12 via a network 30. The management server 12 records the electronic money ID of the registered electronic money in the member record that records the member ID of the member. It should be noted that the registration of the use of electronic money is not mandatory.

[0063] <Before shopping (refer to Figure 3 , Figure 6 )>

[0064] Before entering the franchise store, the member operates the smartphone 40 to start the shopping application. When the shopping application is started, the screen of the touch panel in the smartphone 40 is transferred to the store entry code reading screen Sa ( Figure 6 The store entry code reading screen Sa is used to read the store entry code Cin ( Figure 3 ) screen.

[0065] The store entry code Cin is prepared at the entrance of the franchise store. For example, the store entry code Cin is printed on a paper medium and affixed to the entrance. For example, the store entry code Cin is displayed on a display placed at the entrance. The store entry code Cin encodes the setting information of the franchise store using, for example, a QR code system. The setting information includes a store ID set for the franchise store, wireless LAN data, etc. The wireless LAN data is the SSID, password, security information, etc. required for the smartphone 40 to connect to the access point 29 using the wireless LAN.

[0066] like Figure 6 As shown, a camera activation button Ba is displayed on the store entry code reading screen Sa. The member touches the camera activation button Ba. This activates the camera of the smartphone 40, so the member points the camera lens at the store entry code Cin. The camera scans the store entry code Cin.

[0067] When the store entry code Cin is scanned, the smartphone 40 sends a store entry request command CMa to the management server 12 via the network 30. Figure 3 ). The store entry request command CMa includes at least the member ID and the store ID included in the store entry code Cin.

[0068] The management server 12 obtains the member ID from the store entry request command CMa and performs member authentication ( Figure 3 ACT1). Member authentication is performed by confirming whether the member ID is managed as a valid member ID. For example, an invalid flag is recorded in the member record of the member ID of a consumer who has lost membership due to withdrawal application, improper behavior, etc. When the management server 12 obtains the member ID of the member record with the invalid flag recorded from the store entry request command CMa, the member authentication is rejected. When the management server 12 obtains the member ID of the member record without the invalid flag recorded from the store entry request command CMa, the member authentication is approved.

[0069] The management server 12 determines whether to approve or deny member authentication ( Figure 3 ACT2). When the member authentication is rejected, the management server 12 replies to the smartphone 40 with a store entry rejection command CMb. The store entry rejection command CMb is sent to the smartphone 40 via the network 30. The smartphone 40 that receives the store entry rejection command CMb stops the shopping application. Therefore, the customer cannot operate the smartphone 40 by himself in the franchise store to input data related to the purchased goods and perform so-called self-service shopping.

[0070] When the management server 12 approves the member authentication, it issues a transaction ID ( Figure 3 ACT3). The transaction ID is a unique ID set for each shopping transaction, that is, the sale of goods, at a member's store. For example, a unique transaction ID is issued by combining a store code, a date, and a serial number. The management server 12 associates the member ID and store ID obtained from the store entry request command CMa with the issued transaction ID and records them in the collaboration table 70 ( Figure 3 That is, the collaboration table 70 is a data table that stores the store ID of the affiliated store where the shopping is performed and the unique transaction ID issued for the transaction in association with the member ID of the consumer who performs self-service shopping.

[0071] The management server 12 replies with a store entry approval command CMc to the smartphone 40. The store entry approval command CMc is sent to the smartphone 40 via the network 30. The smartphone 40 that has received the store entry approval command CMc uses the wireless LAN data obtained from the store entry code Cin to connect to the access point 29 via the wireless LAN. When the smartphone 40 is connected to the access point 29 via the wireless LAN, the screen of the touch panel is transferred to the shopping start screen. The shopping start screen is a screen that urges the consumer to start shopping. A shopping start button is displayed on the shopping start screen.

[0072] On the other hand, the management server 12 that has sent the store entry approval command CMc sends a transaction file creation instruction command CMd to the virtual POS server 13. The creation instruction command CMd includes the store ID and the transaction ID stored in association with the member ID in the collaboration table 70. The virtual POS server 13 that has received the creation instruction command CMd creates a transaction file 80 ( Figure 3 ). In addition, the virtual POS server 13 records the store ID and transaction ID included in the creation instruction command CMd in the transaction file 80. The transaction file 80 is a data file that records the sales data of the goods purchased by the member in the transaction with the member identified by the transaction ID and the payment data of the settlement amount of the transaction.

[0073] <Shopping (refer to Figure 4 , Figure 7 , Figure 8 , Fig. 9 )>

[0074] After confirming the shopping start screen, the member touches the shopping start button. Then, the touch panel screen changes to the registration screen Sb ( Figure 7 ).like Figure 7 As shown, the total number of registered products and the total amount are displayed on the registration screen Sb. However, since the purchased products have not been registered yet, the total number and the total amount are both "0". In addition, a no barcode button Bb, a scan button Bc, and a checkout button Bd are displayed on the registration screen Sb. The no barcode button Bb is a button that is touched when inputting data of a product without a barcode. The scan button Bc is a button that is touched when inputting data of a product with a barcode. The checkout button Bd is a button that is touched when instructing to check out after shopping. However, since the products have not been registered yet, the checkout button Bd cannot be operated. For example, the checkout button Bd is gray. The checkout button Bd can also be deleted.

[0075] When a member finds the product he wants to buy in a store where the product is displayed, he will check whether the product is attached with a barcode BC ( Figure 4). If a barcode BC is attached, the member touches the scan button Bc. When the scan button Bc is touched, the screen of the touch panel transitions to the scan screen Sc ( Figure 8 ).like Figure 8 As shown, a frame ARa indicating the barcode reading area is displayed on the scanning screen Sc. In addition, a barcode presence button Bb and a return button Be are displayed. The return button Be is a button that is touched when the scanning of the barcode BC is stopped.

[0076] The member operates the smartphone 40 so that the barcode BC of the purchased product enters the frame ARa. When the barcode BC enters the frame ARa, the barcode BC is scanned by the camera of the smartphone 40.

[0077] The smartphone 40 that scans the barcode BC sends a registration request command CMe to the management server 12 via the wireless LAN. Figure 4 The registration request command CMe includes the member ID and the data of the barcode BC, that is, the so-called barcode data. The registration request command CMe is received by the access point 29 and is sent from the router 28 to the management server 12 via the network 30.

[0078] The management server 12 searches the cooperation table 70 with the member ID included in the registration request command CMe to obtain the store ID and transaction ID associated with the member ID. Then, the management server 12 sends a registration request command CMf to the virtual POS server 13, replacing the member ID with the store ID and transaction ID.

[0079] The virtual POS server 13 obtains the store ID, transaction ID, and barcode data from the registration request command CMf. Then, the virtual POS server 13 obtains the product code from the barcode data. The virtual POS server 13 searches the first product master file 113 of the franchise store identified by the store ID and obtains the product record containing the product code. Furthermore, based on the product data of the product record, the virtual POS server 13 registers the product sales data consisting of the product code, product type, product name, price, sales quantity, sales amount, etc. in the transaction file 80 in which the store ID and transaction ID are recorded.

[0080] The management server 12 obtains the transaction file 80 in which the commodity sales data is registered from the virtual POS server 13. Then, the management server 12 sends a registration response command CMg to the smartphone 40 which is the source of the registration request command CMe. Figure 4 The registration response command CMg includes the data of the transaction file 80 obtained from the virtual POS server 13. The registration response command CMg is sent from the router 15 to the POS system 20 via the network 30, and further sent from the access point 29 to the smartphone 40.

[0081] In the smartphone 40 that has received the registration response command CMg, the screen of the touch panel changes to the registration screen Sd ( Fig. 9 The registration screen Sd is a screen in which the commodity sales data related to the first commodity AAA is registered in the transaction file 80. Fig. 9 As shown, the sales amount "¥100" and the sales quantity "1 piece" of the product AAA are displayed on the registration screen Sd. In addition, the total quantity is "1 piece" and the total amount is "¥100". In addition, a checkout button Bd can be input.

[0082] After that, the member touches the scan button Bc for the second and subsequent purchased products to scan the barcode of the product. As a result, the management server 12 and the virtual POS server 13 operate in the same manner as described above. As a result, the name of the purchased product, the sales amount and the number of units sold, and the total number of units and the total amount are displayed on the registration screen Sd.

[0083] However, as described above, the sales amount of the commodity sales data stored in the transaction file 80 is calculated based on the price of the first commodity master file 113. The price of the first commodity master file 113 does not reflect the change even if the price changes during business hours. Therefore, the sales amount and total amount displayed on the registration screen Sd may be different from the amount based on the price at the current time point. Therefore, the sales amount and total amount displayed on the registration screen Sd are always positioned as reference amounts during shopping.

[0084] It should be noted that, when the purchased product does not have a barcode, the member touches the no-barcode button Bb. In this way, since a list of products without barcodes is displayed on the touch panel of the smartphone 40, the member selects a purchased product from the list. Then, a registration request command CMe including the product ID of the purchased product is sent to the management server 12. The subsequent actions are the same as when the barcode BC of the product is scanned by the smartphone 40.

[0085] <Before checkout (refer to Figure 4 , Fig.10 )>

[0086] After finishing shopping at the store, the member who wants to check out touches the checkout button Bd on the registration screen Sd. When the checkout button Bd is touched, the smartphone 40 sends a checkout request command CMh (CMh) to the management server 12 via the wireless LAN. Figure 4 The settlement request command CMh includes the member ID. The settlement request command CMh is received by the access point 29 and is sent from the router 28 to the management server 12 via the network 30.

[0087] The management server 12 issues a checkout code ( Figure 4 The checkout code is, for example, a combination of date, time, and a serial number, and no duplicate checkout code is issued on at least one working day. The management server 12 generates a first image Ca( Fig.10 )( Figure 4 ACT12). The first image Ca is an image of the checkout code converted into a barcode, that is, an image of the checkout barcode.

[0088] The management server 12 transmits a transaction approval command CMi to the smartphone 40 that is the transmission source of the transaction request command CMh. The transaction approval command CMi includes the first image Ca.

[0089] The checkout approval command CMi is sent to the POS system 20 via the network 30, and is sent from the access point 29 to the smartphone 40 via the wireless LAN. Upon receiving the checkout approval command CMi, the screen of the touch panel of the smartphone 40 changes to the checkout screen Se ( Fig.10 ).like Fig.10 As shown, the first image Ca is displayed on the checkout screen Se. In addition, an end button Bf is displayed on the checkout screen Se. The end button Bf is a button that is touched when stopping checkout.

[0090] In addition, the management server 12 obtains the store ID and transaction ID stored in the cooperation table 70 in association with the member ID obtained from the checkout request command CMh. Then, the management server 12 obtains the transaction file 80 in which the store ID and transaction ID are described from the virtual POS server 13. Then, the management server 12 sends the transaction file 80 and the checkout code to the POS system 20 of the franchise store identified by the store ID. Thus, the data of the transaction file 80 and the checkout code are sent to the store server 21 of the corresponding franchise store via the network 30. The store server 21 transmits the data of the transaction file 80 and the checkout code received from the management server 12 to the checkout server 22. The checkout server 22 stores the transaction file 80 in the memory in association with the checkout code.

[0091] <Checkout (refer to Figure 5 )>

[0092] The member who has confirmed the checkout screen Se uses the dedicated checkout machine 25 to pay using various payment methods such as electronic money, credit card, and cash.

[0093] use Figure 5 , the checkout operation using the dedicated checkout machine 25 is described.

[0094] The member uses the scanner of the dedicated checkout machine 25 to scan the first image Ca of the checkout screen Se, i.e., the checkout barcode. Then, the data of the checkout barcode, i.e., the so-called barcode data BCD ( Figure 5 ).

[0095] The checkout server 22 obtains the checkout code ( Figure 5 Then, the checkout server 22 detects the transaction file 80 ( Figure 5 The checkout server 22 sends the transaction file 80 to the dedicated checkout machine 25.

[0096] The dedicated checkout machine 25 refers to the second commodity master file 211 managed by the store server 21 to recalculate the settlement amount. Specifically, the dedicated checkout machine 25 obtains the commodity code registered in the transaction file 80. Then, the dedicated checkout machine 25 searches the second commodity master file 211 to obtain the price of the commodity record containing the commodity code. The dedicated checkout machine 25 multiplies the price by the number of sales registered in the transaction file 80 together with the commodity code, and calculates the sales amount of the purchased commodity identified by the commodity code. The dedicated checkout machine 25 performs the above processing for all commodity codes registered in the transaction file 80. Then, the dedicated checkout machine 25 totals the sales amount of all purchased commodities and calculates the settlement amount.

[0097] The dedicated checkout machine 25 displays the recalculated settlement amount on the display. Then, it waits for the payment method selected for the payment ( Figure 5 ACT 32). The recalculated settlement amount may be different from the total amount displayed on the smartphone 40. In this case, since the total amount of the smartphone 40 is a reference amount, the customer pays the settlement amount displayed on the dedicated checkout machine 25.

[0098] The member selects a payment method. For example, if cash is selected, the dedicated checkout machine 25 performs payment processing for the settlement amount in cash ( Figure 5 ACT33). For example, when a credit card is selected, the dedicated checkout machine 25 performs payment processing for paying the settlement amount with the credit card. It should be noted that since these payment processing are well-known processing, detailed description is omitted here.

[0099] When the payment process is completed, the dedicated checkout machine 25 sends a payment completion notification command CMj to the checkout server 22. The payment completion notification command CMj includes payment data. For example, in the case of cash payment, the payment data is the pre-deposited amount, the change amount, etc. For example, in the case of credit card payment, the payment data is the payment amount based on the credit card, etc.

[0100] The checkout server 22 registers the payment data in the transaction file 80. In addition, the checkout server 22 sets the end status to the transaction file 80. The transaction file 80 to which the end status is set is stored in the recording unit of the checkout server 22 or the store server 21.

[0101] (Configuration of Monitoring Server 26)

[0102] Fig.11 This is a block diagram schematically showing a configuration example of the monitoring server 26 according to one embodiment.

[0103] The monitoring server 26 includes a processor 261, a main memory 262, an auxiliary storage unit 263, a communication interface 264, and a transmission line 265. The processor 261, the main memory 262, the auxiliary storage unit 263, and the communication interface 264 can communicate via the transmission line 265. The monitoring server 26 is constituted by a computer including the processor 261, the main memory 262, the auxiliary storage unit 263, and the transmission line 265 connecting them.

[0104] The processor 261 corresponds to the central part of the above-mentioned computer. The processor 261 controls each part according to the operating system and the application program to realize various functions as the monitoring server 26. The processor 261 can be, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (for example, a simple programmable logic device (Simple Programmable Logic Device: SPLD), a complex programmable logic device (Complex Programmable Logic Device: CPLD), a field programmable gate array (Field Programmable Gate Array: FPGA) and other processing circuits. The processor 261 is not limited to being configured as a single processing circuit, and a plurality of processing circuits can be combined to form the processor 261.

[0105] The main memory 262 is equivalent to the main storage part of the above-mentioned computer. The main memory 262 includes a non-volatile memory area and a volatile memory area. The main memory 262 stores an operating system or an application program in the non-volatile memory area. The main memory 262 sometimes stores data required for the processor to perform processing for controlling each part in the non-volatile or volatile memory area. The main memory 262 uses the volatile memory area as a working area for the processor to rewrite data appropriately. For example, the non-volatile memory area is ROM (Read Only Memory). The volatile memory area is RAM (Random Access Memory).

[0106] The auxiliary storage unit 263 corresponds to the auxiliary storage part of the above-mentioned computer. For example, EEPROM (registered trademark) (Electric Erasable Programmable Read Only Memory), HDD (Hard Disc Drive) or SSD (Solid State Drive) is used as the auxiliary storage unit 263. The auxiliary storage unit 263 stores data used when the processor 261 performs various processes and data generated by the processing of the processor 261. The auxiliary storage unit 263 sometimes stores the above-mentioned application program.

[0107] The communication interface 264 performs data communication according to a predetermined communication protocol between each unit connected to the in-store network 27. As the communication interface 264, for example, a well-known communication device for LAN can be applied.

[0108] The transmission line 265 includes an address bus, a data bus, a control signal line, etc., and transmits data and control signals exchanged between the connected parts.

[0109] The auxiliary storage unit 263 includes a transaction data storage area 2631 , a photographing condition storage area 2632 , and a photographed image storage area 2633 .

[0110] The transaction data storage area 2631 stores first transaction data based on data stored in the virtual POS server 13. The first transaction data is an example of information related to the registration of goods via multiple smartphones 40. The information related to the registration of goods includes information related to the goods registered via the smartphone 40 before settlement, and information related to the registered goods via the smartphone 40. The information related to the goods includes information indicating the type of goods, the price of the goods, and the number of registered items. The type of goods includes information indicating goods with age restrictions such as alcohol and cigarettes. The price indicates the unit price of the goods. The number of registered items indicates the total number of goods registered by the member. The information related to the registration of goods includes information related to the operation of registering goods via the smartphone 40, and information related to the time of registering goods via the smartphone 40. The information related to the registration operation of goods includes information related to editing such as adding and deleting registered goods, information related to the operation of registering goods without barcodes, information related to the operation of transitioning to the checkout process, and location information of the place where the registration operation of goods was performed. The information related to time includes information indicating the stay time described later and the time of the previous operation.

[0111] The first transaction data includes a first index file and a first transaction file. The first transaction data includes data corresponding to the second transaction data stored in each smartphone 40 described later. The first index file is data associated with multiple first transaction files. The first index file is a collection of index file records made for each terminal ID. The first index file includes data corresponding to the second index file stored in each smartphone 40 described later. The index file record is a data record that associates the terminal ID, transaction ID, stay time, previous operation time, number of registrations, status, and checkout machine ID.

[0112] The terminal ID is a unique code set for each smartphone 40 in order to identify each smartphone 40. The transaction ID is a unique code set for each first transaction file in order to identify each first transaction file that records transactions performed using the smartphone 40. Whenever the first transaction file is created, the transaction ID is set in the first transaction file. The transaction ID is associated with the member ID and the store ID in the same manner as the collaboration table 70. The stay time indicates the time elapsed after the member logs in. The previous operation time indicates the time when the member last operated the smartphone 40. For example, the previous operation time indicates the time when the member registered the last commodity. The status is a terminal status indicating the status of the smartphone 40. The terminal status indicates "waiting", "registering" and "checking out". "Waiting" is a state from the completion of the checkout of the previous member to the login of the next member. "Registering" is a state from the login of the member to the approval of the checkout. The state before the checkout is approved is also called the state before the checkout process is performed. "Checking out" is a state from the approval to the completion of the checkout. The checkout machine ID is a unique code set for each checkout machine in order to identify each checkout machine. In addition to the above items, the first index file may also include information such as the total amount of the registered goods. The first index file is associated with the first transaction file using the transaction ID as a keyword. The first index file will be described in detail later.

[0113] The first transaction file is a collection of transaction file records generated for each transaction ID. The transaction file record is a data record that associates the transaction ID with the commodity sales data. The commodity sales data includes the commodity code, commodity type, commodity name, price, number of units sold, sales amount, etc. The first transaction file can also be data generated in the same way as the transaction file 80. Whenever a member registers a commodity, the first transaction file is updated. The first transaction file will be described in detail later.

[0114] The shooting condition storage area 2632 stores shooting conditions. The shooting conditions indicate conditions for shooting purchased goods through the smartphone 40. The shooting conditions include at least one of conditions related to goods and conditions related to goods registration. The shooting conditions are conditions related to goods or goods registration that are monitored.

[0115] The condition related to the commodity is a condition that becomes the comparison object of the information related to the commodity. The condition related to the commodity includes, for example, the number of items, the amount, and the type of the registered commodity. The condition related to the number of items of the commodity may be, for example, a number determined by "5 items" or the number at regular intervals such as "every increase of 5 items". In the case where the condition related to the number of items of the commodity is every increase of 5 items, the condition related to the number of items of the commodity includes the case where the registered number of items is a multiple of 5 such as 5 items, 10 items, and 15 items. It should be noted that the interval may not be fixed. The interval may be appropriately set such as reducing or expanding the interval according to the increase in the number of registered items. The condition related to the number of items of the commodity includes, for example, that the number of registered commodities initially reaches a prescribed number. The number of registered commodities initially reaches a prescribed number, for example, including that the number of registered commodities initially reaches 5 items, 10 items, and 15 items. The prescribed number includes multiples of 5, such as 5 items, 10 items, and 15 items. The prescribed number may be a number at regular intervals or a randomly set number. The condition related to the amount is, for example, a high-priced product such as "1,000 yen or more". The condition related to the type is, for example, "alcoholic beverages".

[0116] The conditions related to commodity registration include conditions related to commodity registration operation and conditions related to time. The conditions related to commodity registration operation are conditions that are the comparison objects of information related to commodity registration operation. The conditions related to commodity registration operation include, for example, conditions related to editing such as adding, deleting, and canceling registered commodities, conditions related to operations for registering commodities without barcodes, conditions related to operations for transitioning to checkout processing, and conditions related to location information of the place where commodity registration operation is performed. Conditions related to editing include, for example, "the number of sales of additional registered commodities" and "cancelling registered commodities". Conditions related to operations for registering commodities without barcodes include, for example, "the barcode-free button is operated". Conditions related to operations for transitioning to checkout processing include, for example, "the checkout button is operated". Conditions related to location information of the place where commodity registration operation is performed include, for example, "the location where commodity registration is performed is different from the display location of the registered commodity". Conditions related to time are conditions that are the comparison objects of information related to time. Conditions related to time include, for example, conditions related to stay time and conditions related to the previous operation time. Conditions related to stay time include, for example, "stay time exceeds a certain time". The condition related to the last operation time is, for example, “a certain time has passed since the last operation time.” The certain time can be set appropriately.

[0117] The photographing conditions may be preset, or may be appropriately set or updated by a store manager or the like.

[0118] The captured image storage area 2633 stores captured image management data. The captured image management data includes captured image data captured by the smartphone 40. The captured image data includes data of images of purchased goods captured by the member holding the smartphone 40. The captured image storage area 2633 may also store captured image data representing a plurality of captured images in a time series. Whenever captured image data is acquired by the acquisition unit 2612, the captured image management data is updated. The captured image storage area 2633 stores captured image management data associated with captured image data and information related to the shooting of the captured image data. The information related to the shooting includes information related to the shooting conditions such as "the 5th item" and "the 10th item", information related to the shooting such as the location of the shopping cart, the shooting time, and the like. The location of the shopping cart includes, for example, information such as the upper section and the lower section of the cart.

[0119] It should be noted that the hardware configuration of the monitoring server 26 is not limited to the configuration described above. The monitoring server 26 can omit or change the above-described components as appropriate, and can add new components.

[0120] (Configuration of Smartphone 40)

[0121] Fig.12 This is a block diagram schematically showing a configuration example of a smartphone 40 according to an embodiment.

[0122] The smartphone 40 includes a processor 401, a main memory 402, an auxiliary storage unit 403, a touch panel 404, a camera 405, a sound output device 406, a wireless communication unit 407, a mobile communication unit 408, and a transmission line 409. The processor 401 can communicate with the main memory 402, the auxiliary storage unit 403, the touch panel 404, the camera 405, the sound output device 406, the wireless communication unit 407, and the mobile communication unit 408 via the transmission line 409. In addition, the processor 401, the main memory 402, and the auxiliary storage unit 403 are connected via the transmission line 409, thereby constituting a computer for controlling the smartphone 40. It should be noted that the functions of the processor 401, the main memory 402, the auxiliary storage unit 403, and the transmission line 409 are roughly the same as those of the processor 261, the main memory 262, the auxiliary storage unit 263, and the transmission line 265, so their description is omitted. The smartphone 40 is an example of an information processing terminal.

[0123] The touch panel 404 is used as an operation device and a display device of the smartphone 40. The touch panel 404 is an example of a display unit that displays information or a notification unit that visually notifies information.

[0124] The camera 405 includes an optical system and an image sensor, and generates image data representing an image within a field of view formed by the optical system by the image sensor. The camera 405 is an example of an imaging unit.

[0125] The sound output device 406 is a device capable of outputting sound under the control of the processor 401. The sound output device 406 is an example of a sound output unit that outputs sound information or a notification unit that auditorily notifies information.

[0126] The wireless communication unit 407 exchanges data with the access point 29 through wireless communication based on a wireless communication protocol. As the wireless communication unit 407, for example, a known communication device compliant with the IEEE802.11 standard can be used.

[0127] The mobile communication unit 408 is an interface for data communication via the network 30. As the mobile communication unit 408, for example, a well-known communication device for performing data communication via a mobile communication network can be used.

[0128] The auxiliary storage unit 403 includes a smartphone POS application 4031 as one of the information processing programs, a transaction data storage area 4032, and a photographing condition storage area 4033. The auxiliary storage unit 403 is an example of a storage unit.

[0129] The smartphone POS application 4031 is an application program that describes information processing for making the smartphone 40 function as a user interface of the POS system 20. The smartphone POS application 4031 is commonly used by a plurality of smartphones 40.

[0130] The transaction data storage area 4032 stores second transaction data based on the data stored in the virtual POS server 13. The second transaction data is an example of information related to the registration of commodities via the smartphone 40. The second transaction data includes a second index file and a second transaction file. The second index file and the second transaction file include information about commodities registered by the smartphone 40. The second index file is composed of data that associates the terminal ID, transaction ID, stay time, previous operation time, number of registered items, status, checkout machine ID, etc.

[0131] The terminal ID is a unique code set to identify the smartphone 40. The transaction ID is a unique code set for each second transaction file to respectively identify the second transaction file recording the transaction performed using the smartphone 40. Whenever a second transaction file is created, the transaction ID is set in the second transaction file. The transaction ID is associated with the member ID and the store ID in the same manner as the collaboration table 70. The second index file includes an index file record associated with the terminal ID corresponding to each smartphone 40 in the first index file stored in the transaction data storage area 2631 of the monitoring server 26.

[0132] The second transaction file is data that associates the transaction ID with the commodity sales data. The commodity sales data includes the commodity code, the type of commodity, the commodity name, the price, the number of units sold, the sales amount, etc. The second transaction file may also be data generated in the same manner as the transaction file 80. The second transaction file is updated each time a member registers a commodity. The second transaction file includes a transaction file record that is associated with the transaction ID corresponding to each smartphone 40 in the first transaction file stored in the transaction data storage area 2631 of the monitoring server 26.

[0133] The shooting condition storage area 4033 stores shooting conditions. The shooting conditions are shooting conditions received from the monitoring server 26. The shooting conditions may be shooting conditions corresponding to the user (customer) of the smartphone 40, the store where the smartphone 40 is used, the time when the smartphone 40 is used, etc., among the shooting conditions stored in the shooting condition storage area 2632 of the monitoring server 26. The shooting conditions may also be set to different conditions based on the store, the user, the time of entering the store, etc. It should be noted that the shooting conditions may also be stored in the main memory 402.

[0134] It should be noted that the hardware configuration of the smartphone 40 is not limited to the configuration described above. The smartphone 40 can appropriately omit or change the above-described components or add new components.

[0135] Each unit implemented in the processor 401 will be described.

[0136] The processor 401 is provided with a receiving unit 4010, a determining unit 4011, a display control unit 4012, a photographing control unit 4013, an acquiring unit 4014, a storage control unit 4015, and a transmitting unit 4016. Each unit provided in the processor 401 may also be referred to as each function. Each unit provided in the processor 401 may also be provided in a control unit including the processor 401 and the main memory 402.

[0137] The receiving unit 4010 receives the photographing conditions from the monitoring server 26 via the wireless communication unit 407 before the customer starts shopping. Before the customer starts shopping, for example, before the customer can register the product. Before the customer starts shopping, for example, when the smartphone 40 receives the store entry approval command CMc from the management server 12. In this case, the receiving unit 4010 receives the photographing conditions from the monitoring server 26 based on the receipt of the store entry approval command CMc.

[0138] The determination unit 4011 determines whether the second transaction data satisfies the imaging condition.

[0139] The display control unit 4012 displays the automatic shooting guide of the product on the touch panel 404 based on the determination result of the determination unit 4011. The automatic shooting guide is, for example, a notification indicating that the product is automatically photographed. The automatic shooting guide is, for example, text information such as "will be photographed", "will be photographed in 5 seconds", etc. The automatic shooting guide may include any notification such as a symbol, an icon, a sound, a dynamic image, etc. If the automatic shooting guide is a sound, the processor 401 outputs the automatic shooting guide to the sound output device 406. If the notification of automatic shooting is a dynamic image, the processor 401 outputs the automatic shooting guide to the touch panel 404 and the sound output device 406. The automatic shooting guide may include a shooting guide. The shooting guide includes a guide indicating the shooting range. The shooting guide may include, for example, text information such as "please move the product into the shooting range". The shooting guide includes a guide that prompts to shoot multiple shopping baskets. The shooting guide may include text information such as "Is there a shopping basket in the lower section of the cart?", "shoot the basket in the lower section. Please move the camera so that the basket in the lower section enters the shooting range". The customer moves the smartphone 40 according to the shooting guide so that the basket in the lower section enters the shooting range of the camera 405 .

[0140] It should be noted that the display control unit 4012 may also display a notification urging shooting on the touch panel 404 instead of the automatic shooting guide. The notification urging shooting is, for example, a text message such as "Please shoot" for urging customers to shoot. The notification urging shooting may include any notification such as a symbol, an icon, a sound, a dynamic image, etc. When the notification urging shooting is a sound, the processor 401 outputs instructions related to shooting of the product to the sound output device 406. When the notification urging shooting is a dynamic image, the processor 401 outputs instructions related to shooting of the product to the touch panel 404 and the sound output device 406.

[0141] Based on the determination by the determination unit 4011 that the second transaction data satisfies the photographing condition, the photographing control unit 4013 outputs a photographing instruction of the commodity to the camera 405. The photographing instruction of the commodity is an instruction from the processor 401 to control the camera 405. The processor 401 activates the camera 405 in response to the photographing instruction of the commodity, and controls the camera 405 to automatically photograph the commodity.

[0142] The acquisition unit 4014 acquires the captured image data of the captured commodity based on the automatic capturing of the commodity. The acquisition unit 4014 acquires the captured image data of the captured commodity based on the automatic capturing of the commodity under the control of the capturing control unit 4013.

[0143] The storage control unit 4015 stores the captured image data acquired by the acquisition unit 4014 in the auxiliary storage unit 403. When the captured image data is transmitted to the monitoring server 26 by the transmission unit 4016 described later, the storage control unit 4015 deletes the captured image data. The storage control unit 4015 holds the captured image data until the transmission unit 4016 transmits the captured image data to the monitoring server 26. The storage control unit 4015 may delete the captured image data after a certain time has passed since the captured image data was acquired. The certain time is, for example, one day.

[0144] The sending unit 4016 sends the captured image data acquired by the acquiring unit 4014 to the monitoring server 26. When the captured image data cannot be sent to the monitoring server 26, the sending unit 4016 sends the captured image data that cannot be sent to the monitoring server 26 again. When the captured image data acquired in the past is stored in the auxiliary storage unit 403, the sending unit 4016 sends the captured image data acquired in the past to the monitoring server 26. The captured image data acquired in the past is captured image data acquired in the past and not sent to the monitoring server 26. The captured image data acquired in the past may be past captured image data captured during the same transaction or past captured image data captured during other transactions. The other transactions may be, for example, different transactions on the same day or transactions on another day.

[0145] (Configuration of Monitoring Terminal 23)

[0146] Fig.13 This is a block diagram schematically showing a configuration example of the monitoring terminal 23 according to one embodiment.

[0147] The monitoring terminal 23 includes a processor 231, a main memory 232, an auxiliary storage unit 233, a touch panel 234, a sound output device 235, a communication interface 236, and a transmission line 237. The processor 231 and the main memory 232, the auxiliary storage unit 233, the touch panel 234, the sound output device 235, and the communication interface 236 can communicate via the transmission line 237. In addition, the processor 231, the main memory 232, and the auxiliary storage unit 233 are connected via the transmission line 237, thereby forming a computer for controlling the monitoring terminal 23. It should be noted that the functions of the processor 231, the main memory 232, the auxiliary storage unit 233, and the transmission line 237 are roughly the same as those of the processor 261, the main memory 262, the auxiliary storage unit 263, and the transmission line 265, and therefore, their description is omitted.

[0148] The touch panel 234 functions as an operation device and a display device of the monitoring terminal 23. The touch panel 234 is an example of a display unit that displays information or a notification unit that visually notifies information.

[0149] The sound output device 235 is a device capable of outputting sound under the control of the processor 231. The sound output device 235 is an example of a sound output unit that outputs sound information or a notification unit that auditorily notifies information.

[0150] The communication interface 236 performs data communication according to a predetermined communication protocol between each unit connected to the in-store network 27. As the communication interface 236, for example, a known communication device for LAN can be applied.

[0151] It should be noted that the hardware configuration of the monitoring terminal 23 is not limited to the configuration described above. The monitoring terminal 23 can omit or change the above-described components as appropriate, and can add new components.

[0152] Each unit mounted on the processor 231 will be described.

[0153] The processor 231 is implemented with a receiving unit 2310 and a display control unit 2311. Each unit implemented in the processor 231 may also be referred to as each function. Each unit implemented in the processor 231 may also be implemented in a control unit including the processor 231 and the main memory 232.

[0154] The receiving unit 2310 receives captured image data from the captured image storage area 2633 of the monitoring server 26 via the access point 29. The receiving unit 2310 updates the captured image data every time it receives captured image data from the smartphone 40.

[0155] The display control unit 2311 displays a waiter image on the touch panel 234 based on the captured image data. The waiter image includes, for example, a registered product detail screen and a registered product confirmation screen. The registered product detail screen shows the details of the registered products of each customer. The registered product detail screen includes, for example, information such as the registration order of the products, the product name, the number of registered products, and the amount. The registered product confirmation screen shows the image of the captured product. The registered product confirmation screen includes, for example, a thumbnail image of the image of the captured product, an enlarged image of the image of the selected product, and the like. Whenever the captured image data is updated, the display control unit 2311 updates the waiter image. The display control unit 2311 displays the captured image based on the captured image data on the touch panel 234 according to the display instructions of the waiter. The waiter may also be referred to as a clerk, a user, or a person.

[0156] A configuration example of the first index file will be described.

[0157] Fig.14 26 is a diagram showing a main data structure of a first index file stored in the transaction data storage area 2631 according to an embodiment.

[0158] The first index file includes a "terminal ID" item, a "transaction ID" item, a "stay time" item, a "previous operation time" item, a "number of registrations" item, a "status" item, and a "checkout machine ID" item. Each of the multiple terminal IDs is associated with the transaction ID, the stay time, the previous operation time, the number of registrations, the status, and the checkout machine ID as an index file record. For example, the smartphone 40 with terminal ID "00001" indicates that it is associated with transaction ID "1". The smartphone 40 with terminal ID "00001" indicates that the elapsed time after login, that is, the stay time, is "8 minutes", and the last time the smartphone 40 was operated, that is, the previous operation time, is "14:10". The number of registrations of the smartphone 40 is "5", and the status indicates that it is "registering" before the checkout process is performed. In this example, since the smartphone 40 with terminal ID "00001" is in the state before the checkout process is performed, it is not associated with the checkout machine, and the checkout machine ID is a blank column. The information contained in the first index file can be set appropriately. The monitoring server 26 updates the first index file appropriately. It should be noted that the configuration example of the second index file stored in the transaction data storage area 4032 of the smartphone 40 only includes the index file records related to the corresponding smartphone 40 in the configuration example of the first index file. For example, the smartphone 40 with the terminal ID "00001" only stores the index file records associated with the terminal ID "00001" in the transaction data storage area 4032.

[0159] An example of the structure of the first transaction file will be described.

[0160] Fig.15 26 is a diagram showing a main data structure of a first transaction file stored in the transaction data storage area 2631 according to an embodiment.

[0161] The first transaction file includes a "transaction ID" item, a "product code" item, a "product name" item, a "product type" item, a "price" item, a "number of units sold" item, and a "sales amount" item. Each of the multiple transaction IDs is associated with the transaction ID, product code, product type, product name, price, number of units sold, and sales amount as a transaction file record. The first index file and the first transaction file are associated with each other using "transaction ID" as a keyword. The information contained in the first transaction file can be appropriately set. The monitoring server 26 updates the first transaction file appropriately. It should be noted that the configuration example of the second transaction file stored in the transaction data storage area 4032 of the smartphone 40 only includes the information related to the corresponding smartphone 40 in the configuration example of the above-mentioned first transaction file. For example, the smartphone 40 with terminal ID "00001" only stores the first transaction file record of the "transaction ID" associated with the terminal ID "00001" in the transaction data storage area 4032.

[0162] A configuration example of captured image management data will be described.

[0163] Fig.16 This is a diagram showing a main data structure of captured image management data stored in the captured image storage area 2633 according to one embodiment.

[0164] The captured image management data includes a "transaction ID" item and an "image" item. The "image" item sets the captured image data. Each of the multiple transaction IDs is associated with the captured image data as a captured image data record. The monitoring server 26 may also store the captured image data in the captured image storage area 2633 in a time series according to the time when the captured image data is obtained. The captured image management data is associated with the first index file and the first transaction file using the "transaction ID" as a keyword. The monitoring server 26 appropriately updates the captured image management data each time the captured image data is obtained.

[0165] (Boot screen)

[0166] Fig.17 1 is a schematic diagram showing an example of a guidance screen Sf displayed on the smartphone 40 according to an embodiment.

[0167] In response to the photographing condition being satisfied, the smartphone 40 outputs automatic photographing guidance for the product to the touch panel 404 . Fig.17 Indicates that the guide screen Sf for automatic shooting guidance is displayed. Fig.17As shown, a frame line ARb indicating a photographing area of ​​a commodity and an area ARc indicating an automatic photographing guide are displayed on the guidance screen Sf.

[0168] For example, the frame line ARb is set to accommodate the entire shopping basket. The member operates the smartphone 40, for example, so that the purchased goods put into the shopping basket enter the frame line ARb. According to this example, the smartphone 40 can prompt the member to shoot the goods along the frame line ARb. Thereby, the smartphone 40 can obtain the captured image data shot within a somewhat uniform range. Therefore, the deviation of the captured image of the goods is reduced. The waiter who confirms the captured image of the goods in the monitoring terminal 23 can easily compare the multiple captured images taken in time series.

[0169] The area ARc is, for example, an area including a display indicating the timing of automatic photography. The area ARc may include, for example, a countdown display until the timing of automatic photography. Fig.17 The countdown display includes "5" 5 seconds before the timing of automatic shooting. The countdown display changes the display mode according to the passage of time. The countdown display can be a digital display, or a display with different display modes such as a mark, icon, or bar according to the passage of time.

[0170] It should be noted that the smartphone 40 may not display the frame line ARb. In this case, the smartphone 40 may automatically detect the area of ​​the shopping basket and set the shopping basket to be within the shooting range of the camera 405.

[0171] It should be noted that the guide screen Sf may also include a shooting button. The member can manually shoot the product by touching the shooting button. For example, the member touches the shooting button when the purchased product is within the frame ARb. When the member touches the shooting button, the purchased product is shot through the camera 405 of the smart phone 40.

[0172] (Operation of smartphone 40)

[0173] The procedure of information processing by the smartphone 40 will be described.

[0174] It should be noted that the processing steps described below are only examples, and each process can be changed as much as possible. In addition, regarding the processing steps described below, according to the embodiment, steps can be appropriately omitted, replaced, and added.

[0175] The following processing is performed during shopping.

[0176] The following processing is performed by the user (customer) of the smartphone 40 to register the product before the transition to the checkout process. The customer registers the product by reading the code symbol attached to the product displayed in the store through the camera 405. Every time a member registers a product, the information of the registered product is stored in the virtual POS server 13. The information of the registered product is also stored in the transaction data storage area 4032 of the smartphone 40 that cooperates with the virtual POS server 13. Assume that the customer uses Fig.14 The smartphone 40 with the terminal ID “00001” in the transaction data storage area 4032 registers the commodity. The smartphone 40 with the terminal ID “00001” stores only the transaction data of the smartphone 40 with the terminal ID in the transaction data storage area 4032 .

[0177] In the following example, the case where the shooting condition is a condition related to the number of items of goods is described. It is assumed that the condition related to the number of items of goods is set to the case where the number of items of registered goods first reaches a specified number. The specified number is a multiple of 5, that is, 5 items, 10 items, and 15 items.

[0178] Fig.18 1 is a flowchart showing an example of the steps of information processing executed by the smartphone 40 according to one embodiment.

[0179] The determination unit 4011 determines whether the second transaction data satisfies the shooting condition (ACT1). In ACT1, for example, the determination unit 4011 refers to the index file record associated with the terminal ID "00001" stored in the second index file, and compares the conditions related to the "number of registrations" and the number of goods. In the case where the determination unit 4011 determines that the "number of registrations" of the goods registered in the second index file satisfies the shooting condition (YES in ACT101), the processing is transferred from ACT101 to ACT102. In the case where the determination unit 4011 determines that the "number of registrations" of the goods registered in the second index file does not satisfy the shooting condition (NO in ACT101), the processing of ACT101 is repeated.

[0180] For example, the determination unit 4011 compares the number of registrations "5" and the condition related to the number of commodities from the second index file associated with the terminal ID "00001". In this example, the determination unit 4011 determines that the "number of registrations" of the commodities registered in the second index file meets the shooting condition, and enters YES in ACT101.

[0181] The display control unit 4012 displays the automatic shooting guidance of the product on the touch panel 404 according to the determination result of the determination unit 4011 (ACT102). In ACT102, for example, the display control unit 4012 causes the touch panel 404 to display a notification indicating automatic shooting. The display control unit 4012 may also display a countdown indicating the timing of automatic shooting. It should be noted that, when the automatic shooting guidance is sound, the processor 401 outputs the automatic shooting guidance to the sound output device 406. If the automatic shooting guidance is a dynamic image, the processor 401 outputs the automatic shooting guidance to the touch panel 404 and the sound output device 406.

[0182] The shooting control unit 4013 controls the camera 405 to shoot the product (ACT103). In ACT103, for example, the shooting control unit 4013 outputs a shooting instruction of the product to the camera 405. The shooting control unit 4013 may also shoot the product in response to obtaining the determination result performed by the determination unit 4011. The shooting control unit 4013 may also shoot the product based on the timing of reaching automatic shooting. The timing of automatic shooting may be after a specified time has passed after the automatic shooting guide is displayed. The specified time is, for example, 5 seconds. It should be noted that the shooting control unit 4013 may also determine whether the product can be shot by automatic shooting. When the shooting control unit 4013 determines that it can be shot, the processing migrates from ACT103 to ACT104. When the shooting control unit 4013 determines that it cannot be shot, the processing returns from ACT103 to ACT102. In this case, the display control unit 4012 displays the automatic shooting guide of the product on the touch panel 404. The smartphone 40 may repeatedly perform the processes from ACT 102 to ACT 103 until the image of the product is successfully captured.

[0183] It should be noted that the smartphone 40 may repeatedly perform the processing of ACT 102 to ACT 103 until all shopping baskets are photographed. In this case, the display control unit 4012 may display a photographing guide urging photographing of the lower section on the touch panel 404 when there is also a shopping basket in the lower section of the shopping cart. The photographing guide urging photographing of the lower section may be, for example, "If there is a second basket, press the 'Photograph the next basket' button". Fig.19 Shows an example of the shooting guide display. Fig.19The shooting guidance screen Sfa shown includes text information indicating shooting guidance, a "Reshoot" button Baa, a "Shoot Next Basket" button Bab, and a "Finish" button Bac. The text information includes, for example, information such as "If there is a second basket, please press the 'Shoot Next Basket' button." The "Reshoot" button Baa is a button for the customer to reshoot the shopping basket. The "Shoot Next Basket" button Bab is a button for shooting the basket in the lower section if there is a basket in the lower section. The "Finish" button Bac is a button for completing the shooting. For example, when there is a basket in the lower section, the customer selects the "Shoot Next Basket" button Bab according to the shooting guide. In response to the customer selecting the "Shoot Next Basket" button Bab, the display control unit 4012 displays a notification on the touch panel 404 indicating that the next basket is automatically shot.

[0184] The acquisition unit 4014 acquires the captured image data of the captured product based on the automatic shooting of the product (ACT104). In ACT104, for example, the acquisition unit 4014 acquires the captured image data of the product shot by the camera 405. The acquisition unit 4014 acquires additional information related to the captured image data. The additional information includes information such as the time when the captured image data was captured, the shooting conditions, and the position of the basket. The additional information includes, for example, information such as "15:30", "the fifth registered product", and "the upper section". The acquisition unit 4014 acquires the initial captured image data shot by the camera 405 as the captured image data of the upper section. The acquisition unit 4014 can also acquire the captured image data shot by the camera 405 as the captured image data of the lower section by the customer operating the "shoot next basket" button.

[0185] The storage control unit 4015 stores the acquired product image data in the auxiliary storage unit 403 (ACT 105). In ACT 105, for example, the storage control unit 4015 associates the acquired product image data with the transaction ID and additional information and stores them in the auxiliary storage unit 403 in time series.

[0186] The sending unit 4016 sends the captured image data acquired by the acquisition unit 4014 to the monitoring server 26 (ACT 106). In ACT 106, for example, the sending unit 4016 sends the captured image data in association with the transaction ID and the additional information to the monitoring server 26. The monitoring server 26 outputs the captured image data, the transaction ID, and the additional information to the monitoring terminal 23. The monitoring terminal 23 generates a waiter image based on the acquired image data and displays it on the touch panel of the monitoring terminal 23. The monitoring image includes a captured image of the commodity associated with the transaction data. The monitoring image may include a plurality of captured images based on the captured image data captured at different timings in a time series. The sending unit 4016 may also compress the captured image data and send it to the monitoring server 26.

[0187] It should be noted that the processor 401 repeats the processing of ACT101 to ACT106 every time the second transaction data satisfies the shooting condition during a transaction. In this way, when the processor 401 repeats the processing of ACT101 to ACT106 multiple times during a transaction, the monitoring image can include a plurality of captured images based on the captured image data captured at different timings in a time series. In this case, the waiter who confirms the captured image of the commodity in the monitoring terminal 23 can confirm the change in the state in the shopping cart along with the change in the registered number of commodities along with the increase in the number of registered items. Here, the case where the shooting condition is a condition related to the number of commodities that "initially reaches a specified number" is described as an example, but it is not limited to this. For example, when the shooting condition is "reaching a specified number", the processor 401 may also shoot the commodity every time the registered number of commodities reaches a specified number. For example, when the specified number is "5", the commodity may be shot when the registered number of commodities reaches 5, the commodity registration is cancelled, and the registered number reaches "5" again. Even when the imaging condition is any of the above-mentioned conditions other than the condition related to the number of commodities, the processor 401 repeats the processing of ACT 101 to ACT 106 every time the second transaction data satisfies the imaging condition during one transaction.

[0188] Fig. 20 This is a flowchart showing another example of the steps of information processing by the smartphone according to one embodiment.

[0189] The following processing is information processing related to the transmission of captured image data.

[0190] The sending unit 4016 determines whether the sending of the captured image data to the monitoring server 26 is successful (ACT 201). In ACT 201, for example, the sending unit 4016 determines whether the captured image data is sent to the monitoring server 26 within a predetermined time. When the sending unit 4016 determines that the sending of the captured image data is successful (YES in ACT 201), the process moves from ACT 201 to ACT 203. When the sending unit 4016 determines that the sending of the captured image data is unsuccessful (NO in ACT 201), the process moves from ACT 201 to ACT 202. The sending unit 4016 resends the captured image data that failed to be sent.

[0191] The sending unit 4016 determines whether the retransmission of the captured image data to the monitoring server 26 is successful (ACT 202). In ACT 202, for example, the sending unit 4016 determines whether the captured image data is sent to the monitoring server 26 within a predetermined time. When the sending unit 4016 determines that the retransmission of the captured image data is successful (YES in ACT 202), the process moves from ACT 202 to ACT 203. When the sending unit 4016 determines that the retransmission of the captured image data is unsuccessful (NO in ACT 202), the process ends.

[0192] The sending unit 4016 determines whether the captured image data acquired in the past is stored in the auxiliary storage unit 403 (ACT 203). For example, the case where the captured image data captured at the time when the number of registered items is 10 is successfully transmitted, and the captured image data captured at the time when the number of registered items is 5 is not transmitted to the monitoring server 26 is described. The auxiliary storage unit 403 stores the captured image data captured at the time when the number of registered items is 5. The sending unit 4016 acquires the captured image data captured at the time when the number of registered items is 5 from the auxiliary storage unit 403.

[0193] When the sending unit 4016 determines that the captured image data acquired in the past is stored in the auxiliary storage unit 403 (YES in ACT 203), the process moves from ACT 13 to ACT 14. When the sending unit 4016 determines that the captured image data acquired in the past is not stored in the auxiliary storage unit 403 (NO in ACT 203), the process moves from ACT 13 to ACT 15.

[0194] The sending unit 4016 sends the acquired captured image data to the monitoring server 26 (ACT 204). In ACT 204, for example, the sending unit 4016 sends the captured image data to the monitoring server 26 in association with the transaction ID and the additional information. The sending unit 4016 sends the captured image data captured at the time when the number of registered items is 5 to the monitoring server 26. The monitoring server 26 outputs the captured image data, the transaction ID, and the additional information to the monitoring terminal 23. The monitoring terminal 23 updates the waiter image based on the acquired image data.

[0195] The storage control unit 4015 deletes the captured image data transmitted by the transmission unit 4016 from the auxiliary storage unit 403 (ACT 205).

[0196] It should be noted that, when the sending unit 4016 determines that the sending or resending of the captured image data to the monitoring server 26 is unsuccessful, the display control unit 4012 may also pop up a warning message on the touch panel 404. The warning message includes text information such as "Image not sent", "Please call the clerk", etc. The sending unit 4016 may also determine whether there is captured image data that failed to be sent when the customer presses the settlement button. If the sending unit 4016 determines that there is captured image data that failed to be sent, the display control unit 4012 may display a warning message on the touch panel 404. The waiter can confirm the registered goods by the monitoring terminal 23 based on the warning message. After the confirmation work performed by the waiter, the smartphone 40 can perform the processing for settlement. The processing for settlement includes, for example, the checkout processing of the smartphone 40 or the display of the checkout code.

[0197] (Waiter screen)

[0198] Fig.21 This is a schematic diagram showing an example of a registered product list screen Sg displayed on the monitoring terminal 23 according to one embodiment.

[0199] The monitoring terminal 23 displays a registered commodity detail screen Sg on the touch panel 234 based on the captured image data acquired from the monitoring server 26. The registered commodity detail screen Sg is an example of a waiter image. The monitoring terminal 23 is used by a waiter.

[0200] The registered product details screen Sg includes the registered product information such as the registration order, product name, quantity, amount, and remarks of the product. The registered product details screen Sg can also be a list of registered product information. The registration order of the product indicates the order in which the product is registered by the member. The quantity indicates the number of registered products. The registered product information includes a mark Ma and a mark Mb. The mark Ma indicates a product such as an age confirmation product that requires confirmation by the waiter. The mark Mb indicates a product that meets the shooting conditions. The registered product details screen Sg includes a confirmation button Bg and an OK button Bh.

[0201] By selecting the mark Mb, the waiter can confirm the detailed information of the product assigned with the mark Mb. The detailed information of the product includes, for example, a photographic image of the product. In response to the selection of the mark Mb by the waiter, the monitoring terminal 23 may also cause the display screen of the touch panel 234 to migrate from the registered product detail screen Sg to the registered product confirmation screen. It should be noted that the monitoring terminal 23 may also cause the display screen of the touch panel 234 to migrate from the registered product detail screen Sg to the registered product confirmation screen in response to the selection of the area for displaying the product name by the waiter. The area for displaying the product name may be a row containing the product name of the registered product detail list. The selection of the area for displaying the product name by the waiter is equivalent to selecting a record of the registered product. The monitoring terminal 23 may also cause the display screen of the touch panel 234 to migrate from the registered product detail screen Sg to the registered product confirmation screen based on the record of selecting the registered product. In this case, the monitoring terminal 23 displays a photographic image taken at a time close to the registration time of the selected registered product in the photographic image area ARe of the registered product confirmation screen. It should be noted that the monitoring terminal 23 may change the display screen of the touch panel 234 from the registered product detail screen Sg to the registered product confirmation screen in response to the selection of the mark Ma by the service staff.

[0202] By selecting the OK button Bh, the waiter can return to the original screen from the registration product details screen Sg. In response to the waiter selecting the OK button Bh, the monitoring terminal 23 changes the display screen of the touch panel 234 from the registration product details screen Sg to an original screen (not shown), which may be, for example, an image including the registration status of a plurality of members.

[0203] The waiter can confirm the detailed information of the product by selecting the confirmation button Bg. In response to the waiter selecting the confirmation button Bg, the monitoring terminal 23 causes the display screen of the touch panel 234 to transition from the registered product detail screen Sg to the registered product confirmation screen.

[0204] Fig. 22 This is a schematic diagram showing an example of the registered commodity confirmation screen Sh displayed on the monitoring terminal 23 according to one embodiment.

[0205] In response to the attendant selecting the mark Mb on the registered product detail screen Sg, the monitoring terminal 23 displays the registered product confirmation screen Sh on the touch panel 234. The monitoring terminal 23 may display the registered product confirmation screen Sh on the touch panel 234 in response to the selection of the area displaying the product name. The registered product confirmation screen Sh is an example of an attendant image.

[0206] The registered commodity confirmation screen Sh includes a thumbnail image area ARd and a captured image area ARe. The registered commodity confirmation screen Sh includes a return button Bi and an OK button Bj.

[0207] The thumbnail image area ARd includes a thumbnail image of a photographed image based on photographed image data of a commodity. The thumbnail image may include identification information capable of identifying the photographed image data. The identification information includes, for example, photographing conditions. Fig. 22 In the example, each thumbnail image includes the number of registered items of the product as identification information. The identification information may include information indicating the location of the shopping cart. For example, the identification information includes "the 5th item", "upper section", etc. The identification information may include information indicating the condition in which the image was captured. The condition in which the image was captured includes, for example, product cancellation, cancellation correction, checkout, etc. The identification information is included in the additional information related to the captured image data. The monitoring terminal 23 may also display a thumbnail image corresponding to the image displayed in the captured image area ARe in a recognizable display manner. Fig. 22 In the example of , the monitoring terminal 23 displays the thumbnail image corresponding to the image of "item 10" displayed in the captured image area ARe in a manner that allows identification. The thumbnail image area ARd is not limited to thumbnail images, but may also include a mark or icon that allows identification of the captured image.

[0208] The captured image area ARe includes a captured image at the time when the product selected by the waiter was captured. The captured image area ARe may also be an enlarged image of the captured image. The waiter can confirm the captured image at the time when the product selected according to the image displayed in the captured image area ARe was captured. The waiter can confirm the captured image and the number of items of the registered products and the number of items included in the shopping cart.

[0209] The attendant can select a thumbnail image included in the thumbnail image area ARd, and a photographic image corresponding to the selected thumbnail image is displayed in the photographic image area ARe. In response to the attendant selecting a thumbnail image, the monitoring terminal 23 displays the corresponding photographic image in the photographic image area ARe.

[0210] By selecting the return button Bi, the waiter can return to the original screen. The original screen may be the registered product detail screen Sg or the registered product confirmation screen when the waiter selects another product. In response to the waiter selecting the return button Bi, the monitoring terminal 23 causes the display screen of the touch panel 234 to transition from the registered product confirmation screen Sh to the original screen.

[0211] By selecting the OK button Bj, the attendant can return from the registered product confirmation screen Sh to the registered product detail screen Sg. In response to the attendant selecting the OK button Bj, the monitoring terminal 23 causes the display screen of the touch panel 234 to transition from the registered product confirmation screen Sh to the registered product detail screen Sg.

[0212] Fig.23 1 is a schematic diagram showing an example of a registered commodity confirmation screen Si displayed on the monitoring terminal 23 according to one embodiment.

[0213] In response to the waiter selecting the confirmation button Bg on the registered commodity detail screen Sg, the monitoring terminal 23 displays the registered commodity confirmation screen Si on the touch panel 234. The registered commodity confirmation screen Si is an example of a waiter image.

[0214] Registered product confirmation screen Si and Fig. 22 Similarly, the thumbnail image area ARf and the captured image area ARg are included. In this example, the monitoring terminal 23 displays the latest captured image in the captured image area ARg. Fig. 22 Similarly, the registered product confirmation screen Si includes a return button Bi and an OK button Bj. The latest captured image is the latest captured image at the time of capturing in time series.

[0215] The thumbnail image area ARf includes thumbnail images of images obtained by reducing the size of the images based on the image data of the product, similarly to the thumbnail image area ARd. The monitoring terminal 23 may also display thumbnail images corresponding to the images displayed in the image area ARg in a recognizable display manner. Fig.23 In the example of FIG. 1 , the monitoring terminal 23 displays the thumbnail image corresponding to the latest captured image displayed in the captured image area ARg in a display manner that allows identification. The thumbnail image area ARf is not limited to thumbnail images, but may include a mark or icon that allows identification of the captured image.

[0216] The captured image area ARg includes the latest captured image. The captured image area ARg may also be an enlarged image of the captured image. The waiter can confirm the latest captured image of the product through the image displayed in the captured image area ARg. The waiter can confirm the captured image and the number of registered products and the number of products included in the shopping cart.

[0217] The attendant can select a thumbnail image included in the thumbnail image area ARf, and a photographic image corresponding to the selected thumbnail image is displayed in the photographic image area ARg. In response to the attendant selecting a thumbnail image, the monitoring terminal 23 displays the corresponding photographic image in the photographic image area ARg.

[0218] (Operation of monitoring terminal 23)

[0219] The processing procedure of the monitoring terminal 23 is described.

[0220] Fig.24 This is a flowchart showing an example of the procedure of information processing by the monitoring terminal 23 according to one embodiment.

[0221] It should be noted that the processing steps described below are only examples, and each process can be changed as much as possible. In addition, regarding the processing steps described below, according to the embodiment, steps can be appropriately omitted, replaced, and added.

[0222] The following processing is performed during the above shopping.

[0223] The receiving unit 2310 receives the captured image data from the monitoring server 26 via the in-store network 27. The receiving unit 2310 determines whether the captured image data received from the monitoring server 26 is updated (ACT 301). When the receiving unit 2310 determines that the captured image data is updated (YES in ACT 301), the process shifts from ACT 301 to ACT 302. When the receiving unit 2310 determines that the captured image data is not updated (NO in ACT 301), the process repeats ACT 301.

[0224] The display control unit 2311 updates the waiter image based on the captured image data (ACT 302). In ACT 302, for example, the display control unit 2311 displays a mark indicating that the captured image data exists for the products that meet the shooting conditions in the registered product details screen. The mark can be, for example, an icon indicating that the captured image data exists, or a sign. The display control unit 2311 is not limited to the mark, as long as it is a display method that can identify the products that meet the shooting conditions. The display method includes text color, size, font, surrounding text, text decoration such as brightness or brilliance, background color, background brightness or brilliance, background pattern, frame color, flashing display, etc.

[0225] The display control unit 2311 determines whether the waiter has input a display instruction for the registration product confirmation screen (ACT303). The display instruction includes a button input by the waiter in the registration product details screen for displaying the registration product confirmation screen. The display instruction includes an area selected by the waiter in the registration product details screen to represent products that meet the shooting conditions. The area representing the products that meet the shooting conditions is equivalent to the area or mark Mb displaying the product name on the registration product details screen Sg. The display instruction includes a record of the registration product selected by the waiter. For example, when the display control unit 2311 determines that the display instruction has been input (YES in ACT303), the processing migrates from ACT303 to ACT304. When the receiving unit 2310 determines that the display instruction has not been input (NO in ACT303), the processing ends.

[0226] The display control unit 2311 displays a registered product confirmation screen on the touch panel 234 based on the display instruction (ACT 304). In ACT 304, for example, the display control unit 2311 displays a different registered product confirmation screen on the touch panel 234 according to the display instruction. For example, the display instruction is to indicate the selection of an area of ​​products that meet the shooting conditions. The display control unit 2311 displays a different registered product confirmation screen on the touch panel 234. Fig. 22 The registered product confirmation screen Sh is shown. The registered product confirmation screen includes, for example, thumbnail images of the captured images arranged in time series in the order that satisfies the shooting conditions. The registered product confirmation screen includes an enlarged image of the captured image of the product selected by the waiter. The description display instruction is a case where a button for displaying the registered product confirmation screen is input. The button for displaying the registered product confirmation screen is equivalent to the confirmation button Bg of the registered product detail screen Sg. The display control unit 2311 displays on the touch panel 234 the following information: Fig.23 The registered commodity confirmation screen shown. The registered commodity confirmation screen includes, for example, thumbnail images of the photographed images arranged in time series in the order satisfying the photographing conditions. The registered commodity confirmation screen may include an enlarged image of the latest photographed image.

[0227] It should be noted that the monitoring terminal 23 can save the captured image data in the auxiliary storage unit 233, display the waiter image on the touch panel 234 based on the saved captured image data, and can also display the captured image data saved in the monitoring server 26 on the touch panel 234 through the Web client function.

[0228] (Effects of the transaction processing system 100)

[0229] According to the present embodiment, the smartphone 40 determines whether the information related to the registration of the product meets the shooting condition, displays the automatic shooting guide of the product based on the determination result, obtains the shooting image data of the photographed product based on the automatic shooting of the product, and sends the obtained shooting image data. As a result, the smartphone 40 can automatically shoot the products in the shopping cart when the shooting condition is met, and send the captured image to the monitoring server 26. Therefore, the smartphone 40 can remind the customer not to forget to register the product or register it improperly. In addition, the smartphone 40 can display the automatic shooting guide of the product to notify the customer to shoot the product. Since the customer can know the timing when the product is photographed, forgetting to register the product and improper registration can be prevented before it happens. In this way, the smartphone 40 can prevent the customer from forgetting to register the product or registering it improperly. In addition, the monitoring server 26 can manage the shooting image data of the automatic shooting. The waiter can confirm the customer's registered product based on the shot image based on the waiter image displayed on the monitoring terminal 23.

[0230] According to the present embodiment, the smartphone 40 can automatically photograph the goods when the number of registered goods initially reaches a specified number. Thus, the smartphone 40 can omit automatic photographing of the goods when the same photographing condition is met multiple times. For example, when the photographing condition is that the number of registered goods is 5, after the number of registered goods reaches 5, if the customer changes the number of registered goods by cancelling, etc., the smartphone 40 will not automatically photograph when the number of goods reaches 5 again. Therefore, when the smartphone 40 meets the same photographing condition multiple times, the automatic photographing process can be omitted. In addition, the smartphone 40 can eliminate the trouble that customers feel due to multiple automatic photographing.

[0231] According to this embodiment, the smartphone 40 can receive the shooting conditions before the customer starts shopping. Thus, the smartphone 40 can set different shooting conditions according to the store, the customer, the store entry time, etc. Therefore, the smartphone 40 can easily change the setting of the shooting conditions even if the operation of each store is different.

[0232] According to the present embodiment, the smartphone 40 can send the captured image data stored in the storage unit to the monitoring server 26. Thus, the smartphone 40 can send the captured image data stored in the storage unit and not previously sent again to the monitoring server 26. Therefore, the smartphone 40 can supplement the captured image data by sending it at different opportunities. The monitoring server 26 can easily manage the captured image data.

[0233] According to the present embodiment, the smartphone 40 can send the captured image data and the additional information related to the captured image data to the monitoring server 26. Thus, the smartphone 40 can send the captured image data and the information related to the shooting, such as the shooting conditions, to the monitoring server 26 in association. The monitoring server 26 can manage the captured image data and the information related to the shooting, such as the shooting conditions, in association. The waiter can confirm the customer's registered goods based on the captured image and the additional information based on the waiter image displayed on the monitoring terminal 23.

[0234] The embodiment of the transaction processing system 100 has been described above, but the embodiment is not limited thereto.

[0235] In this embodiment, the smartphone operated by the customer in the store may also be a tablet terminal installed on a shopping cart. In this case, the tablet terminal has a built-in camera or is connected to an external camera. In addition, the smartphone operated by the customer may be a smartphone lent to the customer by the store or a smartphone owned by the customer.

[0236] In this embodiment, the example of a smartphone operated by a customer who has registered as a member is described, but the present invention is not limited thereto. A customer may register a product using a smartphone without registering as a member.

[0237] In this embodiment, the server system 10 is constructed using cloud computing. In this regard, the POS system 20 may include a function as the server system 10. That is, the transaction processing system 100 may also be constructed in one store.

[0238] In this embodiment, the server system 10 is composed of the database server 11, the management server 12 and the virtual POS server 13. The server system 10 may be composed of a server that integrates the functions of the database server 11, the management server 12 and the virtual POS server 13.

[0239] Similarly, in the POS system 20 of each store, instead of separating the functions into the store server 21, the checkout server 22, and the monitoring server 26, the functions of the store server 21, the checkout server 22, and the monitoring server 26 can be integrated into one server.

[0240] The information processing terminal of the above-mentioned embodiment can also be expressed as follows.

[0241] (1) An information processing terminal comprising:

[0242] a determination unit that determines whether information related to product registration satisfies a photographing condition;

[0243] a display control unit, which displays an automatic photographing guide of the product on a display unit according to a determination result of the determination unit;

[0244] an acquisition unit that acquires image data of the photographed commodity based on the automatic photographing of the commodity; and

[0245] The transmitting unit transmits the captured image data acquired by the acquiring unit.

[0246] (2) According to the information processing terminal described in (1), the imaging condition includes a condition that the number of registered commodities reaches a predetermined number first.

[0247] (3) The information processing terminal according to (1) or (2), further comprising:

[0248] The receiving unit receives the photographing conditions before the user starts shopping.

[0249] (4) In the information processing terminal according to any one of (1) to (3), the transmission unit transmits the captured image data stored in the storage unit.

[0250] (5) In the information processing terminal according to any one of (1) to (4), the transmission unit transmits the captured image data and additional information related to the captured image data.

[0251] (6) An information processing program that enables a computer to perform the following functions:

[0252] A determination function to determine whether the information related to the product registration meets the shooting conditions;

[0253] A display control function that displays an automatic shooting guide for the product based on the determination result;

[0254] an acquisition function, which acquires the photographed image data of the photographed commodity according to the automatic photographing of the commodity; and

[0255] The sending function sends the captured image data.

[0256] In addition, several embodiments of the present invention are described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other ways, and various omissions, substitutions, and changes can be made without departing from the scope of the main purpose of the invention. These embodiments and their variations are included in the scope of the invention and are included in the invention described in the claims and their equivalents.

Claims

1. An information processing terminal, characterized in that: have: A determination unit, determining whether information related to product registration satisfies a shooting condition; a display control unit that displays an automatic photographing guide of the product on a display unit according to a determination result of the determination unit; an acquisition unit that acquires photographed image data of the photographed product based on the automatic photographing of the product; and The transmitting unit transmits the captured image data acquired by the acquiring unit.

2. The information processing terminal according to claim 1, wherein: The photographing condition includes that the number of registered commodities reaches a predetermined number initially.

3. The information processing terminal according to claim 1, wherein: Also available: The receiving unit receives the photographing conditions before the user starts shopping.

4. The information processing terminal according to claim 1, wherein: The transmitting unit transmits the captured image data stored in the storage unit.

5. The information processing terminal according to claim 1, wherein: The transmitting unit transmits the captured image data and additional information related to the captured image data.

6. A storage medium storing an information processing program, the program enabling a computer to execute the following functions: a determination function to determine whether information related to commodity registration satisfies a photographing condition; Display control function, which displays the automatic shooting guide of the product according to the judgment result; an acquisition function, based on the automatic photographing of the commodity, to acquire the photographed image data of the photographed commodity; and The sending function sends the captured image data.

Images