Information processing apparatus and information processing program

Abstract

To provide a technology for improving work efficiency in collating a purchased commodity with a registered commodity.SOLUTION: An information processing apparatus includes a determination unit, a first output unit, an acquisition unit, and a second output unit. The determination unit determines whether information on registration of a commodity satisfies an imaging condition or not. The first output unit outputs an instruction on imaging of the commodity on the basis of a result determined by the determination unit. The acquisition unit acquires captured image data of the commodity captured based on the instruction. The second output unit outputs the captured image data acquired by the acquisition unit.SELECTED DRAWING: Figure 18

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device and an information processing program. [Background technology]

[0002] A transaction processing system is already known in which, when a customer shops in a sales area where products are displayed, the customer operates a mobile terminal such as a smartphone or tablet terminal to input data related to the products they are purchasing.

[0003] This type of transaction processing system uses a dedicated self-checkout machine to process the payments of customers who have entered data on the purchased items they have placed in their shopping carts, etc. In this type of system, to prevent customers from forgetting to register items or from registering items incorrectly, a store clerk in the checkout area verifies the purchased items placed in the customer's shopping carts, etc., against the registered items.

[0004] However, it is difficult to verify the purchased products for all customers and all purchased products, so there is a need for a method to verify the purchased products against registered products quickly and efficiently. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-107122 Summary of the Invention [Problem to be solved by the invention]

[0006] The problem to be solved by the embodiments of the present invention is to provide a technology that improves the efficiency of the process of matching purchased products with registered products. [Means for solving the problem]

[0007] In one embodiment, the information processing device includes a determination unit, a first output unit, an acquisition unit, and a second output unit. The determination unit determines whether information related to product registration satisfies a photographing condition. The first output unit outputs instructions regarding photographing the product based on the determination result of the determination unit. The acquisition unit acquires photographed image data of the product photographed based on the instructions. The second output unit outputs the photographed image data acquired by the acquisition unit. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system according to an embodiment. [Figure 2] FIG. 2 is a perspective view illustrating an example of a shopping cart according to an embodiment. [Figure 3] FIG. 3 is a sequence diagram used to explain the operations performed by a customer using the transaction processing system before shopping. [Figure 4] FIG. 4 is a sequence diagram used to explain the operations performed during the customer's shopping and before payment. [Figure 5] Figure 5 is a sequence diagram used to explain the operation of the transaction using the customer's personal transaction machine. [Figure 6] FIG. 6 is a schematic diagram illustrating an example of an entrance code reading screen displayed on a smartphone according to an embodiment. [Figure 7] FIG. 7 is a schematic diagram illustrating an example of a registration screen displayed on a smartphone according to an embodiment. [Figure 8] FIG. 8 is a schematic diagram illustrating an example of a scanning screen displayed on a smartphone according to an embodiment. [Figure 9] FIG. 9 is a schematic diagram illustrating an example of a registration screen displayed on a smartphone according to an embodiment. [Figure 10] FIG. 10 is a schematic diagram illustrating an example of a checkout screen displayed on a smartphone according to an embodiment. [Figure 11] FIG. 11 is a block diagram illustrating an example of the configuration of a monitoring server according to an embodiment. [Figure 12]FIG. 12 is a block diagram schematically illustrating an example of the configuration of a smartphone according to an embodiment. [Figure 13] FIG. 13 is a diagram showing the main data structure of a first index file stored in a transaction data storage area according to one embodiment. [Figure 14] FIG. 14 is a diagram showing a main data structure of a first transaction file stored in a transaction data storage area according to one embodiment. [Figure 15] FIG. 15 is a diagram showing the main data structure of photographed image management data stored in a photographed image storage area according to one embodiment. [Figure 16] FIG. 16 is a flowchart illustrating an example of a procedure for information processing by a monitoring server according to an embodiment. [Figure 17] FIG. 17 is a flowchart illustrating an example of a procedure for information processing by a smartphone according to an embodiment. [Figure 18] FIG. 18 is a schematic diagram illustrating an example of a guidance screen displayed on a smartphone according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment will be described below with reference to the drawings. This embodiment illustrates a transaction processing system in which, when a customer purchases a product at a sales floor where the products are displayed, the customer operates a mobile terminal, such as a smartphone, to input data related to the purchased product, and also performs the checkout process by themselves using a dedicated checkout machine. The smartphone is an example of an information processing device operated by a customer. In the following description, the customer may be referred to as a user, consumer, or member.

[0010] [Configuration of transaction processing system 100] FIG. 1 is a block diagram showing a schematic configuration of a transaction processing system 100 according to this embodiment. The transaction processing system 100 includes a server system 10 that uses cloud computing and a POS (Point of Sales) system 20 established at an affiliated store. The server system 10 and the POS system 20 enable two-way data communication via a network 30 that uses the Internet Protocol. The network 30 is a wide-area network that uses public lines or dedicated lines as a relay network and wired local area networks (LANs), wireless LANs, mobile communication networks, mobile phone communication networks, etc. as access networks. Therefore, a smartphone 40 that uses a mobile phone communication network can access the server system 10 via the network 30.

[0011] An affiliated store is a store that has signed a service contract with the management company of the server system 10. Note that FIG. 1 shows the POS system 20 of one affiliated store. Needless to say, the POS systems of other affiliated stores are also connected to the network 30 in the same way. However, the POS systems of other affiliated stores do not necessarily have the same configuration as the POS system 20 shown in FIG. 1. Some of the configuration of the POS system may differ depending on the affiliated store.

[0012] The server system 10 comprises a database server 11, a management server 12, and a virtual POS server 13, and the servers are connected to each other via a communication line 14. The server system 10 also connects a router 15 to the communication line 14, and is connected to a network 30 via the router 15.

[0013] The database server 11 is a computer that provides services for managing an affiliated store database 111, a member database 112, a first product master 113 for each affiliated store, and the like.

[0014] The affiliated store database 111 is a collection of affiliated store records created for each affiliated store. The affiliated store record is a data record that records a unique store ID assigned to each affiliated store and various data related to the affiliated store. The data in the affiliated store record includes the network address and other information required for data communication with the affiliated store via the network 30.

[0015] Member database 112 is a collection of member records created for each consumer who has registered as a member to shop using transaction processing system 100. A member record is a data record that records a unique member ID assigned to each consumer and various data related to that consumer.

[0016] The first product master 113 of each affiliated store is a collection of product records created for each product sold at the corresponding affiliated store. The product record is a data record that records a unique product code assigned to each product and various data related to the product. The data in the product record includes the product name, price, etc. Each first product master 113 is uploaded from the POS system 20 of the corresponding affiliated store to the database server 11, for example, at a predetermined time before the store opens.

[0017] Incidentally, the first product master 113 may be managed by the virtual POS server 13. Also, the affiliated store database 111 and the member database 112 may be managed by the management server 12, and the database server 11 may be omitted.

[0018] The management server 12 is a computer that cooperates with the database server 11 and the virtual POS server 13 to provide a service for managing information related to transactions of customers who shop at affiliated stores using the transaction processing system 100. The virtual POS server 13 is a computer that provides a service for virtually registering sales data of products purchased by customers in accordance with a POS application. The main functions of the management server 12 and the virtual POS server 13 will be made clear in the operation explanation given below.

[0019] The POS system 20 comprises a store server 21, an accounting server 22, a monitoring terminal 23, a POS terminal 24, a dedicated accounting machine 25, and a monitoring server 26, which are connected to an in-store network 27. The POS system 20 also connects a router 28 to the in-store network 27, and is connected to a network 30 via this router 28. The POS system 20 also connects an access point 29 to the in-store network 27, and via this access point 29, enables data communication via wireless LAN with mobile terminals such as smartphones 40 and tablet terminals.

[0020] The store server 21 is a computer that provides a service of managing sales information, product inventory information, order information, etc. of the affiliated store in which the POS system 20 is established, based on information on transactions settled at the affiliated store.

[0021] The store server 21 includes a second product master 211. The second product master 211 is a collection of product records created for each product sold at the affiliated store. The product record is a data record that records a unique product code assigned to each product and various data related to the product. The data in the product record includes the product name, price, etc. In other words, the second product master 211 becomes the first product master 113 when uploaded to the server system 10. However, if the price, etc. is changed during business hours, the second product master 211 reflects the change in data. Therefore, depending on the product, the price, etc. may differ between the first product master 113 and the second product master 211.

[0022] The store server 21 is a computer that provides a service to support the member monitoring function performed by the monitoring terminal 23. The member monitoring function monitors the status of customers shopping using the transaction processing system 100. This member monitoring function displays a list of registered products for which customers have entered data using the smartphone 40 at the sales floor on the monitor of the monitoring terminal 23. Registered products are products for which customers have entered data using the smartphone 40 and have not yet been paid for. Registered products are also simply referred to as products. The monitoring terminal 23 displays monitoring images on the monitor. The monitoring images are images for monitoring information about registered products entered using the smartphone 40 and the status of the smartphone 40. The monitoring images include a list of registered products and photographed images of purchased products. Purchased products are products that customers have placed in a shopping cart or the like and plan to purchase but have not yet been paid for. The member monitoring function includes a function for comparing the information about registered products of a customer with the purchased products for monitored items. Monitored items include products or product registrations that are suspected of having been forgotten to be registered or have been fraudulently registered. In addition, a warning is displayed on the monitor if an event that should be notified to a store clerk occurs as a result of the data entry. Events that require notification to store staff include, for example, when a customer purchases alcohol, cigarettes, or other items with age restrictions, or when a registration error occurs because the product code of the scanned barcode is not set in the second product master 211.

[0023] The accounting server 22 is a computer that provides services to support member accounting using the dedicated accounting machine 25. The functions of the accounting server 22 will be made clear in the operation explanation given below.

[0024] The POS terminal 24 is a payment machine primarily used to settle transactions with customers who do not use the transaction processing system 100. The POS terminal 24 is operated by a store clerk. The POS terminal 24 can also settle transactions with customers who use the transaction processing system 100. The POS terminal may be a semi-self-service POS terminal separated into a registration machine and a payment machine, with the registration machine operated by a store clerk and the payment machine operated by the customer. The POS terminal may also be a full-self-service POS terminal in which the registration process and payment process are performed by the customer themselves.

[0025] The dedicated accounting machine 25 is an accounting machine for settling transactions with customers who use the transaction processing system 100. The dedicated accounting machine 25 is operated by the customer. Customers who do not use the transaction processing system 100 cannot make payments at the dedicated accounting machine 25.

[0026] The dedicated accounting machine 25 has the function of scanning accounting barcodes displayed on smartphones 40 used by customers using the transaction processing system 100, the function of processing payment amounts corresponding to various payment methods such as electronic money, credit cards, and cash, and the function of charging electronic money.

[0027] The monitoring server 26 is a computer that works in conjunction with the server system 10 to monitor information about registered products that customers input using their smartphones 40 at the sales floor. The monitoring server 26 works in conjunction with the store server 21 to process information to enable the member monitoring function of the monitoring terminal 23. The functions of the monitoring server 26 will be made clear in the operation explanation below. The monitoring server 26 is an example of an information processing device.

[0028] [Shopping Cart Configuration] FIG. 2 is a perspective view showing an example of a cart C according to one embodiment. The cart C includes a caster unit C1 for movement, a handle frame unit C2, and a basket holder C3. The caster unit C1 has four wheels for smooth movement on the floor. The handle frame unit C2 includes a pair of vertical frames C21, C21 erected on the rear wheel side of the caster unit C1, and a handle bar C22 connecting the upper ends of these vertical frames C21, C21. The basket holder C3 is located forward from the midpoint of the handle frame unit C2. A shopping basket BA provided by the store can be placed on the cart C on the basket holder C3 and below it.

[0029] A pole C4 is attached to one vertical frame C21 of the cart C. A stand 400 for a smartphone 40 is provided at the tip of the pole C4. A consumer using the cart C can place their own smartphone 40 on the stand 400 and go shopping.

[0030] [Operation of transaction processing system 100] Next, the main operations of the transaction processing system will be explained in the order of preparation, before shopping, during shopping, before checkout, and checkout.

[0031] <Advance preparation> A consumer who wishes to shop at an affiliated store using transaction processing system 100 first installs a dedicated application program (hereinafter referred to as the shopping app) on smartphone 40 and registers as a member. Member registration is performed via network 30 with management server 12 of server system 10. Management server 12 issues a unique member ID to the consumer who has registered as a member, and registers a member record containing that member ID in member database 112. Management server 12 also notifies smartphone 40 of the consumer who has registered as a member of the member ID via network 30. Smartphone 40 stores the member ID in association with the shopping app.

[0032] By registering as a member, a consumer becomes a member. Members can register to use electronic money that can be used at affiliated stores. Registration is performed from smartphone 40 to management server 12 via network 30. Management server 12 records the electronic money ID of the electronic money that has been registered for use in the member record that records the member ID of the member. Note that registration to use electronic money is not mandatory.

[0033] <Before shopping (see Figures 3 and 6)> Before entering an affiliated store, a member operates smartphone 40 to launch a shopping app. When the shopping app is launched, the touch panel screen of smartphone 40 transitions to a store entry code reading screen Sa (FIG. 6). The store entry code reading screen Sa is a screen for reading the store entry code Cin (FIG. 3) using the camera mounted on smartphone 40.

[0034] The store entry code Cin is prepared at the entrance of the affiliated store. For example, the store entry code Cin is printed on paper and pasted at the entrance. For example, the store entry code Cin is displayed on a display placed at the entrance. The store entry code Cin is setting information related to the affiliated store coded, for example, using a two-dimensional code system. The setting information includes the store ID and wireless LAN data set for the affiliated store. The wireless LAN data is the SSID, password, security information, etc. required for the smartphone 40 to connect to the access point 29 using wireless LAN.

[0035] As shown in Fig. 6, a camera activation button Ba is displayed on the store entry code reading screen Sa. The member touches the camera activation button Ba. This activates the camera of the smartphone 40, and the member points the camera lens at the store entry code Cin. The camera scans the store entry code Cin.

[0036] When the store entry code Cin is scanned, the smartphone 40 transmits a store entry request command CMa (FIG. 3) to the management server 12 via the network 30. The store entry request command CMa includes at least the member ID and the store ID included in the store entry code Cin.

[0037] The management server 12 obtains the member ID from the store entry request command CMa and performs member authentication (Figure 3, ACT1). Member authentication is performed by confirming whether the member ID is managed as a valid member ID. For example, an invalid flag is recorded in a member record that holds the member ID of a consumer who has lost their membership due to reasons such as a withdrawal request or fraudulent activity. If the management server 12 obtains the member ID of a member record in which an invalid flag is recorded from the store entry request command CMa, it denies the member authentication. If the management server 12 obtains the member ID of a member record in which an invalid flag is not recorded from the store entry request command CMa, it approves the member authentication.

[0038] The management server 12 determines whether to approve or deny the member authentication (ACT2 in FIG. 3). If the member authentication is denied, the management server 12 returns an entry denial command CMb to the smartphone 40. The entry denial command CMb is sent to the smartphone 40 via the network 30. Upon receiving the entry denial command CMb, the smartphone 40 stops the shopping app. Therefore, the customer cannot perform so-called self-service shopping at the affiliated store, where the customer operates the smartphone 40 and inputs data related to the purchased items.

[0039] If the member authentication is approved, the management server 12 issues a transaction ID (ACT3 in Figure 3). The transaction ID is a unique ID assigned to each purchase, i.e., the transaction of buying and selling goods, made by a member each time the member makes a purchase at an affiliated store. For example, a unique transaction ID is issued by combining a store code, a date, and a serial number. The management server 12 associates the member ID and store ID obtained from the store entry request command CMa with the transaction ID it issued, and records this in the association table 70 (Figure 3). In other words, the association table 70 is a data table that associates the member ID of a consumer making a self-service purchase with the store ID of the affiliated store where the purchase was made, and the unique transaction ID issued for that transaction.

[0040] The management server 12 replies with a store entry approval command CMc to the smartphone 40. The store entry approval command CMc is sent to the smartphone 40 via the network 30. Upon receiving the store entry approval command CMc, the smartphone 40 connects to the access point 29 via wireless LAN using the wireless LAN data acquired from the store entry code Cin. When the smartphone 40 is connected to the access point 29 via wireless LAN, the touch panel screen transitions to a shopping start screen. The shopping start screen is a screen that prompts the consumer to start shopping. A shopping start button is displayed on the shopping start screen.

[0041] On the other hand, the management server 12 that sent the store entry approval command CMc sends a transaction file creation instruction command CMd to the virtual POS server 13. The creation instruction command CMd includes the store ID and transaction ID that are stored in association with the member ID in the linkage table 70. Upon receiving the creation instruction command CMd, the virtual POS server 13 creates a transaction file 80 (FIG. 3) in its storage unit. The virtual POS server 13 then writes the store ID and transaction ID included in the creation instruction command CMd into the transaction file 80. The transaction file 80 is a data file that records sales data of the products purchased by a member identified by the transaction ID in a transaction with that member, and payment data for the settlement amount of the transaction.

[0042] <Shopping (see Figures 4, 7, 8, and 9)> After checking the shopping start screen, the member touches the start shopping button. This causes the touch panel screen to transition to the registration screen Sb (Figure 7). As shown in Figure 7, the registration screen Sb displays the total number of registered items and the total price. However, because no purchased items have yet been registered, the total number of registered items and the total price are both "0." The registration screen Sb also displays a no barcode button Bb, a scan button Bc, and a checkout button Bd. The no barcode button Bb is the button to be touched when entering data for an item that does not have a barcode. The scan button Bc is the button to be touched when entering data for an item that does have a barcode. The checkout button Bd is the button to be touched when completing shopping and instructing checkout. However, because product registration has not yet been performed, the checkout button Bd is inoperable. For example, the checkout button Bd is grayed out. The checkout button Bd may also be erased.

[0043] When a member finds a product they wish to purchase in a sales area where products are displayed, they check whether the product has a barcode BC (Figure 4) attached to it. If the product has a barcode BC attached, the member touches the scan button Bc. When the scan button Bc is touched, the touch panel screen transitions to the scanning screen Sc (Figure 8). As shown in Figure 8, the scanning screen Sc displays a frame ARa indicating the barcode reading area. Also displayed are a no barcode button Bb and a back button Be. The back button Be is the button that is touched to cancel scanning of the barcode BC.

[0044] The member operates the smartphone 40 so that the barcode BC of the purchased product is within the frame line ARa. When the barcode BC is within the frame line ARa, the camera of the smartphone 40 scans the barcode BC.

[0045] After scanning the barcode BC, the smartphone 40 transmits a registration request command CMe (FIG. 4) to the management server 12 via wireless LAN. The registration request command CMe includes the member ID and the data of the barcode BC, i.e., barcode data. The registration request command CMe is received by the access point 29 and then transmitted from the router 28 to the management server 12 via the network 30.

[0046] The management server 12 searches the linkage table 70 for the member ID included in the registration request command CMe, and obtains the store ID and transaction ID associated with the member ID.The management server 12 then sends the registration request command CMf, in which the member ID has been replaced with the store ID and transaction ID, to the virtual POS server 13.

[0047] The virtual POS server 13 obtains the store ID, transaction ID, and barcode data from the registration request command CMf. Then, the virtual POS server 13 obtains a product code from the barcode data. The virtual POS server 13 searches the first product master 113 of the affiliated store identified by the store ID to obtain a product record containing the product code. Then, based on the product data in the product record, the virtual POS server 13 registers product sales data consisting of the product code, product type, product name, price, sales quantity, sales amount, etc. in the transaction file 80 in which the store ID and transaction ID are written.

[0048] The management server 12 acquires a transaction file 80 in which product sales data is registered from the virtual POS server 13. Then, the management server 12 transmits a registration response command CMg (FIG. 4) to the smartphone 40 that sent the registration request command CMe. The registration response command CMg includes the data of the transaction file 80 acquired from the virtual POS server 13. The registration response command CMg is transmitted from the router 15 to the POS system 20 via the network 30, and further transmitted from the access point 29 to the smartphone 40.

[0049] When the smartphone 40 receives the registration response command CMg, the touch panel screen transitions to the registration screen Sd (FIG. 9). The registration screen Sd is the screen displayed when the product sales data for the first item of product AAA has been registered in the transaction file 80. As shown in FIG. 9, the registration screen Sd displays the sales price of product AAA, "¥100," and the number of items sold, "1 item." The total number of items becomes "1 item," and the total amount becomes "¥100." The checkout button Bd can also be pressed.

[0050] After that, for the second and subsequent purchased items, the member touches the scan button Bc to scan the barcode of the item. By doing so, the management server 12 and virtual POS server 13 operate in the same manner as above. As a result, the registration screen Sb displays the name of the purchased item, the sales price, the number of items sold, and the total number and price.

[0051] As mentioned above, the sales price of the product sales data stored in the transaction file 80 is calculated based on the price in the first product master 113. Even if a price change occurs during business hours, the price change is not reflected in the price in the first product master 113. Therefore, the sales price and total price displayed on the registration screen Sd may differ from the amount based on the current price. For this reason, the sales price and total price displayed on the registration screen Sd are merely considered as a reference price during shopping.

[0052] If the purchased item does not have a barcode attached, the member touches the No Barcode button Bb. A list of items without barcodes is then displayed on the touch panel of the smartphone 40, and the member selects the item to purchase from the list. A registration request command CMe containing the item ID of the purchased item is then sent to the management server 12. The subsequent operations are the same as when the barcode BC of the item is scanned with the smartphone 40.

[0053] <Before payment (see Figure 4 and Figure 10)> When a member has finished shopping in the sales area and wishes to proceed to checkout, they touch the checkout button Bd on the registration screen Sd. When the checkout button Bd is touched, the smartphone 40 sends a checkout request command CMh (FIG. 4) to the management server 12 via wireless LAN. The checkout request command CMh includes the member ID. The checkout request command CMh is received by the access point 29 and transmitted from the router 28 to the management server 12 via the network 30.

[0054] The management server 12 issues an accounting code (ACT 11 in Figure 4). The accounting code is, for example, a combination of date, time, and a serial number, and no duplicate accounting codes are issued within at least one business day. The management server 12 generates a first image Ca (ACT 12 in Figure 4) (ACT 10 in Figure 4). The first image Ca is an image of the accounting code converted into a barcode, that is, an image of an accounting barcode.

[0055] The management server 12 sends an accounting approval command CMi to the smartphone 40 that sent the accounting request command CMh. The accounting approval command CMi includes the first image Ca.

[0056] The billing approval command CMi is sent to the POS system 20 via the network 30, and then from the access point 29 to the smartphone 40 via wireless LAN. When the smartphone 40 receives the billing approval command CMi, the touch panel screen transitions to the billing screen Se (FIG. 10). As shown in FIG. 10, the billing screen Se displays the first image Ca. The billing screen Se also displays an end button Bf. The end button Bf is the button that is touched to cancel the transaction.

[0057] The management server 12 also acquires the store ID and transaction ID stored in the linkage table 70 in association with the member ID acquired from the transaction request command CMh. The management server 12 then acquires a transaction file 80 containing the store ID and transaction ID from the virtual POS server 13. The management server 12 then transmits the transaction file 80 and accounting code to the POS system 20 of the affiliated store identified by the store ID. The transaction file 80 and accounting code data are then transmitted via the network 30 to the store server 21 of the corresponding affiliated store. The store server 21 then transfers the transaction file 80 and accounting code data received from the management server 12 to the accounting server 22. The accounting server 22 stores the transaction file 80 in memory in association with the accounting code.

[0058] <Accounting (see Figure 5)> After checking the accounting screen Se, the member uses the dedicated accounting machine 25 to pay using various payment methods such as electronic money, credit card, or cash.

[0059] The operation of a transaction using the dedicated transaction machine 25 will be described with reference to FIG. The member scans the first image Ca on the accounting screen Se, i.e., the accounting barcode, with the scanner of the dedicated accounting device 25. The dedicated accounting device 25 then transmits the accounting barcode data, known as barcode data BCD (FIG. 5), to the accounting server 22.

[0060] The accounting server 22 obtains the accounting code from the barcode data BCD (ACT 21 in Figure 5). The accounting server 22 then retrieves the transaction file 80 associated with the accounting code from memory (ACT 22 in Figure 5). The accounting server 22 then transmits the transaction file 80 to the dedicated accounting machine 25.

[0061] The dedicated accounting machine 25 recalculates the payment amount by referencing the second product master 211 managed by the store server 21. Specifically, the dedicated accounting machine 25 obtains the product code registered in the transaction file 80. The dedicated accounting machine 25 then searches the second product master 211 to obtain the price of the product record containing that product code. The dedicated accounting machine 25 multiplies that price by the sales quantity registered in the transaction file 80 along with the product code to calculate the sales amount of the purchased product identified by that product code. The dedicated accounting machine 25 performs the above process for all product codes registered in the transaction file 80. The dedicated accounting machine 25 then adds up the sales amounts of all purchased products to calculate the payment amount.

[0062] The dedicated payment machine 25 displays the recalculated payment amount on the display and waits for the customer to select a payment method (Figure 5, ACT 32). The recalculated payment amount may differ from the total amount displayed on the smartphone 40. Even in this case, the total amount displayed on the smartphone 40 is only a reference amount, so the customer will pay the payment amount displayed on the dedicated payment machine 25.

[0063] The member selects a payment method. For example, if cash is selected, the dedicated accounting machine 25 executes a payment process to pay the settlement amount in cash (Figure 5, ACT 33). For example, if credit card is selected, the dedicated accounting machine 25 executes a payment process to pay the settlement amount by credit card. Note that these payment processes are well known, so detailed explanations will be omitted here.

[0064] When the payment process is complete, the dedicated accounting machine 25 sends a payment completion notification command CMj to the accounting server 22. The payment completion notification command CMj includes payment data. For example, in the case of a cash payment, the payment data includes the amount of the deposit, the amount of change, etc. In the case of a credit card payment, the payment data includes the amount of the credit card payment, etc.

[0065] The accounting server 22 registers the payment data in the transaction file 80. The accounting server 22 also sets a completion status in the transaction file 80. The transaction file 80 with the completion status set is saved in the recording unit of the accounting server 22 or the store server 21.

[0066] [Configuration of monitoring server 26] FIG. 11 is a block diagram schematically illustrating an example of the configuration of the monitoring server 26 according to an embodiment. The monitoring server 26 includes a processor 261, a main memory 262, an auxiliary storage unit 263, a communication interface 264, and a transmission path 265. The processor 261, the main memory 262, the auxiliary storage unit 263, and the communication interface 264 are capable of communicating with each other via the transmission path 265. The monitoring server 26 constitutes a computer by the processor 261, the main memory 262, the auxiliary storage unit 263, and the transmission path 265 connecting these components.

[0067] The processor 261 corresponds to the central part of the computer. The processor 261 controls each part to realize various functions of the monitoring server 26 in accordance with an operating system and application programs. The processor 261 is, for example, a CPU (Central Processing Unit).

[0068] The main memory 262 corresponds to the main storage portion of the computer. The main memory 262 includes a nonvolatile memory area and a volatile memory area. The main memory 262 stores an operating system and application programs in the nonvolatile memory area. The main memory 262 may store data required for the processor to execute processes to control each part in either the nonvolatile or volatile memory area. The main memory 262 uses the volatile memory area as a work area where data is rewritten by the processor as appropriate. For example, the nonvolatile memory area is ROM (Read Only Memory). The volatile memory area is RAM (Random Access Memory).

[0069] The auxiliary storage unit 263 corresponds to the auxiliary storage portion of the computer. For example, an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive) is used as the auxiliary storage unit 263. The auxiliary storage unit 263 stores data used by the processor 261 when performing various processes and data created by the processes in the processor 261. The auxiliary storage unit 263 may also store the application programs described above.

[0070] The communication interface 264 performs data communication in accordance with a predetermined communication protocol with each unit connected to the in-store network 27. As the communication interface 264, for example, a well-known communication device for a LAN can be applied.

[0071] The transmission path 265 includes an address bus, a data bus, and control signal lines, and transmits data and control signals exchanged between the connected components.

[0072] The auxiliary storage unit 263 includes a transaction data storage area 2631 and a captured image storage area 2632 .

[0073] The transaction data storage area 2631 stores first transaction data based on data stored in the virtual POS server 13. The first transaction data is an example of information related to product registration. The information related to product registration includes information related to products registered via the smartphone 40 before payment and information related to product registration via the smartphone 40. The information related to product registration includes information indicating the product type, product price, and registered points. The product type includes information indicating products with age restrictions, such as alcoholic beverages and tobacco. The price indicates the unit price of the product. The registered points indicate the total points of the product registered by the member. The information related to product registration includes information related to product registration operations via the smartphone 40 and information related to the time associated with product registration via the smartphone 40. The information related to product registration operations includes information related to editing, such as adding or deleting registered products, information related to operations to register products without barcodes, information related to operations to proceed to checkout, and location information of the location where the product registration operation was performed. The information related to time includes information indicating the duration of stay, the time of the most recent operation, etc., as described below.

[0074] The first transaction data includes a first index file and a first transaction file. The first index file is data associated with a plurality of first transaction files. The first index file is a collection of index file records created for each terminal ID. The index file record is a data record that associates and records the terminal ID with the transaction ID, stay time, most recent operation time, registered points, status, and accounting machine ID, etc.

[0075] The terminal ID is a unique code assigned to each smartphone 40 to identify it individually. The transaction ID is a unique code assigned to each first transaction file that records transactions made using the smartphone 40 to identify each first transaction file individually. The transaction ID is set in the first transaction file each time the first transaction file is created. The transaction ID is associated with the member ID and store ID, as in the linkage table 70. The stay time indicates the time elapsed since the member logged in. The most recent operation time indicates the time when the member last operated the smartphone 40. For example, the most recent operation time indicates the time when the member registered the last product. The status is a terminal status that indicates the state of the smartphone 40. Terminal statuses include "waiting," "registering," and "checkout." "Waiting" refers to the state from when the previous member has finished checking out until the next member logs in. "Registering" refers to the state from when the member logs in until the checkout is approved. The state before the checkout is approved is also referred to as the state before the checkout process is performed. "Checkout in progress" is the state from when the checkout is approved until it is completed. The checkout machine ID is a unique code assigned to each checkout machine to identify it individually. The first index file may contain information such as the total price of registered products in addition to the above items. The first index file is associated with the first transaction file using the transaction ID as a key. Details of the first index file will be described later.

[0076] The first transaction file is a collection of transaction file records created for each transaction ID. The transaction file record is a data record that records the transaction ID in association with product sales data. The product sales data includes the product code, product type, product name, price, number of items sold, sales amount, etc. The first transaction file may be data generated in the same way as transaction file 80. The first transaction file is updated each time a product is registered by a member. Details of the first transaction file will be described later.

[0077] The photographed image storage area 2632 stores photographed image management data. The photographed image management data includes photographed image data taken by the smartphone 40. The photographed image data includes data of images of products purchased by the member who owns the smartphone 40. The photographed image storage area 2632 may store photographed image data indicating multiple photographed images in chronological order. The photographed image management data is updated each time photographed image data is acquired by the acquisition unit 2612.

[0078] The hardware configuration of the monitoring server 26 is not limited to the above configuration, and the monitoring server 26 allows the above components to be omitted or changed, and new components to be added, as appropriate.

[0079] Each unit implemented in the above-mentioned processor 261 will be described. The processor 261 implements a determination unit 2610, a first output unit 2611, an acquisition unit 2612, and a second output unit 2613. The units implemented in the processor 261 can also be referred to as functions. The units implemented in the processor 261 can also be referred to as being implemented in a control unit including the processor 261 and the main memory 262.

[0080] The determination unit 2610 determines whether the first transaction data satisfies the image capture conditions. The image capture conditions include at least one of a product-related condition and a product registration condition. The image capture conditions are conditions related to the product to be monitored or product registration. The product-related conditions are conditions to be compared with information related to the product. The product-related conditions include, for example, conditions related to the number of registered products, price, and type. The product-related condition may be, for example, a fixed number of products, such as "5 points," or a fixed interval of points, such as "every 5-point increment." If the product-related condition is an increment of 5 points, the product-related condition includes registered product numbers that are multiples of 5, such as 5 points, 10 points, and 15 points. Note that the intervals do not have to be fixed. The intervals can be appropriately set, such as narrowing or widening as the registered number of products increases. The price-related condition is, for example, a high-priced product, such as "1,000 yen or more." The type-related condition is, for example, "alcoholic beverages." The conditions related to product registration include conditions related to the product registration operation and conditions related to time. The conditions related to the product registration operation are conditions that are compared with information related to the product registration operation. The conditions related to the product registration operation include, for example, conditions related to editing, such as adding or deleting registered products, conditions related to the operation of registering a product without a barcode, conditions related to the operation of transitioning to checkout, and conditions related to the location information of the location where the product registration operation was performed. An example of a condition related to editing is ``adding the number of sales units of the registered product.'' An example of a condition related to the operation of registering a product without a barcode is ``the no barcode button has been operated.'' An example of a condition related to the operation of transitioning to checkout is ``the checkout button has been operated.'' An example of a condition related to the location information of the location where the product registration operation was performed is ``the location where the product was registered is different from the display position of the registered product.'' A time-related condition is a condition that is compared with information related to time. An example of a condition related to time is a condition related to the length of stay and a condition related to the time of the most recent operation. An example of a condition related to the length of stay is ``the length of stay exceeds a certain period of time.'' The condition regarding the immediately preceding operation time is, for example, "a certain period of time has elapsed since the immediately preceding operation time." The certain period of time can be set as appropriate.

[0081] The first output unit 2611 outputs instructions regarding photographing a product to the smartphone 40 based on the determination result of the determination unit 2610. For example, the first output unit 2611 outputs an instruction to notify the smartphone 40 of product photographing guidance based on the determination by the determination unit 2610 that the first transaction data satisfies the photographing conditions. The instruction to notify the smartphone 40 of product photographing guidance is, for example, an instruction for the processor 401 of the smartphone 40 to control the touch panel 404. In response to the instruction to notify the product photographing guidance, the processor 401 causes the touch panel 404 to display the photographing guidance. The product photographing guidance includes a notification encouraging the member to photograph the product. The notification encouraging the member to photograph the product is, for example, text information such as "Please take a photograph." The notification encouraging the member to photograph may include any type of notification, such as a symbol, an icon, audio, or video. If the notification prompting photographing is audio, the processor 401 of the smartphone 40 controls the audio output device 406 to output audio photographing guidance in response to the notification instruction for product photographing guidance. If the notification prompting photographing is video, the processor 401 of the smartphone 40 controls the audio output device 406 and the touch panel 404 to output photographing guidance in response to the notification instruction for product photographing guidance. The product photographing guidance may include a photographing guide. The photographing guide includes a frame indicating the area to be photographed. In another example, the first output unit 2611 outputs a photographing instruction for the product to the smartphone 40 based on the determination unit 2610 determining that the first transaction data satisfies the photographing conditions. The photographing instruction for the product is an instruction for the processor 401 of the smartphone 40 to control the camera 405. In response to the photographing instruction for the product, the processor 401 activates the camera 405 and controls the camera 405 to automatically photograph the product. In the following description, "in response" may be read as "based on."

[0082] The acquisition unit 2612 acquires photographed image data of the product photographed by the camera 405 based on the instruction relating to photographing the product output by the first output unit 2611.

[0083] The second output unit 2613 outputs the photographed image data acquired by the acquisition unit 2612 to the monitoring terminal 23.

[0084] [Configuration of Smartphone 40] FIG. 12 is a block diagram schematically illustrating an example of the configuration of a smartphone 40 according to an embodiment. The smartphone 40 includes a processor 401, a main memory 402, an auxiliary storage unit 403, a touch panel 404, a camera 405, an audio output device 406, a wireless communication unit 407, a mobile communication unit 408, and a transmission path 409. The processor 401 is capable of communicating with the main memory 402, the auxiliary storage unit 403, the touch panel 404, the camera 405, the audio output device 406, the wireless communication unit 407, and the mobile communication unit 408 via the transmission path 409. The processor 401, the main memory 402, and the auxiliary storage unit 403 are connected via the transmission path 409 to form a computer for controlling the smartphone 40. The functions of the processor 401, the main memory 402, the auxiliary storage unit 403, and the transmission path 409 are generally the same as those of the processor 261, the main memory 262, the auxiliary storage unit 263, and the transmission path 265, and therefore will not be described here.

[0085] The touch panel 404 functions as an operation device and a display device for the smartphone 40. The touch panel 404 is an example of a display unit that displays information or a notification unit that visually notifies information.

[0086] The camera 405 includes an optical system and an image sensor, and generates image data representing an image within a field of view formed by the optical system using the image sensor. The camera 405 is an example of an imaging unit.

[0087] The audio output device 406 is a device that can output sound under the control of the processor 401. The audio output device 406 is an example of an audio output unit that outputs information by voice or a notification unit that notifies information audibly.

[0088] The wireless communication unit 407 transmits and receives data via wireless communication in accordance with a wireless communication protocol to and from the access point 29. As the wireless communication unit 407, for example, a well-known communication device conforming to the IEEE802.11 standard can be used.

[0089] The mobile communication unit 408 is an interface for data communication via the network 30. As the mobile communication unit 408, for example, a well-known communication device for performing data communication via a mobile communication network can be used.

[0090] The auxiliary storage unit 403 includes a smartphone POS app 4031, which is one of the information processing programs, and a transaction data storage area 4032. The smartphone POS app 4031 is an application program that describes information processing for making the smartphone 40 function as a user interface for the POS system 20. The smartphone POS app 4031 is used in common by multiple smartphones 40.

[0091] The transaction data storage area 4032 stores second transaction data based on data stored in the virtual POS server 13. The second transaction data is an example of information related to product registration. The second transaction data includes a second index file and a second transaction file. The second index file and the second transaction file include information related to products registered by the smartphone 40. The second index file is data that associates a terminal ID, a transaction ID, a stay time, a time of the most recent operation, a registered number of items, a status, an accounting machine ID, etc.

[0092] The terminal ID is a unique code set to identify the smartphone 40. The transaction ID is a unique code set for each second transaction file to individually identify the second transaction file that records transactions performed using the smartphone 40. The transaction ID is set in the second transaction file each time the second transaction file is created. The transaction ID is associated with the member ID and the store ID, similar to the linkage table 70. The second index file includes index file records from the first index file stored in the transaction data storage area 2631 of the monitoring server 26, which are associated with the terminal ID corresponding to each smartphone 40.

[0093] The second transaction file is composed of data that associates transaction IDs with product sales data. The product sales data includes product codes, product types, product names, prices, sales units, sales amounts, etc. The second transaction file may be data generated in the same manner as transaction file 80. The second transaction file is updated each time a product is registered by a member. The second transaction file includes transaction file records from the first transaction file stored in transaction data storage area 2631 of monitoring server 26 that are associated with transaction IDs corresponding to each smartphone 40.

[0094] The hardware configuration of the smartphone 40 is not limited to the above-described configuration. The smartphone 40 allows the omission or modification of the above-described components and the addition of new components as appropriate.

[0095] Each unit implemented in the above-mentioned processor 401 will be described. The processor 401 is equipped with a determination unit 4010, a first output unit 4011, an acquisition unit 4012, and a second output unit 4013. The units implemented in the processor 401 can also be referred to as functions. The units implemented in the processor 401 can also be referred to as being implemented in a control unit including the processor 401 and the main memory 402.

[0096] The determination unit 4010 determines whether the second transaction data satisfies the photographing conditions, which are the same as the photographing conditions described above.

[0097] The first output unit 4011 outputs instructions regarding photographing the product to the touch panel 404 based on the determination result of the determination unit 4010. For example, the first output unit 4011 outputs an instruction to notify product photographing guidance to the touch panel 404 based on the determination by the determination unit 4010 that the second transaction data satisfies the photographing conditions. The instruction to notify product photographing guidance is an instruction to notify product photographing guidance. The instruction to notify product photographing guidance is, for example, an instruction by the processor 401 to control the touch panel 404. In response to the instruction to notify product photographing guidance, the processor 401 causes the touch panel 404 to display photographing guidance. The product photographing guidance includes a notification encouraging the member to photograph the product. The notification encouraging the member to photograph the product is, for example, text information such as "Please take a photograph." The notification encouraging the member to photograph may include any type of notification, such as a symbol, an icon, audio, or video. When the notification prompting the user to take a photograph is audio, processor 401 outputs instructions regarding the photographing of the product to audio output device 406. In response to the instruction to notify product photographing guidance, processor 401 controls audio output device 406 to output the photographing guidance as audio. When the notification prompting the user to take a photograph is video, processor 401 outputs instructions regarding the photographing of the product to touch panel 404 and audio output device 406. In response to the instruction to notify product photographing guidance, processor 401 controls audio output device 406 and touch panel 404 to output the photographing guidance. The product photographing guidance may include a photographing guide. In another example, first output unit 4011 outputs an instruction to photograph the product to camera 405 based on the determination unit 4010 determining that the second transaction data satisfies the photographing conditions. The instruction to photograph the product is an instruction from processor 401 to control camera 405. In response to an instruction to photograph a product, the processor 401 activates the camera 405 and controls the camera 405 to automatically photograph the product.

[0098] The acquisition unit 4012 acquires photographed image data of the product photographed by the camera 405 based on instructions relating to photographing the product output by the first output unit 4011.

[0099] The second output unit 4013 outputs the captured image data acquired by the acquisition unit 4012 to the monitoring server 26.

[0100] An example of the structure of the first index file will be described. FIG. 13 is a diagram showing the main data structure of the first index file stored in the transaction data storage area 2631 according to one embodiment.

[0101] The first index file includes a "Terminal ID" field, a "Transaction ID" field, a "Dwell Time" field, a "Last Operation Time" field, a "Registered Points" field, a "Status" field, and a "Payment Machine ID" field. Each of the multiple terminal IDs is associated with a transaction ID, duration, last operation time, registered points, status, and payment machine ID as an index file record. For example, the smartphone 40 with terminal ID "00001" is associated with transaction ID "1." The smartphone 40 with terminal ID "00001" indicates that its stay time, which is the elapsed time since login, is "8 minutes," and its last operation time, which is the last time the smartphone 40 was operated, is "14:10." The smartphone 40's registered points are "5," and its status indicates "Registering," which indicates that the transaction has not yet been processed. In this example, the smartphone 40 with terminal ID "00001" is in a state where the transaction has not yet been processed, and therefore is not associated with a payment machine, and the payment machine ID is blank. The information included in the first index file can be set as appropriate. The monitoring server 26 updates the first index file as appropriate. Note that the configuration example of the second index file stored in the transaction data storage area 4032 of the smartphone 40 includes only the index file records related to the corresponding smartphone 40 among the configuration examples of the first index file described above. For example, the smartphone 40 with the terminal ID "00001" stores only the index file records associated with the terminal ID "00001" in the transaction data storage area 4032.

[0102] An example of the structure of the first transaction file will be described. FIG. 14 is a diagram showing the main data structure of the first transaction file stored in the transaction data storage area 2631 according to one embodiment.

[0103] The first transaction file includes a "transaction ID" field, a "product code" field, a "product name" field, a "product type" field, a "price" field, a "number of units sold" field, and a "sales amount" field. Each of the multiple transaction IDs is associated as a transaction file record with the transaction ID, product code, product type, product name, price, number of units sold, and sales amount. The first index file and the first transaction file are associated with each other using the "transaction ID" as a key. The information contained in the first transaction file can be set as appropriate. The monitoring server 26 updates the first transaction file as appropriate. Note that an example of the configuration of the second transaction file stored in the transaction data storage area 4032 of the smartphone 40 includes only information related to the corresponding smartphone 40 from the above-mentioned example of the first transaction file. For example, a smartphone 40 with terminal ID "00001" stores only the first transaction file record for the "transaction ID" associated with terminal ID "00001" in the transaction data storage area 4032.

[0104] An example of the configuration of captured image management data will be described. FIG. 15 is a diagram showing the main data structure of the photographed image management data stored in the photographed image storage area 2632 according to one embodiment.

[0105] The captured image management data includes a "transaction ID" item and an "image" item. The "image" item sets captured image data. Each of the multiple transaction IDs is associated with the captured image data as a captured image data record. The monitoring server 26 may store the captured image data in the captured image storage area 2632 in chronological order based on the time the captured image data was acquired. The captured image management data is associated with the first index file and the first transaction file using the "transaction ID" as a key. The monitoring server 26 updates the captured image management data appropriately each time captured image data is acquired.

[0106] [Operation of monitoring server 26] The processing procedure performed by the monitoring server 26 will now be described. FIG. 16 is a flowchart showing an example of a procedure for information processing by the monitoring server 26 according to an embodiment. The processing procedures described below are merely examples, and each process may be modified as much as possible. Furthermore, steps may be omitted, replaced, or added as appropriate depending on the embodiment.

[0107] The following process is performed during the above-mentioned shopping. The following process is assumed to be performed after a product is registered by a member using smartphone 40 and before the transaction process begins. The member uses camera 405 to read the code symbol attached to a product displayed in the store and registers the product. Information about the registered product is stored in virtual POS server 13 each time a product is registered by the member. Information about the registered product is also stored in transaction data storage area 2631 of monitoring server 26, which cooperates with virtual POS server 13. It is assumed that a product is registered by a member using smartphone 40 with terminal ID "00001" in FIG. 13.

[0108] In ACT101, the processor 261 determines whether the transaction data satisfies the photography conditions. In this example, a case will be described where the photography conditions are conditions related to the number of items. Assume that the condition related to the number of items is set such that the registered number, which is the total number of items sold, is increased by "5 items." For example, the processor 261 references the index file record associated with the terminal ID "00001" stored in the first index file, and compares the "registered number" with the condition related to the number of items. If the processor 261 determines that the "registered number" of the items registered in the first index file satisfies the photography conditions, the process proceeds to YES in ACT101. The processor 261 proceeds to processing in ACT102. If the processor 261 determines that the "registered number" of the items registered in the first index file does not satisfy the photography conditions, the process proceeds to NO in ACT101. The processor 261 repeats the processing in ACT101. For example, the processor 261 compares the registered number "5" from the first index file associated with the terminal ID "00001" with the condition related to the number of items. In this example, the processor 261 determines that the "registered number" of the item registered in the first index file satisfies the photography condition, and proceeds to YES in ACT101. The processor 261 functions as the determination unit 2610 in ACT101.

[0109] In ACT102, the processor 261 outputs instructions regarding photographing the product to the smartphone 40 based on the determination result of ACT101. In one example, the processor 261 outputs an instruction to notify the smartphone 40 of photographing guidance for the product in response to the "number of registered items" of the product registered in the first index file satisfying the photographing conditions. In response to the instruction to notify the photographing guidance for the product, the processor 401 of the smartphone 40 controls the touch panel 404 to display the photographing guidance. Note that if the photographing guidance is audio, the processor 401 of the smartphone 40 controls the audio output device 406 to output the photographing guidance as audio in response to the instruction to notify the photographing guidance for the product. If the notification prompting photographing is video, the processor 401 of the smartphone 40 controls the audio output device 406 and the touch panel 404 to output the photographing guidance in response to the instruction to notify the photographing guidance for the product.

[0110] In another example, in response to the "registered number" of the product registered in the first index file satisfying the photographing condition, the processor 261 outputs an instruction to photograph the product to the smartphone 40. In response to the instruction to photograph the product, the processor 401 of the smartphone 40 starts the camera 405 and controls the camera 405 to automatically photograph the product. The processor 261 functions as the first output unit 2611 in ACT102.

[0111] The processor 261, as ACT103, acquires photographed image data of the product photographed by the camera 405 based on an instruction related to photographing the product. The processor 261 functions as an acquisition unit 2612 in ACT103.

[0112] In ACT104, the processor 261 stores the acquired photographed image data of the product in the photographed image storage area 2632. For example, the processor 261 associates the acquired photographed image data of the product with the transaction ID and stores them in chronological order in the photographed image storage area 2632.

[0113] The processor 261, as ACT105, outputs the acquired photographed image data to the monitoring terminal 23. The monitoring terminal 23 generates a monitoring image based on the acquired image data and displays it on a monitor (not shown) of the monitoring terminal 23. The monitoring image includes a photographed image of the product associated with the transaction data. The processor 261 functions as a second output unit 2613 in ACT105.

[0114] Note that the processor 261 repeats the processing of ACT101 to ACT105 each time the first transaction data satisfies the photographing condition during one transaction from when the member logs in to when the transaction processing is completed. An example will be described in which the photographing condition is a condition regarding the number of items, which is "every increase of 5 items." The processor 261 determines in ACT101 that the photographing condition is met when the registered number of items is 5, and processes ACT102 to ACT105. The processor 261 determines in ACT101 that the photographing condition is met when the registered number of items is 10, and processes ACT102 to ACT105. Thereafter, the processor 261 similarly determines in ACT101 that the photographing condition is met when the registered number of items is a multiple of 5, and processes ACT102 to ACT105.

[0115] In this way, when processor 261 repeats the processes of ACT101 to ACT105 multiple times during one transaction, the monitoring image can include multiple captured images in chronological order based on captured image data captured at different times. In this case, an attendant checking the captured images of the products on monitoring terminal 23 can confirm the change in the state of the shopping cart accompanying the change in the number of registered products as the number of registered products increases.

[0116] Here, the case where the photographing condition is a condition related to the number of items, that is, "every 5-item increase," has been described as an example, but is not limited to this. Even if the photographing condition is any of the above-mentioned various conditions other than the condition related to the number of items, processor 261 repeats the processes of ACT101 to ACT105 during one transaction each time the first transaction data satisfies the photographing condition.

[0117] [Smartphone 40 Operation] The procedure for information processing by the smartphone 40 will be described. FIG. 17 is a flowchart showing an example of a procedure for information processing by the smartphone 40 according to an embodiment. The processing procedures described below are merely examples, and each process may be modified as much as possible. Furthermore, steps may be omitted, replaced, or added as appropriate depending on the embodiment.

[0118] The following processing is performed during the above-described shopping. The following processing is performed when information processing is performed by the smartphone 40 instead of the information processing by the monitoring server 26 described above. The following processing is assumed to be performed, similar to the case of information processing by the monitoring server 26, after a product is registered by a member using the smartphone 40 and before transitioning to the checkout process. The member uses the camera 405 to read the code symbol attached to a product displayed in the store and registers the product. Information about the registered product is stored in the virtual POS server 13 each time a product is registered by the member. Information about the registered product is also stored in the transaction data storage area 4032 of the smartphone 40 linked to the virtual POS server 13. It is assumed that a product is registered by a member using the smartphone 40 with the terminal ID "00001" in FIG. 13. The smartphone 40 with the terminal ID "00001" stores only the transaction data from the smartphone 40 with that terminal ID in the transaction data storage area 4032.

[0119] In ACT201, the processor 401 determines whether the transaction data satisfies the photography conditions. In this example, a case will be described where the photography conditions are conditions related to the number of items. Assume that the condition related to the number of items is set such that the registered number, which is the total number of items sold, is increased by "every 5 items." For example, the processor 401 references the index file record associated with the terminal ID "00001" stored in the second index file, and compares the "registered number" with the condition related to the number of items. If the processor 401 determines that the "registered number" of the item registered in the second index file satisfies the photography conditions, the processor 401 proceeds to YES in ACT201. The processor 401 proceeds to processing in ACT202. If the processor 401 determines that the "registered number" of the item registered in the second index file does not satisfy the photography conditions, the processor 401 proceeds to NO in ACT201. The processor 401 repeats the processing in ACT201. For example, the processor 401 compares the registered number "5" from the second index file associated with the terminal ID "00001" with the condition related to the number of items. In this example, the processor 401 determines that the "registered number" of the item registered in the second index file satisfies the photography condition, and proceeds to YES in ACT101. The processor 401 functions as the determination unit 4010 in ACT201.

[0120] In ACT202, the processor 401 outputs an instruction regarding photographing the product to the touch panel 404 based on the determination result of ACT201. In one example, the processor 401 outputs an instruction to notify photographing guidance for the product to the touch panel 404 in response to the "number of registered products" of the products registered in the second index file satisfying the photographing conditions. The instruction to notify photographing guidance for the product is an instruction by the processor 401 to control the touch panel 404 to display the photographing guidance. Note that if the photographing guidance is audio, the processor 401 outputs the instruction to notify photographing guidance for the product to the audio output device 406. In this case, the instruction to notify photographing guidance for the product is an instruction by the processor 401 to control the audio output device 406 to output the photographing guidance as audio. If the notification prompting photographing is a video, the processor 401 outputs the instruction to notify photographing guidance for the product to the touch panel 404 and the audio output device 406. In this case, the instruction to notify the product photography guidance is an instruction by the processor 401 to control the audio output device 406 and the touch panel 404 to output the photography guidance.

[0121] In another example, in response to the "registered number" of the product registered in the second index file satisfying the photographing condition, the processor 401 outputs an instruction to photograph the product to the camera 405. The instruction to photograph the product is an instruction by the processor 401 to start the camera 405 and control the camera 405 to automatically photograph the product. The processor 401 functions as a first output unit 4011 in ACT202.

[0122] The processor 401, as ACT203, acquires photographed image data of the product photographed by the camera 405 based on an instruction related to photographing the product. The processor 401 functions as an acquisition unit 4012 in ACT203.

[0123] The processor 401 outputs the acquired photographed image data of the product to the monitoring server 26 in ACT204. The monitoring server 26 outputs the photographed image data to the monitoring terminal 23. The monitoring terminal 23 generates a monitoring image based on the acquired image data and displays it on a touch panel of the monitoring terminal 23 (not shown). The monitoring image includes photographed images of the product associated with the transaction data. The monitoring image may include multiple photographed images in chronological order based on photographed image data taken at different times. The processor 401 functions as a second output unit 4013 in ACT205.

[0124] Note that, similar to the processing of processor 261 described above, processor 401 repeats the processing of ACT201 to ACT205 during one transaction each time the second transaction data satisfies the image capturing condition. When processor 401 repeats the processing of ACT201 to ACT205 multiple times during one transaction, the monitoring image can include multiple captured images based on captured image data captured at different times in chronological order. In this case, an attendant checking the captured images of the items on monitoring terminal 23 can confirm the change in the state of the shopping cart accompanying the change in the registered number of items as the registered number increases. While the description here uses as an example a case where the image capturing condition is a condition related to the number of items, i.e., "every increase of 5 items," this is not limiting. Even if the image capturing condition is any of the above-mentioned various conditions other than the condition related to the number of items, processor 401 repeats the processing of ACT201 to ACT205 during one transaction each time the second transaction data satisfies the image capturing condition.

[0125] [Shooting guidance notification screen] FIG. 18 is a schematic diagram showing an example of a guidance screen Sf displayed on the smartphone 40 according to an embodiment.

[0126] In response to an instruction regarding photographing a product, the smartphone 40 outputs a notification instruction for product photographing guidance to the touch panel 404. FIG. 18 shows a guidance screen Sf on which the notification instruction for photographing guidance is displayed. As shown in FIG. 18, a frame ARb indicating the photographing area of ​​the product is displayed on the guidance screen Sf. A photographing button Bg is also displayed. The photographing button Bg is a button that a member touches when photographing a product. The frame ARb is set, for example, so that the entire shopping cart fits within it.

[0127] For example, the member operates the smartphone 40 so that the purchased item placed in the shopping cart is within the frame line ARb. The member touches the capture button Bg when the purchased item is within the frame line ARb. When the member touches the capture button Bg, the purchased item is photographed by the camera 405 of the smartphone 40. In this example, the smartphone 40 can prompt the member to photograph the item along the frame line ARb. This allows the smartphone 40 to acquire photographed image data photographed within a relatively consistent range. This reduces variation in the photographed images of the item. An attendant checking the photographed images of the item on the monitoring terminal 23 can easily compare multiple photographed images taken in chronological order.

[0128] [Effects of the transaction processing system 100] According to this embodiment, the processor 261 determines whether information regarding product registration satisfies the photographing conditions, and based on the determination result, outputs instructions for photographing the product and acquires photographed image data of the photographed product. This allows the monitoring server 26 to instruct the attendant to photograph the product when the registered product information entered by the smartphone 40 satisfies the monitoring conditions. Therefore, the monitoring server 26 can display, on the monitoring terminal 23, an image of the product to be monitored, photographed by the smartphone 40. The attendant can quickly and efficiently compare the purchased product with the registered product based on the monitoring image displayed on the monitoring terminal 23. Furthermore, by instructing the attendant to photograph the product when the monitoring conditions are met, the monitoring server 26 can warn the customer not to forget to register the product or to register it improperly. Furthermore, the smartphone 40, like the monitoring server 26, can use the functions of the processor 401 to assist the attendant in quickly and efficiently comparing the purchased product with the registered product.

[0129] According to this embodiment, the processor 261 determines whether a product satisfies a condition related to the product, and based on the determination result, outputs instructions for photographing the product and acquires photographed image data of the photographed product. This allows the monitoring server 26 to instruct the product to be photographed when the number, price, or type of the product among the registered product information entered by the smartphone 40 satisfies the conditions for monitoring. Therefore, the monitoring server 26 can set the monitoring target in detail. This allows the attendant to more quickly and efficiently match the purchased product with the registered product based on the monitoring image displayed on the monitoring terminal 23. Furthermore, the smartphone 40, using the functions of the processor 401, can assist the attendant in quickly and efficiently matching the purchased product with the registered product, just like the monitoring server 26.

[0130] According to this embodiment, the processor 261 determines whether the conditions for product registration are met, and based on the determination result, outputs instructions for photographing the product and acquires photographed image data of the photographed product. This allows the monitoring server 26 to instruct the product to be photographed when the conditions for the product registration operation or the time conditions among the information on the registered product entered by the smartphone 40 meet the conditions for monitoring. Therefore, the monitoring server 26 can set the monitoring target in detail. The attendant can more quickly and efficiently match the purchased product with the registered product based on the monitoring image displayed on the monitoring terminal 23. Furthermore, the smartphone 40, using the function of the processor 401, can support the attendant in quickly and efficiently matching the purchased product with the registered product, just like the monitoring server 26.

[0131] According to this embodiment, the processor 261 can output a notification instruction for product photography guidance and acquire photographed image data of the photographed product. As a result, the monitoring server 26 can cause the smartphone 40 to display photography guidance when the photography conditions are met, thereby encouraging the customer to take a photograph. Therefore, the customer carrying the smartphone 40 can photograph the product according to the displayed photography guidance when the photograph is required. The attendant can quickly and efficiently match the purchased product with the registered product based on the monitoring image including the image captured when the photograph is required. Furthermore, the smartphone 40, using the function of the processor 401, can support the attendant in quickly and efficiently matching the purchased product with the registered product, just like the monitoring server 26.

[0132] According to this embodiment, the processor 261 can output an instruction to photograph a product and acquire photographed image data of the photographed product. This allows the monitoring server 26 to control the smartphone 40 to automatically photograph the product when the photographing conditions are met. Therefore, the monitoring server 26 can reliably acquire photographed image data of the product at the timing when photographing is required, without requiring operation by the customer. The attendant can quickly and efficiently match the purchased product with the registered product based on the monitoring image including the image captured at the timing when photographing is required. Furthermore, the smartphone 40, using the function of the processor 401, can support the attendant in quickly and efficiently matching the purchased product with the registered product, just like the monitoring server 26.

[0133] Although the embodiment of the transaction processing system 100 has been described above, the embodiment is not limited to this.

[0134] In the above embodiment, the smartphone operated by the customer in the sales area may be a tablet terminal attached to a shopping cart. In this case, the tablet terminal has a built-in camera or an external camera. The smartphone operated by the customer may be a smartphone loaned to the customer by the store or a smartphone owned by the customer.

[0135] In the above embodiment, the server system 10 is constructed using cloud computing. In this regard, the POS system 20 may also include the functions of the server system 10. In other words, it is also possible to construct the transaction processing system 100 in one store.

[0136] In the above embodiment, the server system 10 is configured from the database server 11, the management server 12, and the virtual POS server 13. The server system 10 may be configured from a single server that integrates the functions of the database server 11, the management server 12, and the virtual POS server 13.

[0137] Similarly, in the POS system 20 of each store, rather than dividing the functions among the store server 21, accounting server 22, and monitoring server 26, the functions of the store server 21, accounting server 22, and monitoring server 26 may be consolidated into a single server.

[0138] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0139] 10...server system, 11...database server, 12...management server, 13...virtual POS server, 14...communication line, 15...router, 20...POS system, 21...store server, 22...accounting server, 23...monitoring terminal, 24...POS terminal, 25...dedicated accounting machine, 26...monitoring server, 27...in-store network, 28...router, 29...access point, 30...network, 40...smartphone, 70...linkage table, 80...transaction file, 90...scanner module, 91...reading window, 100...transaction processing system, 111...affiliated store database, 112...member database, 113...first product master, 211...second product master, 261...processor, 262...main memory, 263...auxiliary storage unit, 264...communication interface base, 265...transmission path, 400...stand, 401...processor, 402...main memory, 403...auxiliary memory unit, 404...touch panel, 405...camera, 406...audio output device, 407...wireless communication unit, 408...mobile communication unit, 409...transmission path, 2610...determination unit, 2611...first output unit, 2612...acquisition unit, 2613...second output unit, 2631...transaction data memory area, 2632...captured image memory area, 4010...determination unit, 4011...first output unit, 4012...acquisition unit, 4014...second output unit, 4031...smartphone POS app, 4032...transaction data memory area, C...cart, C1...caster unit, C2...handle frame unit, C3...basket holder unit, C4...pole, C21...vertical frame, C22...handle bar.

Claims

1. a determination unit that determines whether or not information related to the registration of a product satisfies a photographing condition; a first output unit that outputs instructions regarding photographing of the product based on the determination result of the determination unit; an acquisition unit that acquires photographed image data of the photographed product based on the instruction; a second output unit that outputs the photographed image data acquired by the acquisition unit; An information processing device comprising:

2. The photographing conditions include conditions related to the product. The information processing device according to claim 1 .

3. The photographing conditions include conditions related to product registration. The information processing device according to claim 1 .

4. the first output unit outputs a notification instruction for product photography guidance; The information processing device according to claim 1 .

5. the first output unit outputs an instruction to photograph a product. The information processing device according to claim 1 .

6. On the computer, a determination function for determining whether or not the information regarding the product registration satisfies the photographing conditions; a first output function that outputs instructions regarding photographing a product based on a determination result of the determination function; an acquisition function for acquiring photographed image data of the photographed product based on the instruction; a second output function for outputting the photographed image data acquired by the acquisition function; An information processing program for executing the above.

Images