Information processing apparatus, store system, program

Abstract

To enable whether operation by a customer for commodity purchase is correctly executed to be easily checked without making the operation by the customer complicated.SOLUTION: An information processing device in the present embodiment includes video receiving means, receiving means, counting means, comparison means, and storage means. The video receiving means receives video imaged by a camera. The receiving means receives identification data specific to a customer. The counting means counts the number of times the customer performs a predetermined motion for purchasing commodities based on the video in which the customer is imaged. The comparison means compares the number of times and the number of commodities for which commodity information has been registered as a target of purchase by the customer. The storage means causes comparison information indicating a result of the comparison to be stored.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus in a store such as a retail store. , Store System, Program

Background Art

[0002] In the retail industry, it has become difficult to secure store staff due to the worsening labor shortage. For this reason, in order to reduce the staff involved in operating the POS (Point Of Sales) terminal and customer service, the form of the POS system used in stores is also changing. Conventionally, many POS systems of the type in which a store clerk performs operations from product scanning (product registration) for registering information on products to be sold to money transactions have been used. In recent years, the number of self-POS systems in which customers themselves perform operations from product scanning to settlement processing has been increasing.

[0003] However, in the self-POS system, in order to assist customers who require complicated operations and to confirm whether the customers are performing correct operations to purchase products, it has been necessary to arrange store clerks called attendants. Therefore, even if a self-POS system is introduced, it has not been possible to achieve reliable staff reduction.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] An embodiment of the present invention has been made in consideration of the above-described circumstances, and an object thereof is to provide an information processing apparatus that can easily confirm whether an operation by a customer is correctly executed without complicating the operation for purchasing a product by the customer. , Store System, Program ​​

Means for Solving the Problem

[0006] To solve the above problems, the information processing apparatus in the present embodiment includes a video receiving means, a receiving means, a counting means, a collating means, and a storage means. The video receiving means receives the video captured by the camera. The receiving means receives Terminal code specific to the terminal operated by the customer from the electronic device. The counting means identifies the customers corresponding to the Terminal code in the video, counts the number of times the customers perform a predetermined operation for purchasing goods, and stores the number in association with the Unique to the customer identification data. The collating means collates the number of goods for which product information is registered as the purchase target by the customer with the number of times. The storage means stores the collation information indicating the result of the collation in association with the identification data.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Mode for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0009] FIG. 1 is an overall configuration diagram of a store system 1 according to this embodiment. The store system 1 is a self-service system capable of executing from product scanning for registering information on products to be sold to settlement processing by the operation of a customer. In product scanning, reading of the product code attached to the product is executed.

[0010] The store system 1 in this embodiment includes a store server 10, a plurality of cash registers 20 (20-1,..., 20-K), a check-in terminal 30, an access point (AP) 50, and cameras 60 (60-1,..., 60-M). The cash registers 20 are installed, for example, in multiple units (e.g., K units) in one store. Note that at least one cash register 20 may be installed. In FIG. 1, only one check-in terminal 30 and one access point 50 are shown, but there may be two or more depending on the scale of the store, etc. The cameras 60 are installed at positions where they can photograph customers moving within the store.

[0011] At least one camera 60 is installed. Basically, the cameras 60 are arranged to cover the store where customers can move. When a customer moving within the store enters the shooting range, the camera 60 can shoot a video in which the customer is reflected. Also, the camera 60 is arranged at a position where it can shoot a video that can confirm that the customer has performed a predetermined operation for purchasing a product. Therefore, it is desirable that the camera 60 can, for example, shoot a customer and the products displayed on the product shelves at the same time. Further, for one customer, the cameras 60 may be installed so that the customer can be shot by a plurality of cameras 60 at the same time. That is, by shooting the customer with a plurality of cameras 60 from different directions, it is possible to reduce the dead angles where it is difficult to confirm the customer's actions.

[0012] The store server 10, the cash registers 20, the check-in terminal 30, and the access point 50 are connected to a network 70 such as a LAN (Local Area Network).

[0013] The store system 1 can incorporate electronic devices 40 (40-1, …, 40-N) owned by customers into the system. The electronic devices 40 are, for example, smartphones, mobile phones, personal computers, tablet terminals, portable game terminals, etc. When incorporated into the store system 1, the electronic devices 40 perform wireless communication with the access point 50. Also, the electronic devices 40 may be dedicated terminals used for registering product information provided in the store. For example, the electronic devices 40 are attached to carts used in the store and are terminals dedicated to the store.

[0014] In the store system 1 of this embodiment, registration of product information may be executed using the electronic devices 40, or may be executed in the cash register 20. Also, in the store system 1, settlement processing for the registered product information may be executed by an external settlement system using the electronic devices 40, or may be executed in the cash register 20.

[0015] The check-in terminal 30 is installed at the entrance of the store or the like. The check-in terminal 30 has a check-in function of authenticating customers who visit the store and notifying the store server 10. Authentication of customers by the check-in terminal 30 can be performed by any one of a plurality of methods, or a combination of a plurality of methods. For example, as a first authentication method, there is a method of displaying a store-specific code (e.g., a two-dimensional code) on the display device (touch panel) of the check-in terminal 30 and having it read by the electronic device 40. Also, as a second authentication method, there is a method of displaying a terminal-specific terminal code (e.g., including a member ID which is identification data specific to the customer) on the electronic device 40 and having it read by the check-in terminal 30. Furthermore, as a third authentication method, there is a method of authenticating by inputting the biometric information of the customer without using the electronic device 40 and comparing it with the biometric authentication data recorded in advance at the time of membership registration. As biometric information for authenticating customers (members), for example, data indicating features such as face images, fingerprints, voices, irises, and vein patterns can be used.

[0016] The electronic device 40 is implemented with a function that allows a customer to perform operations such as inputting information about the product to be purchased by themselves. A plurality of electronic devices 40 are each set with unique identification information (terminal ID) and are managed by the store server 10.

[0017] The store server 10 controls the entire store system 1. The store server 10 in this embodiment has the function of a POS system for sales management of products and the function of a video analysis system for analyzing the video received from the camera 60. By collating (matching process) the data generated by the POS system and the video analysis system, the store server 10 can easily confirm whether the operations by the customer are being correctly executed without complicating the operations for purchasing products by the customer. For example, the store server 10 in this embodiment collates the number of product information (number of products) registered as purchase targets by the customer with the number of times the customer has performed a predetermined action for purchasing the product. The number of product information registered as purchase targets by the customer is managed by the POS system. Also, the number of times the customer has performed a predetermined action for purchasing the product is detected by the video analysis system by analyzing the video captured during the product purchase action for one transaction by the customer. When it is determined through the collation (matching process) that there is a possibility that the customer is not performing the correct operation, the store server 10 can store the result of the collation (result data (collation information)). Conditions for determining that there is a possibility that the customer is not performing the correct operation include, for example, when the number of times a predetermined action for purchasing a product has been performed is more than the number of products actually registered as purchase targets.

[0018] The store server 10 (POS system) creates transaction detail data including product information of the products purchased by the customer from the product codes input in the electronic device 40 or the accounting machine 20. The transaction detail data of the product includes data on the number of registered products (the number of times the product code is input). When accounting is performed in the accounting machine 20, the store server 10 identifies the customer who is being accounted for and transmits the transaction detail data of the products purchased by that customer to the accounting machine 20 used by the customer for accounting. Note that the transaction detail data may be generated in the electronic device 40 or the accounting machine 20 where the product code is input. In this case, the store server 10 receives and stores the transaction detail data from the electronic device 40 or the accounting machine 20.

[0019] Also, the store server 10 (video analysis system) receives the video captured by the camera 60 and performs various analysis processes based on this video. The analysis processes include, for example, customer identification, tracking of customers moving within the store, detection of customer actions, and identification of products that the customer intends to purchase.

[0020] In customer identification, for example, feature data is detected from the face image of the customer, and the customer is identified by comparing it with face authentication data (biometric information) indicating the features of the face image previously registered by the customer. In customer tracking, for example, the customer in the video is identified based on the color and shape of the customer's face image, the clothing and bags carried by the customer, and other belongings. The store server 10 performs analysis for identifying customers for each of the multiple videos received from the multiple cameras 60. The store server 10 manages by associating the customers detected from each video. In the detection of customer actions, various actions such as picking up a product displayed on a product shelf, returning a product taken out from the product shelf to the product shelf, putting a product into a shopping basket, and performing actions for registering product information using the electronic device 40 are detected. In product identification, analysis processes such as reading the product code attached to the product from the video and identifying the product based on the design and shape of the product package are performed.

[0021] In the video analysis system, for example, while one customer moves within the store for one transaction of shopping, the number of times the customer performs a predetermined action to purchase a product is counted.

[0022] In this embodiment, the video analysis system is realized in the store server 10, but it is also possible to realize the video analysis system by a device (video analysis device) different from the store server 10. The video analysis device is realized, for example, by a computer connected to the network 70. The video analysis device receives video data from the camera 60 and executes various analysis processes as described above. The video analysis device transmits data indicating the analysis result to the store server 10.

[0023] FIG. 2 is a block diagram showing the main circuit configuration of the store server 10. The store server 10 includes a processor 11, a main memory 12, an auxiliary storage device 13, a communication unit 14, and a system transmission path 15. The system transmission path 15 includes an address bus, a data bus, control signal lines, and the like. In the store server 10, the processor 11, the main memory 12, the auxiliary storage device 13, and the communication unit 14 are connected to the system transmission path 15. In the store server 10, a computer is configured by the processor 11, the main memory 12, and the auxiliary storage device 13, and the system transmission path 15 connecting them.

[0024] The processor 11 corresponds to the central part of the computer. The processor 11 controls each part in order to realize various functions as the store server 10 according to an operating system or an application program. The processor 11 is, for example, a CPU.

[0025] The main memory 12 corresponds to the main storage part of the computer. The main memory 12 includes a non-volatile memory area and a volatile memory area. In the non-volatile memory area, the main memory 12 stores an operating system or an application program. The main memory 12 may also store data necessary for the processor 11 to execute processes for controlling each part in either the non-volatile or volatile memory area. The main memory 12 uses the volatile memory area as a work area where data can be appropriately rewritten by the processor 11. The non-volatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

[0026] The auxiliary storage device 13 corresponds to the auxiliary storage part of the computer. For example, an HDD or an SSD etc. can be the auxiliary storage device 13. The auxiliary storage device 13 stores data used by the processor 11 to perform various processes, or data created by the processes in the processor 11, etc. The auxiliary storage device 13 may also store the above application program.

[0027] Stored in the auxiliary storage device 13 are, for example, a product database 61 and a member database 62. Also provided in the auxiliary storage device 13 are areas for storing transaction detail data 63 corresponding to a plurality of transactions, areas for storing video analysis data 64 generated by video analysis, video data 65 received from the camera 60, and areas for storing result data 66 indicating the results of collating data generated by the POS system and the video analysis system, etc.

[0028] Described in the product database 61 are product data records created for each product sold in the store. The product data records include items such as a product ID, a product name, and a price. The product ID is a unique code set for each product to individually identify each product. The product name and the price are the name of the product specified by that product ID and the selling price per unit.

[0029] The communication unit 14 controls data communication performed in accordance with a communication protocol between the accounting machine 20, the check-in terminal 30, the camera 60, or the electronic device 40 connected via the network 70.

[0030] Figure 3 is a block diagram showing the main circuit configuration of the accounting machine 20. The accounting machine 20 includes a processor 21, a main memory 22, an auxiliary storage device 23, a communication unit 24, a touch panel 25, a printer 26, a reader / writer 27, a human sensor 29, a change dispenser 210, a camera 211, and a system transmission path 212. The system transmission path 212 includes an address bus, a data bus, control signal lines, and the like. In the accounting machine 20, the processor 21, the main memory 22, the auxiliary storage device 23, the communication unit 24, the touch panel 25, the printer 26, the reader / writer 27, the human sensor 29, and the camera 211 are connected to the system transmission path 212. In the accounting machine 20, a computer is configured by the processor 21, the main memory 22, and the auxiliary storage device 23, and the system transmission path 212 that connects them.

[0031] The processor 21 corresponds to the central part of the computer. The processor 21 controls each part in order to realize various functions as the accounting machine 20 according to an operating system or an application program. The processor 21 is, for example, a CPU.

[0032] The main memory 22 corresponds to the main storage part of the computer. The main memory 22 includes a non-volatile memory area and a volatile memory area. The main memory 22 stores an operating system or an application program in the non-volatile memory area. The main memory 22 may store data necessary for the processor 21 to execute processes for controlling each part in the non-volatile or volatile memory area. The main memory 22 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 21. The non-volatile memory area is, for example, a ROM. The volatile memory area is, for example, a RAM.

[0033] The auxiliary storage device 23 corresponds to the auxiliary storage part of the computer. For example, an HDD or an SSD or the like can be the auxiliary storage device 23. The auxiliary storage device 23 stores data used by the processor 21 to perform various processes, or data created by the processes in the processor 21, etc. The auxiliary storage device 23 may store the above application program.

[0034] The communication unit 24 controls data communication performed according to a communication protocol with the store server 10 connected via the network 70.

[0035] The touch panel 25 is a device that serves as both an input device and a display device of the cash register 20. The touch panel 25 detects the touch position on the displayed image and outputs the touch position information to the processor 21.

[0036] The printer 26 issues a receipt by printing various character strings or images or the like on the receipt paper. As this type of printer 26, for example, a thermal printer or a dot impact printer or the like can be used.

[0037] The reader / writer 27 has a function of reading data recorded on a recording medium and a function of writing data to the recording medium. The reader / writer 27 may be a contact type reader / writer or a non-contact type reader / writer. The recording medium is a settlement card such as a credit card, a debit card, an electronic money card, a prepaid card, etc. A wireless communication device such as a smartphone having a function equivalent to that of a settlement card may be used as the recording medium.

[0038] The human presence sensor 29 detects the presence of a person standing in front of the cash register 20 using, for example, infrared rays, ultrasonic waves, etc. When the presence of a person is detected, the human presence sensor 29 turns on.

[0039] The change dispenser 210 receives the coins and banknotes inserted. The change dispenser 210 also discharges coins and banknotes as change.

[0040] The camera 211 captures an image (video) for, for example, identifying the customer and the product the customer intends to purchase. In customer identification, for example, feature data is detected from the face image of the customer, and the customer is identified by comparing it with face authentication data (biometric information) indicating the features of the face image registered in advance by the customer. In product identification, analysis processing such as reading the product code attached to the product from the image or identifying the product based on the design or shape of the product package is executed.

[0041] The accounting machine 20 is set with a unique accounting machine ID for individual identification. The accounting machine ID is stored, for example, in the auxiliary storage device 23.

[0042] FIG. 4 is a block diagram showing the main circuit configuration of the check-in terminal 30. The check-in terminal 30 includes a processor 31, a main memory 32, an auxiliary storage device 33, a communication unit 34, a touch panel 35, a camera 36, a speaker 37, and a system transmission path 39. The system transmission path 39 includes an address bus, a data bus, control signal lines, etc. In the check-in terminal 30, the processor 31, the main memory 32, the auxiliary storage device 33, the communication unit 34, the touch panel 35, the camera 36, and the speaker 37 are connected to the system transmission path 39. In the check-in terminal 30, a computer is constituted by the processor 31, the main memory 32, and the auxiliary storage device 33, and the system transmission path 39 connecting them.

[0043] The processor 31 corresponds to the central part of the computer. The processor 31 controls each part in order to realize various functions as the check-in terminal 30 according to the operating system or application program. The processor 31 is, for example, a CPU (Central Processing Unit).

[0044] The main memory 32 corresponds to the main storage part of the computer. The main memory 32 includes a non-volatile memory area and a volatile memory area. In the non-volatile memory area, the main memory 32 stores an operating system or an application program. The main memory 32 may store, in either the non-volatile or volatile memory area, data necessary for the processor 31 to execute processes for controlling each part. The main memory 32 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 31. The non-volatile memory area is, for example, a ROM (Read Only Memory). The volatile memory area is, for example, a RAM (Random Access Memory).

[0045] The auxiliary storage device 33 corresponds to the auxiliary storage part of the computer. For example, an HDD (Hard Disc Drive), an SSD (Solid State Drive), or the like can be the auxiliary storage device 33. The auxiliary storage device 33 stores data used by the processor 31 to perform various processes, data created by the processes in the processor 31, and the like. The auxiliary storage device 33 may store the above application program.

[0046] The communication unit 34 controls data communication performed according to a communication protocol with the store server 10 connected via the network 70. Note that the check-in terminal 30 may be configured to be provided with a wireless communication unit and perform wireless communication according to a wireless communication protocol with the access point 50.

[0047] The touch panel 35 is a device that serves as both an input device and a display device of the check-in terminal 30. The touch panel 35 detects a touch position on the displayed image and outputs the touch position information to the processor 31.

[0048] The camera 36 captures video (images) to authenticate the customers who visit the store. The camera 36 captures an image of a terminal-specific terminal code (for example, a two-dimensional code including member ID data) displayed on the electronic device 40 held by the customer who visits the store (second authentication method). Also, when the camera 36 authenticates the customer based on the features of the face image as the biometric information of the customer, the camera 36 captures the face image of the customer (third authentication method).

[0049] The speaker 37 outputs a message voice or the like to the customers who visit the store.

[0050] Figure 5 is a block diagram showing the main configuration of the electronic device 40. The electronic device 40 includes a processor 41, a main memory 42, an auxiliary storage device 43, a touch panel 44, a camera unit 45, a wireless unit 46, and the like. The processor 41, the main memory 42, the auxiliary storage device 43, the touch panel 44, the camera unit 45, and the wireless unit 46 are connected by a system transmission path 47 including an address bus, a data bus, control signal lines, and the like.

[0051] The electronic device 40 constitutes a computer by the processor 41, the main memory 42, the auxiliary storage device 43, and the system transmission path 47 connecting these.

[0052] The processor 41 corresponds to the central part of the computer. The processor 41 controls each part to realize various functions as the electronic device 40 according to the operating system and application programs.

[0053] The main memory 42 corresponds to the main storage part of the computer. The main memory 42 includes a non-volatile memory area and a volatile memory area. In the non-volatile memory area, the main memory 42 stores an operating system and application programs. Also, the main memory 42 may store, in either the non-volatile or volatile memory area, data necessary for the processor 41 to execute processes for controlling each part. The main memory 42 uses the volatile memory area as a work area where data is appropriately rewritten by the processor 41.

[0054] The auxiliary storage device 43 corresponds to the auxiliary storage part of the computer. As the auxiliary storage device 43, for example, an HDD, an SSD, etc. may be used. The auxiliary storage device 43 stores data used by the processor 41 for various processes and data generated by the processes in the processor 41. The auxiliary storage device 43 may also store an application program.

[0055] The touch panel 44 functions as an input device and a display device of the electronic device 40. Icons for starting application programs are displayed on the touch panel 44.

[0056] The camera unit 45 has a resolution capable of photographing a barcode indicating information of a product attached to the product. The wireless unit 46 transmits and receives data using wireless communication with the access point 60. The electronic device 40 can be used in the store system 1 when it incorporates the camera unit 45 and the wireless unit 46 among portable information processing devices such as smartphones, mobile phones, and tablet terminals.

[0057] The electronic device 40 installs the product registration program P1 and operates as an information terminal corresponding to the store system 1 by the functions realized by the product registration program P1.

[0058] The product registration program P1 controls the processor 41 so as to be a terminal for registering product information by a customer's operation. The product registration program P1 is stored in the auxiliary storage device 43. Also, an area for storing information such as product information is provided in the auxiliary storage device 43 together with the product registration program P1. Further, the product registration program P1 receives, from the store server 10, a member ID unique to a customer issued by registering as a member (registering member information) in the store server 10 and manages (stores) it.

[0059] Next, the data stored in the auxiliary storage device 13 of the store system 1 will be described.

[0060] FIG. 6 is a schematic diagram showing the main data structure of the member data record 62R registered in the member database 62 of the store server 10. FIG. 6 shows data corresponding to one member. As shown in FIG. 6, the member data record 62R has member information provided by the member registered in association with a member ID unique to the customer who registered as a member. The member information includes biometric information and personal information as information for identifying or authenticating the member (customer). As the biometric information, for example, data indicating features such as a face image, fingerprint, voice, iris, and vein pattern can be used. The personal information includes, for example, data indicating name, address, telephone number, payment method (for example, cash, electronic money, credit), and the like.

[0061] FIG. 7 is a diagram showing the data structure in the transaction detail data 63 of the store server 10. The transaction detail data 63 is product information registered by a customer's product scanning (input of product code). The transaction detail data 63 includes a member ID 63A, a transaction ID 63B, a purchase point 63C, and purchase product information 63D. The member ID 63A is set to the member ID of the customer authenticated at the time of visiting the store. The transaction ID 63B is generated to identify one transaction by the authenticated customer and is set in association with the member ID 63A. The purchase product information 63D is information on the products purchased by the customer identified by the member ID. For the purchase product information 63D, the product ID, product name, price, etc. corresponding to the detected product code are read from the product database 61 and registered. The purchase point 63C is information indicating the number of products for which the purchase product information 63D is registered. The transaction detail data is received from the electronic device 40 when product information is registered in the electronic device 40, and is received from the accounting machine 20 when product information is registered in the accounting machine 20.

[0062] FIG. 8 is a diagram showing the data structure in the video analysis data 64 of the store server 10. The video analysis data 64 is data showing the analysis result of the video received from the camera 60. The video analysis data 64 includes data such as a member ID 64A, a check-in date and time 64B, the number of operations 64C, and a check-out date and time 64D. The check-in date and time 64B indicates the date and time (indicating year, month, day, hour, and minute) when the customer checked in (visited the store). The number of operations 64C indicates the number of times the authenticated customer performed a predetermined operation to purchase a product during the period when the customer was being tracked (during the product purchase action for one transaction). The check-out date and time 64D indicates the date and time (indicating year, month, day, hour, and minute) when the customer checked out (completion of the settlement process).

[0063] Furthermore, although not shown, the video analysis data 64 may store data for identifying the camera 60 that captured the video in which a predetermined action for a customer to purchase a product was detected. For example, the camera 60 is installed in association with a sales floor of a specific product. Thereby, when a predetermined action by a customer is detected by analyzing the video, the camera 60 that captured the video to be analyzed can be identified, and the sales floor of the product where the camera 60 is installed can be specified. Therefore, it is possible to determine at which sales floor of which product the customer performed the predetermined action. Furthermore, by determining whether product information of the product sold at the sales floor where the customer performed the predetermined action is registered, the correspondence between the registration of the product information and the sales floor where the customer performed the predetermined action can be verified. By this verification, for example, it can be determined that the product information of the product sold at the sales floor where the predetermined action was performed is not registered, or that the number of actions at the sales floor where the predetermined action was performed does not match the number of registered products sold at the corresponding sales floor.

[0064] FIG. 9 is a diagram showing the data structure of the video data 65 of the store server 10. The video data 65 includes tracking video data 65B used for tracking one customer moving inside the store. The tracking video data 65B is stored in association with the transaction ID 65A and includes video data in which the product purchase behavior for one transaction by one customer is captured. Usually, the tracking video data 65B includes video data captured by a plurality of cameras 60.

[0065] FIG. 10 is a diagram showing the data structure of the result data 66 of the store server 10. The result data 66 shows the result (matching information) of collating the data generated by the POS system and the video analysis system. In the result data 66, in association with the member ID 66A and the transaction ID 66B, collation result data 66C indicating that the number of times a predetermined operation was performed is greater than the number of products is stored as the collation result (collation information). Note that the collation result data 66C may include data indicating not only that the number of times a predetermined operation was performed is greater than the number of products, but also that the number of products is greater than the number of times the operation was performed. Further, the collation result data 66C may include data indicating the difference (number of mismatches) between the number of times the operation was performed and the number of registered products. Furthermore, the collation result data 66C may include data indicating that the installation location of the camera 60 that captured the video in which a predetermined operation was detected (for example, corresponding to the sales floor of a specific product) and the registered product information were determined to be mismatched, and the like.

[0066] Next, the operation of the store system 1 in the present embodiment will be described. First, the operation when product information of a product to be purchased is registered by the customer's own operation using the electronic device 40 will be described.

[0067] FIG. 11 is a flowchart for explaining the operation of the store server 10 (processor 11) in the present embodiment. FIG. 12 is a flowchart showing the operation of the electronic device 40 (processor 41) in the present embodiment. Note that the operations shown in FIGS. 11 to 12 are examples, and the processing procedures and processing contents are not particularly limited as long as the same results can be obtained.

[0068] First, when the customer visits the store, the customer operates the electronic device 40 to start the product registration program P1 and start the product registration process. When the product registration process is started, the processor 41 executes the check-in process based on the product registration program P1 (FIG. 12, Act21).

[0069] FIG. 13 is a flowchart showing the first check-in process executed by the electronic device 40 in the present embodiment, and FIG. 14 is a flowchart showing the second check-in process. The first check-in process identifies a customer using the electronic device 40, and the second check-in process identifies a customer using the biometric information of the customer.

[0070] It is assumed that the store system 1 has determined in advance which of the first check-in process and the second check-in process to use. Also, a configuration in which the first check-in process and the second check-in process are used in combination may be employed.

[0071] First, the first check-in process will be described with reference to the flowchart shown in FIG. 13. When the processor 41 is instructed to make an in-store purchase by an operation on the touch panel 44 of the customer (YES in Act 41 in FIG. 13), the camera unit 45 is activated and the process shifts to the state of reading the store code.

[0072] The check-in terminal 30 displays a store code unique to the store, for example, a two-dimensional code pattern, on the touch panel 35. The customer operates the electronic device 40 so that the store code displayed on the check-in terminal 30 is read by the camera unit 45 of the electronic device 40.

[0073] When the processor 41 reads the store code (Act 42), it transmits a check-in request to the store server 10 together with the member ID stored in advance in the electronic device 40 and the date and time (year / month / day / hour / minute / second) when the store code was read (Act 43).

[0074] The store server 10 receives the check-in request from the electronic device 40 through the access point 50. When the processor 11 of the store server 10 receives the member ID together with the check-in request (YES in Act 1), it writes the member ID in the work area of the main memory 12. Then, the processor 11 searches the member database 62 using the member ID as a search key and determines the presence or absence of a member data record 62R in which the member ID is described (Act 2).

[0075] When the processor 11 confirms that there is no corresponding member data record 62R (NO in Act3), it sends a check-in not permitted notification to the electronic device 40 (FIG. 11, Act4). When the processor 41 of the electronic device 40 receives the check-in not permitted notification from the store server 10 (NO in Act44 of FIG. 13), it outputs a not permitted notification for notifying that the check-in is not permitted (Act45). For example, the processor 41 causes a message indicating that the check-in is not permitted to be displayed on the touch panel 44. In this case, the electronic device 40 ends the first check-in process and also ends the product registration process.

[0076] On the other hand, when the processor 11 confirms that there is a corresponding member data record 62R, it further checks whether data (matching information) indicating a preset matching result is set in the result data 66 in association with the corresponding member ID. That is, it discriminates whether it has been determined that there is a possibility that the customer corresponding to the member ID did not perform correct operations in the subsequent matching (collation process) when coming to the store and shopping before the previous time. For example, when it is determined that the number of times of performing a predetermined operation for purchasing products during the previous shopping is more than the number of products actually registered as purchase targets, collation result data 66C indicating that the number of times of performing the predetermined operation is more than the number of products is stored.

[0077] Here, when the collation result data 66C is stored, the processor 11 sends a check-in not permitted notification to the electronic device 40 according to the collation result indicated by the collation result data 66C (FIG. 11, Act4). For example, the processor 11 sends a check-in not permitted notification indicating that the store cannot be used, the membership registration is stopped, confirmation by a store clerk is required, etc., according to the collation result indicated by the collation result data 66C. When the processor 41 of the electronic device 40 receives the check-in not permitted notification from the store server 10 (NO in Act44 of FIG. 13), it outputs a not permitted notification for notifying that the check-in is not permitted, and ends the first check-in process and also ends the product registration process (Act45).

[0078] In addition, when the collation result data 66C is stored, instead of always disallowing check-in, the permission / non-permission of check-in may be determined according to the content of the collation result data 66C. For example, when the collation result data 66C contains data indicating the difference (number of mismatches) between the number of times a predetermined operation for purchasing a product has been performed and the number of registered products, if the number of mismatches is equal to or greater than a preset number (for example, 3 or more), check-in can be disallowed. Also, if the same member ID is registered in the result data 66 more than a preset number (for example, 2), check-in can be disallowed. It is possible to preset other conditions for allowing check-in.

[0079] In addition, when the member ID is registered in the result data 66, the store server 10 or the information processing device used by the store clerk may be notified of the information of the corresponding member. Also, when the information of the member is notified, check-in may be permitted so that the store clerk can handle the customers who have come to the store. In this case, although it is necessary for the store clerk to handle the customers, it is not necessary to always arrange a store clerk called an attendant, so it does not become an obstacle to reducing the number of store clerks.

[0080] When the member ID received in response to the check-in request exists in the member data record 62R and it is confirmed that the corresponding member ID is not registered in the result data 66 (YES in Act3), the processor 11 sends a check-in permission notification to the electronic device 40 (Act5). The processor 11 generates a member ID of the customer who has been permitted to check in and a transaction ID for identifying one transaction by this customer, and stores them in the auxiliary storage device 13 (Act6).

[0081] When the processor 41 of the electronic device 40 receives a check-in permission notification from the store server 10 (YES in Act44 of FIG. 13), it becomes possible to register product information by the operation of the customer. The registration of product information will be described later.

[0082] Next, the second check-in process will be described with reference to the flowchart shown in FIG. 14. In the second check-in process, the customer is identified using the customer's biometric information. Here, for example, a case where feature data of the customer's face image is used as biometric information will be described.

[0083] The customer requests an authentication process by having the camera 36 of the check-in terminal 30 take a face image. The processor 31 of the check-in terminal 30 detects feature data from the face image captured by the camera 36 and compares it with the biometric information (feature data of the face image) of each member registered in the member database 62 of the store server 10.

[0084] When the processor 31 can authenticate the member through the biometric information comparison, in the same manner as in the first check-in process, the processor 31 inquires of the store server 10 whether the member ID of the corresponding member is registered in the result data 66. Here, when the verification result data 66C is stored, the processor 31, in the same manner as in the first check-in process, sends a check-in prohibition notice to the electronic device 40. As a method for notifying the authentication result, for example, an authentication code (e.g., a two-dimensional code pattern) indicating the authentication result is displayed on the touch panel 35. In this case, the customer operates the electronic device 40 to have the camera unit 45 of the electronic device 40 read the authentication code in the same manner as in the first check-in process.

[0085] The processor 41 of the electronic device 40 receives the authentication result from the check-in terminal 30 (FIG. 14, Act51). When the notification is a check-in prohibition notice (NO in FIG. 14, Act52), the processor 41 outputs a prohibition notice for notifying that the check-in is prohibited in the same manner as described above, ends the second check-in process, and ends the product registration process (Act55).

[0086] In addition, when it is determined that check-in is not permitted at the check-in terminal 30, it may simply display a notice of non-permission of check-in on the touch panel 35 of the check-in terminal 30 (and further output a message from the speaker 37).

[0087] On the other hand, the processor 31 can authenticate a member by collating biometric information, and when the member ID of the corresponding member is not registered in the result data 66, it notifies an authentication result indicating permission of check-in in the same manner as described above.

[0088] When the processor 41 of the electronic device 40 receives a notice of check-in permission (YES in Act52), it transmits a check-in request to the store server 10 together with the member ID stored in advance in the electronic device 40 and the date and time (year / month / day / hour / minute) when the authentication code was read (Act53).

[0089] When in-store purchase is instructed by an operation on the touch panel 44 of the customer (YES in Act54), the processor 41 becomes in a state where product information can be registered by the operation of the customer. That is, the processor 41 activates the camera unit 45 and shifts to a state of reading the product code.

[0090] Hereinafter, the case where check-in by the customer is completed will be described.

[0091] The processor 11 of the store server 10 identifies the customer who has checked in from the video captured by the camera 60. As features for identifying the customer from the video, for example, the face image of the customer, the color and shape of the belongings such as the clothes and bags worn by the customer can be used. Note that the identification of the customer may be executed using the video captured by the camera 36 provided in the check-in terminal 30.

[0092] Hereinafter, the processor 11 analyzes whether a customer is being photographed based on the characteristics of the customer from the videos taken by a plurality of cameras 60 arranged at various locations, and tracks the customers moving within the store (Act7). The processor 11 stores the video data identified as the customer to be tracked in association with the transaction authentication 65A as the video data 65 as shown in FIG. 9.

[0093] On the other hand, when the processor 41 of the electronic device 40 receives the check-in permission notification, it causes the product list screen to be displayed on the touch panel 44 and enters a standby state for input operations by the customer (Act22). Further, the processor 41 causes a message for notifying that the operation state has shifted to a state where product registration is possible to be displayed on the touch panel 35. For example, the processor 41 causes a message such as "Check-in was successful. Please start shopping" to be displayed on the touch panel 35. Note that the electronic device 40 may output a message by voice.

[0094] The processor 41 of the electronic device 40 causes, for example, the product list screen SC1 shown in FIG. 15 to be displayed on the touch panel 44. Note that the product list screen SC1 shown in FIG. 15 shows an example of a state in which some products have already been registered, and no product information is displayed in the initial state.

[0095] The product list screen SC1 shown in FIG. 15 is provided with a registered product list area 70, a product to be registered area 71, a total display area 72, a checkout button 73, a product registration button 74, and the like. The registered product list area 70 is an area for displaying a list of product names, quantities, unit prices, and amounts for registered products. The product to be registered area 71 is an area for displaying the product name, quantity, and total amount of the last registered product. The total display area 72 is an area for displaying the quantity and total amount of registered products.

[0096] The checkout button 73 is a button for notifying the store server 10 that product registration has ended and the process is to shift to the checkout process. The product registration button 74 is a button for instructing a shift to the product registration mode.

[0097] Note that the product list screen SC1 only represents a basic form for displaying various types of information regarding the registered products, and it is possible to display various types of information related to product registration other than those shown in FIG. 15.

[0098] When an operation on the product registration button 74 is detected on the touch panel 44 (YES in Act23), the processor 41 activates the camera unit 45 and displays a product code reading screen for reading product information on the touch panel 44 in place of the product list screen SC1 (Act24).

[0099] FIG. 16 is a diagram showing an example of the product code reading screen SC2. The product code reading screen SC2 is provided with an image display area 81 and a product list button 83. The image display area 81 is an area for displaying an image captured by the camera unit 45, and a line 82 indicating the barcode reading position is displayed. The product list button 83 is a button for instructing the cancellation of the reading of product information and the display of the product list screen SC1. On the product code reading screen SC2, a message such as "Please align the barcode with the line" is displayed. Thereby, the customer can easily grasp the product information registration operation using the product code reading screen SC2.

[0100] When the customer finds a product to be purchased at the store, the customer captures the product code symbol such as the barcode attached to the product with the camera unit 45 of the electronic device 40. The image captured by the camera unit 45 is displayed in the image display area 81. The customer adjusts the display range so that the product code symbol (barcode) is displayed at the position of the line 82.

[0101] The processor 41 of the electronic device 40 extracts a code symbol from the image captured by the camera unit 45, and reads, for example, a product ID (product code) unique to the product indicated by the code symbol. When the product ID is read from the image captured by the camera unit 45 (YES in Act25), the processor 41 transmits product registration data including the product ID and the member ID used in the check-in request to the store server 10 (Act27).

[0102] If the code symbol cannot be extracted from the image for a certain period of time (YES in Act26), the processor 41 changes the display from the product code reading screen SC2 to the product list screen SC1 (Act22).

[0103] When the processor 11 of the store server 10 receives the product registration data from the electronic device 40, it searches the product database 61 using the product ID as a search key. The processor 11 obtains a product data record corresponding to the product ID from the product database 61. The product data record includes the product ID, product name, price, etc. The processor 11 transmits the product data record to the electronic device 40.

[0104] When the processor 41 of the electronic device 40 receives the product data record from the store server 10 (Act28), it generates transaction detail data and stores it in the auxiliary storage device 43. Also, the processor 41 switches the display of the touch panel 44 from the product code reading screen SC2 to the product list screen SC1, and updates the display of the registered product list area 70, the registration target product area 71, and the total display area 72 based on the product name, price, etc. included in the purchased product information of the transaction detail data.

[0105] A customer who has confirmed that the product name, price, etc. of the product whose product information (product ID) has been read are displayed on the touch panel 44 continues shopping. When the customer finds a product to purchase, the customer inputs the product ID of that product into the electronic device 40. When the processor 41 of the electronic device 40 detects that the product ID has been input, in the same manner as described above, it transmits the product registration data including the product ID to the store server 10. The processor 41 updates the transaction details data based on the product data record received from the store server 10 and changes the display of the product list screen SC1. The transaction details data includes data indicating the number of products registered as purchase targets.

[0106] The customer repeats the operation of finding a product to purchase while moving around the store, taking it out of the product shelf, and inputting the product ID of that product into the electronic device 40. After inputting the product ID from the electronic device 40, the customer places the product in the cart or puts it in a bag. Also, the customer may take out a product from the product shelf, check it, and then put the product back on the product shelf without purchasing it. Furthermore, as an improper operation by the customer, there is a possibility that the customer may put a product in a bag without inputting the product ID from the electronic device 40.

[0107] While tracking a customer moving around the store, the processor 11 of the store server 10 analyzes whether the customer has performed a predetermined operation to purchase a product based on the video captured by the camera 60 (FIG. 11, Act8). When the processor 11 determines by analyzing the video that the customer has, for example, taken out a product from the product shelf (YES in Act9), it increments (+1) the number of times the customer has performed a predetermined operation to purchase a product (Act10). Also, when the processor 11 determines by analyzing the video that the customer has, for example, put back the product taken out from the product shelf on the product shelf (YES in Act11), it decrements (-1) the number of times the customer has performed a predetermined operation to purchase a product (Act12). That is, the processor 11 of the store server 10 analyzes the actions of the customer to be tracked and counts the number of times the customer has performed a predetermined operation to purchase a product until checkout (Act7~13).

[0108] After the customer finishes shopping in the store, the customer touches the payment button 73 displayed on the touch panel 44 to declare the execution of payment. When the processor 41 of the electronic device 40 detects that the payment button 73 has been touched (YES in Act29), it accepts the specification of the payment method. Here, when cashless payment is specified (YES in Act30), the processor 11 allows the customer to specify the payment method (Act31). Here, for example, the credit company available to the customer is specified.

[0109] The processor 11 executes the payment for the products registered in the transaction details data by the payment method specified by the customer (for example, the payment system by the credit company) (Act32).

[0110] On the other hand, when cashless payment is not specified (NO in Act30), the processor 11 generates a transaction barcode including the content of the transaction details data and displays it on the touch panel 44 (Act33). The customer moves to the installation location of the payment machine 20 where the payment process is performed while holding the electronic device 40. The payment machine 20 turns on the human presence sensor 29 when the customer approaches. When the human presence sensor 29 is turned on, the processor 21 of the payment machine 20 displays a message for requesting the input of the transaction barcode on the touch panel 25. The customer allows the transaction barcode displayed on the touch panel 44 to be read by the camera 211 of the payment machine 20.

[0111] The processor 21 of the payment machine 20 displays on the touch panel 25 the payment details (product name, price, total, number of products, etc.) indicated by the transaction barcode. In addition, the processor 21 displays on the touch panel 25 a screen for inputting the specification of the payment method and allows the customer to specify the payment method. Here, for example, either cash or electronic money is specified. When the payment method is specified by the operation on the touch panel 25, the processor 21 executes the payment process by the specified payment method.

[0112] For example, in the case of payment in cash, an amount equivalent to the total amount is paid through the change machine 210. Also, in the case of payment by electronic money, the payment is made from the electronic money card read by the reader / writer 27. When the settlement process is completed, the processor 21 notifies the electronic device 40 of the completion of the settlement.

[0113] When the processor 41 of the electronic device 40 receives the settlement completion notification (YES in Act 34 of FIG. 12), it transmits the transaction detail data of one transaction for which the settlement process has been completed to the store server 10 (Act 35). The store server 10 stores the transaction detail data 63 received from the electronic device 40 in the auxiliary storage device 13. Also, the processor 41 transmits a checkout request including each data of the member ID, the date and time when the settlement process was completed, and the transaction ID to the store server 10 (Act 36).

[0114] When the processor 11 of the store server 10 receives the checkout request from the electronic device 40 (YES in Act 13 of FIG. 11), it ends the tracking for the customer corresponding to the member ID included in the checkout request (Act 14). Also, the processor 11 stores the video analysis data 64 for one transaction for which the settlement has been completed in the auxiliary storage device 13. As shown in FIG. 8, the video analysis data 64 includes data such as the member ID 64A, the check-in date and time 64B, the number of operations 64C, and the checkout date and time 64D.

[0115] The processor 11 specifies the purchase amount 63C in the transaction detail data 63 and the number of operations 64C registered in the video analysis data 64 using the transaction ID of the transaction for which the settlement process has been completed as a key. The processor 11 executes a matching process (Act 16) to collate the purchase amount 63C and the number of operations 64C to determine whether there is a numerical discrepancy. That is, the processor 11 compares the number of pieces of product information (number of products) registered as purchase targets by the customer with the number of times the customer has performed a predetermined operation (such as taking out a product from the product shelf) to purchase the product.

[0116] When the processor 11 determines as a result of the matching process that, for example, the number of times a predetermined operation for purchasing a product has been performed is greater than the number of products actually registered as purchase targets, the verification result data 66C is stored in the auxiliary storage device 13 (result data 66) in association with the member ID 66A and the transaction ID 66B. In this case, the verification result data 66C indicates that the number of times the predetermined operation has been performed is greater than the number of products.

[0117] In this way, by storing the result data 66, the store server 10 can call the corresponding tracking video data 65B corresponding to the transaction ID 66B from the video data 65 based on the transaction ID 66B associated with the verification result data 66C. That is, the video taken while the customer is being tracked can be displayed on a display device (not shown) provided in the store server 10, the touch panel 25 of the cash register 20, or the display device of another information processing device (personal computer) connected to the network 70. As a result, the customer's product purchase behavior can be confirmed by video. By this confirmation, the content of the data stored in the result data 66 can be changed, for example, by the operation of the administrator of the store server 10. For example, when it is confirmed that the customer is performing a correct operation, the verification result data 66C can be deleted (or the content changed) so that check-in is not prohibited in the check-in process at the next visit to the store.

[0118] In this way, in the store system 1 according to the present embodiment, by checking the operations for purchasing products by the customer and performing a matching process on the number of times a predetermined operation for purchasing a product has been performed and the number of products actually registered as purchase targets, it is possible to easily confirm whether the operations by the customer are being executed correctly. Since the customer only needs to perform the operations for purchasing normal products using the electronic device 40, the operations for purchasing products by the customer are not complicated.

[0119] Next, the operation when the product information of the products to be purchased is registered by the customer's own operation using the accounting machine 20 will be described. For example, in the accounting machine 20, it is possible to register the product information of a plurality of products at once.

[0120] FIG. 17 is a flowchart for explaining the operation of the store server 10 (processor 11) in the present embodiment. FIG. 18 is a flowchart showing the operation of the check-in terminal 30 (processor 31) in the present embodiment. FIG. 19 is a flowchart showing the operation of the accounting machine 20 (processor 21) in the present embodiment. Note that the operations shown in FIGS. 17 to 19 are examples, and the processing procedures and processing contents are not particularly limited as long as the same results can be obtained.

[0121] The check-in terminal 30 manages the customer's check-in using the customer's biometric information. Here, for example, the case of using the feature data of the customer's face image as biometric information will be described.

[0122] The customer requests an authentication process by having the camera 36 of the check-in terminal 30 take a face image. The processor 31 of the check-in terminal 30 detects the feature data from the face image captured by the camera 36 and collates it with the biometric information (feature data of the face image) of each member registered in the member database 62 of the store server 10 (FIG. 18, Act81).

[0123] When the processor 31 cannot authenticate the member by collating the biometric information (NO in Act82), it may cause the check-in terminal 30 to display a check-in not permitted notification on the touch panel 35 (and further output a message from the speaker 37) (Act85).

[0124] Also, when the processor 31 can authenticate a member through biometric information verification, it inquires of the store server 10 whether the member ID of the corresponding member is registered in the result data 66 in the same manner as in the first check-in process. Here, when the verification result data 66C is stored, the processor 31 causes a check-in not-permitted notification to be displayed on the touch panel 35 of the check-in terminal 30 (Act85).

[0125] On the other hand, when the processor 31 can authenticate a member through biometric information verification and the member ID of the corresponding member is not registered in the result data 66, it may cause a check-in permitted notification to be displayed on the touch panel 35 of the check-in terminal 30 (and further output a message from the speaker 37) (Act83). The processor 31 generates a member ID of the customer who has been permitted to check in and a transaction ID for identifying one transaction made by this customer. The processor 31 transmits a check-in notification including the member ID and the transaction ID to the store server 10 (Act84).

[0126] When the processor 11 of the store server 10 receives a check-in request from the check-in terminal 30 (YES in FIG. 17, Act61), it stores the member ID and the transaction ID included in the check-in notification in the auxiliary storage device 13 (Act62).

[0127] Hereinafter, when the check-in by the customer is completed, the store server 10 starts tracking the customer who has checked in (Act63). Note that the description of the processes of Act63 to Act68 shown in FIG. 17 is omitted as it is executed in the same manner as the processes of Act7 to Act10 shown in FIG. 11.

[0128] While moving inside the store, the customer takes out the products to be purchased from the product shelves and places them on, for example, a cart. When the selection of the products to be purchased is completed, the customer moves to the location where the cashier 20 is installed. When the customer approaches, the cashier 20 turns on the human presence sensor 29. When the human presence sensor 29 is turned on, the processor 21 of the cashier 20 shifts to the operation state of product registration (Fig. 19, Act91). The processor 21 activates the camera 211 and takes pictures of the products placed in the cart or on a reading stand, etc. Here, for example, it is assumed that all products are photographed at once.

[0129] The processor 21 reads the product codes attached to the products from the images taken by the camera 211, or executes analysis processing such as identifying the products based on the design or shape of the product packages, etc., to identify the products to be purchased.

[0130] The processor 21 transmits the product registration data indicating the identified products to the store server 10 and receives the product data records stored in the product database 61 of the corresponding products. The processor 21 generates transaction detail data based on the product data records received from the store server 10. Also, the processor 21 displays, for example, the product list screen shown in Fig. 15 on the touch panel 25 according to the content of the transaction detail data.

[0131] When the processor 21 detects that the checkout button on the product list screen has been touched (YES in Act92), it accepts the designation of the payment method. The processor 21 determines the payment method according to the operation by the customer to designate the payment method (Act93).

[0132] Here, the processor 21 executes authentication processing for the customer (Act94). For example, the processor 21 executes authentication using the biometric information (e.g., face image) of the customer in the same way as the above-described check-in terminal 30.

[0133] When the processor 21 fails to authenticate a member through biometric information verification (NO in Act 95), it causes the touch panel 25 of the cash register 20 to display a non - permission notice (Act 97). On the other hand, when the processor 21 authenticates a member through biometric information verification (YES in Act 95), it causes the touch panel 25 of the cash register 20 to display a permission notice (Act 96). The processor 21 executes settlement processing according to the specified settlement method (Act 98).

[0134] When the settlement processing is completed, the processor 21 transmits a checkout notice including the transaction details data of one transaction for which the settlement processing has been completed, and the data of the member ID, the date and time when the settlement processing was completed, and the transaction ID to the store server 10 (Act 99). As shown in FIG. 7, the transaction details data includes data on the number of items (purchase points) for which product information has been registered as the purchase target.

[0135] When the processor 11 of the store server 10 receives a checkout notice (YES in FIG. 17, Act 69), it stores the transaction details data (including the purchase points data) received from the cash register 20 in the auxiliary storage device 13. Also, the processor 11 terminates the tracking for the customer corresponding to the member ID received from the cash register 20 (Act 70). The store server 10 executes the processes of Acts 71 - 73 shown in FIG. 17 in the same manner as the processes of Acts 15 - 17 shown in FIG. 11. In Act 72, a matching process is executed using the purchase points (number of items) indicated by the transaction details data received from the cash register 20. Hereinafter, the description of Acts 71 - 73 shown in FIG. 17 is omitted.

[0136] In this way, in the store system 1 of the present embodiment, even when the cash register 20 executes the registration of product information without using the electronic device 40, by performing a matching process on the number of times the predetermined operations for purchasing a product have been performed and the number of products actually registered as the purchase target, it is possible to easily confirm whether the operations by the customer have been correctly executed.

[0137] In the store system 1 according to this embodiment, it is not necessary to arrange dedicated store clerks for registering product information (product scanning) and accounting checks, and personnel reduction can be achieved. In addition, it is possible to check the actions of customers without imposing any burden on them and suppress any improper actions by customers. Further, in the store system 1 according to this embodiment, analysis of customer actions and product registration are performed using images captured by a camera. That is, it is possible to easily analyze customer actions without requiring many sensors. For this reason, it is possible to suppress the introduction cost of the system.

[0138] Note that the transfer of the store server 10 is generally performed with the program stored in a recording medium such as a ROM or HDD. However, it is not limited to this, and the transfer may be performed with the program not stored in the recording medium. And in this case, the program transferred separately from this store server 10 is written to the writable storage device provided in the store server 10 according to the operation of a user or the like. The transfer of the program can be performed by recording it on a removable recording medium or by communication via a network. The recording medium may be in any form as long as it can store the program and can be read by the device, such as a CD-ROM or a memory card.

[0139] In addition, although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0140] 1…Store system, 10…Store server, 11, 21, 31, 41…Processor, 12, 22, 32, 42…Main memory, 13, 23, 33, 43…Auxiliary storage device, 14, 24, 46…Communication unit, 20(20-1, …, 20-K)…Accounting machine, 25, 35…Touch panel, 26…Printer, 27…Reader / writer, 30(30-1, …, 30-M)…Information terminal, 34…Wireless unit, 36…Camera, 37…Speaker, 40(40-1, …, 40-N)…Electronic device, 50…Access point, 60…Network.

Claims

1. A video receiving means for receiving a video captured by a camera; a receiving means for receiving a terminal code unique to an electronic device operated by a customer from said electronic device; a counting means for identifying a customer corresponding to the terminal code in the video, counting the number of times the customer has performed a predetermined action to purchase a product, and storing the number of times in association with identification data unique to the customer; A verification means for verifying the number of products whose product information is registered as purchase targets for the customer with the number of times; and a storage means for storing collation information indicating a result of the collation in association with the identification data.

2. 2. The information processing device of claim 1, further comprising an output means for outputting a notification that authentication of the customer corresponding to the identification data is not permitted if, at the time of receiving the identification data, the matching information indicating a matching result that the number of products registered in the product information does not match the number of times is stored in correspondence with the identification data.

3. an image storage means for storing the image of the customer in association with transaction identification data corresponding to the identification data; 3. The information processing device according to claim 1, further comprising: a change means for changing the transaction identification data and the matching information corresponding to the identification data after the image corresponding to the transaction identification data is displayed.

4. A camera to photograph the customers, An electronic device that registers product information of products to be purchased by a customer; a receiving means for receiving a terminal code unique to the electronic device operated by the customer from the electronic device; a counting means for identifying a customer corresponding to the terminal code in the image captured by the camera, counting the number of times the customer has performed a predetermined action to purchase a product, and storing the number of times in association with identification data unique to the customer; a verification means for verifying the number of products whose product information is registered by the electronic device with the number of times; A store system having a storage means for storing collation information indicating a result of the collation in association with the identification data.

5. Computer, A video receiving means for receiving a video captured by a camera; a receiving means for receiving a terminal code unique to an electronic device operated by a customer from said electronic device; a counting means for identifying a customer corresponding to the terminal code in the video, counting the number of times the customer has performed a predetermined action to purchase a product, and storing the number of times in association with identification data unique to the customer; A verification means for verifying the number of products whose product information is registered as purchase targets for the customer with the number of times; A program for causing the computer to function as a storage means for storing collation information indicating the result of the collation in association with the identification data.

Images