Information processing system, information processing device, method, and program

Abstract

To obtain action history of a customer generated before member ID information and biometric information are associated with each other.SOLUTION: An information processing system 1 includes a first acquisition unit 101, an identification information registration unit 102, an action information registration unit 103, and an update unit 104. The first acquisition unit 101 acquires biometric information of a customer. The identification information registration unit 102 registers the acquired biometric information in association with identification information of the customer. The action information registration unit 103 registers action information of the customer in association with the identification information. When acquiring member identification information of the customer, the update unit 104 updates the identification information associated with the action information and the biometric information of the customer, with the acquired member identification information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system and the like. [Background technology]

[0002] There is a technology for registering biometric information at a store or online site. In the technology disclosed in Patent Document 1, if a newly acquired face image matches an already registered face image, the new face image is additionally registered. In addition, Patent Document 2 discloses that face images are registered in a customer list in the store, and furthermore, the face images are associated with purchase histories. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2017 / 146160 [Patent Document 2] JP 2002-41770 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, if identification information such as a member ID (identifier) ​​and biometric information have not been registered, there is a problem in that behavioral history such as purchase history and store visit history cannot be registered.

[0005] One example of an object of the present invention is to provide an information processing system or the like that can grasp a customer's behavior history before member ID information and biometric information are associated with each other. [Means for solving the problem]

[0006] An information processing system in one embodiment of the present invention comprises a first acquisition means for acquiring biometric information of a customer, an identification information registration means for registering the acquired biometric information in association with identification information of the customer, a behavioral information registration means for registering behavioral information of the customer in association with the identification information, and an update means for updating, when member identification information of the customer is acquired, the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information.

[0007] An information processing device in one embodiment of the present invention comprises a first acquisition means for acquiring biometric information of a customer, an identification information registration means for registering the acquired biometric information in association with identification information of the customer, a behavioral information registration means for registering behavioral information of the customer in association with the identification information, and an update means for updating, when member identification information of the customer is acquired, the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information.

[0008] A method in one aspect of the present invention acquires biometric information of a customer, associates the acquired biometric information with identification information of the customer and registers it, registers behavioral information of the customer in association with the identification information, and when member identification information of the customer is acquired, updates the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information.

[0009] A program in one aspect of the present invention causes a computer to execute a process of acquiring biometric information of a customer, registering the acquired biometric information in association with identification information of the customer, registering behavioral information of the customer in association with the identification information, and, when member identification information of the customer is acquired, updating the identification information associated with the behavioral information and biometric information of the customer to the acquired member identification information. Effect of the Invention

[0010] According to the present invention, it is possible to ascertain the behavioral history of a customer before member ID information and biometric information are associated with each other. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram of an example of a configuration of an information processing system according to a first embodiment. [Diagram 2] FIG. 2 is a flowchart of an operation example of the information processing system according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram of a configuration example of an information processing system according to the second embodiment. As shown in FIG. [Figure 4] FIG. 4 is an explanatory diagram showing an example of the member DB. [Diagram 5] FIG. 5 is an explanatory diagram illustrating an example of a face DB. [Figure 6] FIG. 6 is an explanatory diagram showing an example in which face feature amounts are registered in a face DB and behavior information is registered in the face DB. [Figure 7] FIG. 7 is an explanatory diagram showing an example in which face feature amounts have already been registered in a face DB, and behavior information is registered in the face DB. [Figure 8] FIG. 8 is an explanatory diagram showing an example of updating. [Figure 9] FIG. 9 is a flowchart of an operation example of the information processing system according to the second embodiment. [Figure 10] FIG. 10 is a block diagram of a configuration example of an information processing system according to the third embodiment. As shown in FIG. [Figure 11] FIG. 11 is an explanatory diagram showing an example of updating after confirming a face image. [Figure 12] FIG. 12 is a flowchart of an operation example of the information processing system according to the third embodiment. [Figure 13] FIG. 13 is a block diagram of an example of a configuration of an information processing system according to the fourth embodiment. As shown in FIG. [Figure 14] FIG. 14 is an explanatory diagram showing an example of accepting a request for registration or non-registration. [Figure 15] FIG. 15 is a flowchart of an operation example of the information processing system according to the fourth embodiment. [Figure 16]FIG. 16 is an explanatory diagram illustrating an example of a hardware configuration of an information processing system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an information processing system, an information processing device, an information processing method, and a program according to embodiments of the present invention will be described in detail with reference to the drawings. The disclosed technology is not limited to the embodiments.

[0013] (Embodiment 1) FIG. 1 is a block diagram showing a configuration example of an information processing system according to a first embodiment. The information processing system 1 manages the member identification information of a customer and the behavior history of the customer who has come to a predetermined place. Here, it is assumed that the customer has registered as a member in advance and has a member card or a member code, but the biometric information has not yet been linked to the member ID information. The information processing system 1 has a first acquisition unit 101, an identification information registration unit 102, a behavior information registration unit 103, and an update unit 104. Each unit may be realized by the same device (for example, an information processing device). Alternatively, each unit may be realized by a plurality of different devices. For example, the first acquisition unit 101, the identification information registration unit 102, and the behavior information registration unit 103 may be the same device, and the update unit 104 may be a different device.

[0014] The first acquisition unit 101 acquires biometric information of a customer. More specifically, the first acquisition unit 101 acquires biometric information of a customer who has come to a predetermined place, for example. The predetermined place is not particularly limited. The predetermined place is a place where people gather, such as a store, an event space, or a station. The store may be a single store, or a commercial facility where multiple stores are gathered. In the case of a store, the biometric information may be acquired near the entrance when the customer comes to the store. For example, in the case of an event space, the biometric information may be acquired when the customer enters the event venue. For example, in the case of a station, the biometric information may be acquired when the customer leaves the train onto the station platform or arrives at the station building.

[0015] Moreover, the biometric information is not particularly limited. Examples of the biometric information include facial feature amount information, facial image data, fingerprint or palm print information, iris information, hand or finger vein information, ear information, voice information, or a combination of these. The facial feature amount is hereinafter referred to as a facial feature amount. Here, the facial feature amount information will be described as an example. The format of the facial feature amount information is not particularly limited. In the case of facial feature amount information, unlike facial image data, there is an advantage that it may not correspond to personal information. The first acquisition unit 101 acquires facial feature amounts obtained from an image of a customer. The facial feature amount is obtained, for example, by extracting facial feature points and the like from an image of a face. Note that the technique for extracting the facial feature amount may be an existing technique. Therefore, the technique for extracting the facial feature amount is not particularly limited. The first acquisition unit 101 may have a function for extracting facial feature amounts from an image of a face. In addition, when another device has a function for extracting facial feature amounts from an image of a face, the first acquisition unit 101 may acquire facial feature amount information from the other device via a communication network or the like.

[0016] The identification information registration unit 102 registers the acquired biometric information and the customer's identification information in association with each other. The customer's identification information here is, for example, customer ID information. For example, the customer ID information at the time of registration is different from each member's identification information (member ID information) and is information that is assigned to all customers whose biometric information has been acquired. For example, if the member ID is a number starting with A (e.g., "A00001"), the customer ID should be a number starting with B (e.g., "B00001"), and it is better to separate the member ID from the newly assigned customer ID. Specifically, the identification information registration unit 102 registers the customer's behavior information in the store in association with the customer ID information associated with the biometric information.

[0017] The behavior information registration unit 103 registers the behavior information of the customer in association with the information of the customer ID associated with the biometric information. Examples of the behavior of the customer in a predetermined location include entering a store, leaving a store, movement in the store, and purchase history. Specifically, the behavior information of the customer may be, for example, information on the date and time when the customer came to the predetermined location. For example, if the movement line of the customer in a predetermined location can be tracked by an imaging device, the behavior information of the customer may be information obtained from a specific movement line of the customer. For example, in the case of a store, the behavior information may be information on which product the customer picked up. Also, the behavior information may be information on which product the customer looked at. The technology for tracking the movement line of the customer may be an existing technology. Therefore, the technology for tracking the movement line of the customer is not particularly limited. For example, a system for tracking the movement line of the customer may be linked to the information processing system 1. For example, the movement line of the customer is identified by constantly capturing an image of the customer in a predetermined location by an imaging device.

[0018] When the information on the member ID of the customer is acquired, the update unit 104 updates the information on the customer ID associated with the behavioral information and the biometric information to the acquired information on the member ID. The information on the member ID is identification information associated with the member information. The member information is information including at least one of the customer's name, sex, age, and contact information. For example, in the case of a store, the information on the member ID may be acquired at the time of payment. In addition, in the case of a station, the information on the member ID may be acquired when entering or leaving the ticket gate. In the case of an event space, the information on the member ID may be acquired when receiving a predetermined member service. Specifically, when tracking the customer's movement line, the update unit 104 updates the information on the customer ID associated with the characteristic amount of the customer to the acquired information on the member ID. For example, when tracking the customer's movement line, the update unit 104 updates the information on the customer ID associated with the movement line having the same time and position at which the information on the member ID was acquired to the member ID information. Here, the position being the same includes being approximately the same. For example, the position of the movement line of a person who is in the vicinity (within a specified range) of a device that reads member ID information is assumed to be the same as the position where the member ID information was acquired. If the customer movement line is not tracked, the first acquisition unit 101 may acquire the customer's features again when the customer's member ID information is acquired. Then, the update unit 104 updates the customer ID information associated with the same feature as the newly acquired customer feature to the acquired member ID information.

[0019] Fig. 2 is a flowchart showing an example of an operation of the information processing system 1 according to the first embodiment. Here, for example, the processing results of each step by the information processing system 1 in Fig. 2 are stored in a storage unit. First, the first acquisition unit 101 acquires biometric information (step S101). Next, the identification information registration unit 102 associates the biometric information with information of a customer ID and registers the same (step S102). Then, the behavioral information registration unit 103 associates the behavioral information with information of the customer ID and registers the same (step S103).

[0020] Next, the update unit 104 judges whether or not the member ID information has been acquired (step S104). In step S104, the update unit 104 judges whether or not the member ID information has been acquired by a reading device or the like. If the member ID information has been acquired (step S104: Yes), the update unit 104 updates the customer ID information associated with the biometric information to the member ID information (step S105). After step S105, the information processing system 1 ends the operation of the flow. Also, if the member ID information has not been acquired (step S104: No), the information processing system 1 ends the operation of the flow. For example, the update unit 104 may determine that the member ID information has not been acquired when it is detected that the customer has left a predetermined location. Alternatively, in the case of No in step S104, if the member ID information has not been acquired even after a certain period of time has elapsed, the information processing system 1 may end the operation of the flow.

[0021] In this way, if the answer is No in step S104, steps S101 to S103 are repeated. Specifically, for example, a certain customer visited the store on May 10. However, the customer did not purchase any products at the store. In this case, the information processing system 1 ends the operation of the flow as No in step S104. Meanwhile, the same customer visited the store again on May 11. And the customer presented the membership code when purchasing a product at the store on that day. In this case, the information processing system 1 starts the process again from step S101 on May 11 and the answer is Yes in step S104, so that the information processing system 1 updates the customer ID information to the membership ID information in step S105. Specifically, on May 10, the facial feature amount of the customer and the customer ID information (for example, "B00001") are associated with the behavioral information that the customer visited the store on May 10. Then, on May 11th, the customer's facial features and customer ID information (e.g., "B00002") are associated with behavioral information indicating that the customer visited the store on May 11th. Then, at the time of payment on May 11th, the information processing system 1 acquires the member ID information (e.g., "A00001"). Then, the information processing system 1 updates the customer ID information ("B00001" and "B00002") associated with the customer's facial features to the member ID information ("A00001").

[0022] Next, the effects of the first embodiment will be described. For example, the information processing system 1 registers behavioral information in association with customer ID information and biometric information. Then, when the information processing system 1 acquires member ID information, it updates the customer ID information to the member ID information. This enables the information processing system 1 to link, for example, the behavioral history of a customer at a specific location before linking the biometric information to the member ID information to the member ID information. As a result, the information processing system 1 can assist in collecting more behavioral history of customers.

[0023] Also, for example, a customer may take an image of his / her face. Then, the customer may upload face image information as online member information. In such a case, even if there are many customers, it is estimated that the number of customers who actually upload face images is small. For example, if only a few customers upload image data for 100 customers, it is difficult to analyze the behavior of multiple customers. On the other hand, in the information processing system 1, the history of the customer's behavior in a specified place before the biometric information is linked to the member ID information can also be linked to the member ID information. Therefore, the information processing system 1 can support the analysis of the behavior of more customers. In addition, the information processing system 1 can support the analysis of the customer's behavior with higher accuracy. For example, the information processing system 1 can improve the analysis accuracy when performing an analysis similar to the conversion rate analyzed in an electronic commerce (EC (Electronic Commerce)) site on customers who visit a store. The conversion rate is, for example, the ratio of the number of customers who actually purchased to the number of customers who visited a website. On the other hand, in the case of customers who visited an actual place, for example, an analysis of the ratio of the number of customers who actually performed a specified behavior to the number of customers who visited a specified place can be performed. Alternatively, an analysis may be performed on a customer-by-customer basis to determine the ratio of the number of times a customer actually purchased items to the number of times the customer visited a store. In this way, the information processing system 1 can also aggregate the behavioral history of customers before linking the information of the member ID, thereby improving the accuracy of customer behavior analysis.

[0024] Also, as described above, facial feature values ​​have the advantage that, unlike facial image data, they may not be personal information. Compared to facial image data, facial feature values ​​are less likely to identify an individual when leaked. Therefore, the information processing system 1 uses facial feature values ​​as biometric information. This allows the information processing system 1 to reduce the risk of a customer being identified when biometric information is leaked.

[0025] (Embodiment 2) Next, the second embodiment will be described in detail with reference to the drawings. In the second embodiment, for example, an example will be described in which it is determined whether newly acquired biometric information has been registered. In the second embodiment, if newly acquired biometric information has not been registered, the customer ID information and the biometric information are newly registered in association with each other. Below, as long as the explanation of the second embodiment is not unclear, the explanation of the contents that overlap with the above explanation will be omitted. Also, in the second embodiment, an example will be described in which face features are used as biometric information.

[0026] 3 is a block diagram showing a configuration example of an information processing system according to embodiment 2. The information processing system 2 has an information processing device 20, an imaging device 22, and a reading device 23. In embodiment 2, it is newly added that the information processing system 2 has the information processing device 20, the imaging device 22, and the reading device 23. In addition, it is newly added that the information processing device 20 has each functional unit.

[0027] Each device may be the same device. For example, the information processing device 20 and the reading device 23 may be the same device. For example, the imaging device 22 and the reading device 23 may be the same device. There is no particular limitation on which combination of devices is the same device.

[0028] The imaging device 22 is a device that captures an image or a moving image of a customer. For example, the imaging device 22 may capture images of a customer at all times in order to track the customer's movement line. As shown in FIG. 3 as imaging device 22-1 to imaging device 22-n, the information processing system 2 may have a plurality of imaging devices 22. Note that n is an integer of 1 or more. There is no particular limitation on the position at which the imaging device 22 is attached.

[0029] In FIG. 3, the information processing device 20 includes, for example, a first acquisition unit 201, an identification information registration unit 202, a behavior information registration unit 203, an update unit 204, a determination unit 205, and a storage unit 206. Each functional unit may be realized by a different device. In the second embodiment, the determination unit 205 and the storage unit 206 are added to the first embodiment. The first acquisition unit 201 has the function of the first acquisition unit 101 according to the first embodiment. The identification information registration unit 202 has the function of the identification information registration unit 102 according to the first embodiment. The behavior information registration unit 203 has the function of the behavior information registration unit 103 according to the first embodiment. The update unit 204 has the function of the update unit 104 according to the first embodiment.

[0030] First, the storage unit 206 will be described. The storage unit 206 stores the processing results of each function of the information processing device 20. The storage unit 206 also stores data used in the processing of each unit. Although a detailed description will be given of an example of a hardware configuration in FIG. 16 described later, the information processing device 20 loads a program including various commands for implementing each functional unit into the storage unit 206 and executes it. Examples of the storage unit 206 include a read only memory (ROM), a random access memory (RAM), a semiconductor memory, a hard disk drive (HDD), and a solid state drive (SSD). The storage unit 206 may also be a combination of these.

[0031] In FIG. 3, the memory unit 206 stores, as memory contents, for example, a member DB 2011 and a face DB 2012. The memory contents of the memory unit 206 in FIG. 3 are an example. For example, these various pieces of information may be stored in a memory unit of another device (for example, a storage device) connected to the information processing device 20 via a communication network. The member DB 2011 and the face DB 2012 may be the same as those used on an EC site, for example. For this reason, the member DB 2011 and the face DB 2012 may be stored in a storage device that is available to both the EC site and the information processing system 2. This allows the information processing system 2 to aggregate the behavioral history of customers on both the EC site and the actual store.

[0032] FIG. 4 is an explanatory diagram showing an example of the member DB 2011. In the member DB 2011, member ID information, which is member identification information, is associated with member information for each member. In the member DB 2011 of FIG. 4, row-wise records include member ID information and member information for each member. In FIG. 4, the member information includes name information, gender information, age information, and contact information. The member information is not limited to these and may be changed in various ways. Here, for example, the member ID is represented by a number starting with "A."

[0033] FIG. 5 is an explanatory diagram showing an example of the face DB 2012. In the face DB 2012, customer ID information, which is customer identification information, facial feature information, which is biometric information, and behavior history information, which is behavior information, are associated with each other. In the member DB 2011 of FIG. 5, records in the row direction include customer ID information, facial feature information, and behavior history information. In FIG. 5, when the predetermined place is a store, various examples of the behavior history include store visits, store exits, the number of store visits by month, products picked up, products gazed at for a predetermined period, and positions where a person stood for a predetermined period. In addition, in the case of an event space, various examples of the behavior history include the time of entry, the time of exit, and stopping by or participating in each event area that the person was interested in. In addition, in the case of a station, various examples of the behavior history include the time of entry through the ticket gate, the time of exit through the ticket gate, the time of arrival at the station, the time of getting off at the station platform, and whether or not a product was purchased at a shop or the like in the station. In the example of FIG. 5, when the customer ID information is "B0002", Y is registered as the facial feature and "Visited the store on August 1st" is registered as the behavioral history.

[0034] Returning to the description of FIG. 3, each functional unit will be described. The first acquisition unit 201 acquires, for example, biometric information of a customer who has come to a predetermined location. As described above, in the second embodiment, the biometric information includes facial feature information of the customer. For this reason, the first acquisition unit 201 acquires facial feature information obtained from an image of a customer who has come to a predetermined location. As described in the first embodiment, the predetermined location is not particularly limited. For example, the predetermined location is a location where membership information can be managed for each member by a membership card and a membership code. For example, the image here is captured by the imaging device 22. For example, in the case of a store, the imaging device 22 installed near the entrance captures the face of a customer who has come to the store. The process of extracting facial feature information from a facial image is not particularly limited. For example, a plurality of facial feature points may be extracted from a facial image. As described in the first embodiment, the first acquisition unit 201 performs a process of extracting facial feature information from a facial image. As a result, the first acquisition unit 201 may acquire facial feature information by an extraction process. Alternatively, the first acquisition unit 201 may receive information on facial feature amounts from an existing device that extracts facial feature amounts.

[0035] The determination unit 205 determines whether there is customer ID information associated with the acquired facial feature amount information. Specifically, the determination unit 205 searches the face DB 2012 for facial feature amount information that matches the acquired facial feature amount information.

[0036] If the customer ID information associated with the acquired facial feature information is not registered in the face DB 2012, the identification information registration unit 202 registers the acquired facial feature information and the customer ID information in association with each other. The customer ID here is, for example, an ID different from the member ID and the customer ID already registered in the face DB 2012.

[0037] Furthermore, when there is customer ID information associated with the acquired facial feature amount information, the identification information registration unit 202 does not newly register the facial feature amount information. In this case, the identification information registration unit 202 may update the registered facial feature amount information to the newly acquired facial feature amount information. Also, when the biometric information is information other than the facial feature amount information, the identification information registration unit 202 registers the acquired facial feature amount information and the customer ID information in association with each other according to the determination result of the determination unit 205.

[0038] Then, the behavior information registration unit 203 registers the behavior information of the customer at a predetermined location in association with the information of the customer ID. As described in the first embodiment, for example, when the imaging device 22 is attached to the entrance of the store, the behavior information of the customer may be, for example, information indicating whether the customer has passed through the entrance gate. The behavior information of the customer may be, for example, information indicating what date and time the customer came, or information indicating what time the customer came and left. For example, the behavior information registration unit 203 may register the behavior information at the timing when the occurrence of the customer's behavior is detected. Specifically, the behavior information registration unit 203 may register the behavior information at the timing when the facial feature amount of the customer is acquired. Also, for example, when the customer's movement line can be tracked by constantly photographing the customer with the imaging device 22, the behavior information of the customer may be information obtained from the specific movement line of the customer. The behavior information is information registered in the behavior history as described above in the face DB 2012 in FIG. 5.

[0039] When the member ID information of the customer is acquired, the update unit 204 updates the customer ID information associated with the biometric information to the acquired member ID information. For example, in the case of a store, the member ID information may be acquired at the time of payment. In addition, in the case of a station, the member ID information may be acquired when entering or leaving the ticket gate. In the case of an event space, the member ID information may be acquired when receiving a predetermined member service. Specifically, when the customer's movement line is tracked, the update unit 204 updates the customer ID information associated with the characteristic amount of the customer to the acquired member ID information. When the customer's movement line is not tracked, the first acquisition unit 201 may acquire the characteristic amount of the customer again when the member ID information of the customer is acquired. Then, the update unit 204 updates the customer ID information associated with the same characteristic amount as the newly acquired characteristic amount of the customer to the acquired member ID information.

[0040] More specifically, when the first acquisition unit 201 acquires the member ID information of the customer, the first acquisition unit 201 may acquire the facial feature amount of the customer again. Then, the update unit 204 searches the face DB 2012 for facial feature amount information that matches the acquired facial feature amount information. The update unit 204 updates the customer ID information associated with the detected facial feature amount information to the member ID information.

[0041] FIG. 6 is an explanatory diagram showing an example in which a facial feature amount is registered in the face DB 2012 and behavior information is registered in the face DB 2012. In FIG. 6, the first acquisition unit 201 acquires, for example, information on a facial feature amount X obtained from a face image. The determination unit 205 determines whether or not information on the facial feature amount X is registered in the face DB 2012. In FIG. 6, since information on the facial feature amount X is not registered in the face DB 2012, the identification information registration unit 202 registers customer ID information "B00001" and information on the facial feature amount X in the face DB 2012. The customer ID information "B00001" and information on the facial feature amount X are associated with each other. That is, in the face DB 2012, the same record includes customer ID information "B00001" and information on the facial feature amount X. In FIG. 6, information on behavior history is empty in the face DB 2012. Then, the behavior information registration unit 203 registers the behavior information of the customer at the predetermined location in association with the customer ID information "B00001." In FIG. 6, new information "Visited on October 1, 2020" is registered as behavior history information.

[0042] FIG. 7 is an explanatory diagram showing an example in which information on facial features has already been registered in the face DB 2012, and behavioral information is registered in the face DB 2012. In FIG. 7, the first acquisition unit 201 acquires, for example, information on facial features X obtained from a face image 2013 of a customer. The determination unit 205 determines whether or not information on the facial features X is registered in the face DB 2012. In FIG. 7, information on the facial features X is registered in the face DB 2012. For this reason, the identification information registration unit 202 does not newly register information on the facial features X in the face DB 2012. In the face DB 2012, information "visited the store on October 1, 2020" is registered as information on the behavioral history. That is, this customer has already visited the store on October 1, 2020. Then, the behavioral information registration unit 203 registers behavioral information of the customer at a predetermined location in association with the information on the customer ID "B00001". 6, new information such as "Visited the store on October 5, 2020" and "Picked up product A on October 5, 2020" is registered as information on the behavioral history. As a result, according to the face DB 2012 after the behavioral information registration in FIG. 7, the customer visited the store on October 1, 2020 and October 5, 2020. Furthermore, according to the face DB 2012, it is known that the customer became interested in product A when he visited the store on October 5, 2020.

[0043] Returning to the description of FIG. 3, when the information of the member ID of the customer is acquired, the update unit 204 updates the information of the customer ID associated with the information of the facial feature amount to the information of the acquired member ID. For example, the reading device 23 reads the member card or the member code presented by the customer. In this way, the information of the member ID of the customer is acquired. If the predetermined place is a store or an event space, the reading device 23 reads the member card or the member code. For example, the timing of reading can be when making a payment or when receiving a predetermined service. In this way, the information of the member ID of the customer corresponding to the member card or the member code may be acquired. Also, if the predetermined place is a station, the customer holds a non-contact type IC (Integrated Circuit) card over the reading device 23. For example, the timing of holding the IC card over the reading device 23 can be when the customer enters a ticket gate, when exiting a ticket gate, or when purchasing a product from a ticket machine, a vending machine, or the like. Then, the reading device 23 reads the non-contact type IC card. In this way, the information of the member ID registered in the IC card may be acquired. In FIG. 3, a plurality of reading devices 23 may be provided.

[0044] FIG. 8 is an explanatory diagram showing an example of an update. For example, member ID information "A00001" is acquired. Member ID information "A00001" is registered in member DB 2011. Therefore, update unit 204 updates customer ID information "B00001" associated with facial feature amount X information to member ID information "A00001". As a result, member DB 2011 and face DB 2012 are associated. That is, facial feature amount X and member information are associated via the member ID. In FIG. 8, for example, the name of a customer having facial feature amount X can be specified as "●●". As a result, according to the updated face DB 2012 and member DB 2011 in FIG. 8, a 30-year-old female customer named "●●" visited the store on October 1, 2020 and October 5, 2020. Furthermore, according to the face DB 2012 and the member DB 2011, it is known that this female customer was interested in product A when she visited the store on October 5, 2020. In this way, it becomes possible to know the behavior of the customer before the member ID information “A00001” was registered in the face DB 2012.

[0045] FIG. 9 is a flowchart showing an example of an operation of the information processing system 2 according to the second embodiment. Here, for example, the processing results of each step by the information processing system 2 in FIG. 9 are stored in the memory unit 206 or a storage device connected via a communication network. The first acquisition unit 201 acquires a facial feature amount obtained from a captured face image (step S201). Then, the determination unit 205 determines whether the facial feature amount has been registered in the face DB 2012 (step S202). If the facial feature amount has been registered (step S202: Yes), the determination unit 205 proceeds to step S204. On the other hand, if the facial feature amount has not been registered (step S202: No), the identification information registration unit 202 registers the customer ID information and the facial feature amount information in association with each other (step S203).

[0046] Next, after step S203 or if the answer is Yes in step S202, the behavioral information registration unit 203 registers the behavioral information of the customer in association with the information of the customer ID (step S204). Next, the update unit 204 judges whether or not the member ID information has been acquired (step S205). For example, the reading device 23 reads a member card or a member code. As a result, the update unit 204 judges that the member ID information has been acquired (step S205: Yes). If the member ID information has been acquired (step S205: Yes), the update unit 204 updates the customer ID information to the member ID information (step S206). After step S206, the information processing system 2 ends the operation of the flow. Also, if the member ID information has not been acquired (step S205: No), the information processing system 2 ends the operation of the flow.

[0047] As described in the first embodiment, if the answer is No in step S205, steps S201 to S204 are repeated. Specifically, for example, if the customer leaves the store without acquiring member ID information, the flow operation ends. If the customer re-enters the store, steps S201 to S204 are performed again. Also, for example, step S204 may be repeated when tracking the customer's movement path. For example, if it is possible to track the customer's movement path within the store, not just when the customer visits the store, new customer behavior information is registered every time the customer takes some action within the store.

[0048] The second embodiment is not limited to the above-mentioned examples, and various modifications are possible. Modifications of the second embodiment will be described.

[0049] <First Modification of the Second Embodiment> When there is facial feature information that matches the acquired facial feature information, the determination unit 205 may further determine whether or not member ID information is registered as customer ID information. For example, when member ID information is not registered as customer ID information, the update unit 204 performs an update process when member ID information is acquired. On the other hand, when member ID information is registered as customer ID information, the behavior information registration unit 203 registers the behavior information of the customer in association with member ID information, and the update unit 204 does not need to perform an update process. This allows the information processing system 2 to suppress multiple updates with the same update content.

[0050] Here, the method of determining whether or not the member ID information is registered as the customer ID information is not particularly limited. For example, the determination unit 205 may search the member DB 2011 for member ID information that matches the customer ID information associated with the facial feature amount information. When the member DB 2011 contains member ID information that matches the customer ID information, the determination unit 205 determines that the member ID information has already been registered as the customer ID information. On the other hand, when the member DB 2011 does not contain member ID information that matches the customer ID information, the determination unit 205 determines that the member ID information has already been registered as the customer ID information. Also, for example, assuming that a number starting with "A" is the member ID, when a number starting with "A" is set as the customer ID information, the determination unit 205 determines that the member ID information has already been registered as the customer ID information in the face DB 2012. For example, when a number starting with "B" is set as the customer ID information, the determination unit 205 determines that the member ID information has not already been registered as the customer ID information in the face DB 2012. Then, the update unit 204 may be configured not to perform an update process even if the member ID information is acquired.

[0051] <Modification 2 of the second embodiment> A customer may have multiple membership cards or multiple membership codes. In such a case, the acquired member ID information is different from the member ID information registered in the face DB 2012. In this case, the update unit 204 may additionally register the acquired member ID information as face ID information when the member ID information registered as face ID information in the face DB 2012 is different from the acquired member ID information. For example, assume that the member ID information "A00001" is acquired, but the member ID information "A00002" is associated with the face feature information in the face DB 2012. In this case, the update unit 204 associates the member ID information "A00002" and the member ID information "A00001" with the face feature information. As a result, in the face DB 2012, the face feature information is linked to the information of the multiple member IDs as the customer ID information. Therefore, the information processing system 2 can link all the behavior history information with all the member information even when the customer has multiple membership cards or membership codes.

[0052] <Modification 3 of the Second Embodiment> In the case where a customer has multiple membership cards or membership codes as in the second modification, in the third modification, the customer may be asked whether he / she has multiple membership cards or membership codes. Although not shown, for example, when the member ID information registered in the face DB 2012 as face ID information is different from the acquired member ID information, the information processing system 2 may ask the customer whether he / she has multiple membership cards, membership codes, etc., using an output device or the like. Furthermore, when the member ID information registered in the face DB 2012 as face ID information is different from the acquired member ID information, the information processing system 2 may ask the customer which member ID (member card or member code) to use. Then, the update unit 204 may register the member ID information currently used by the customer, out of the registered member ID information and the acquired member ID information, in the face DB 2012 as face ID information. The member ID information currently used by the customer may be input, for example, via a reception unit (third reception unit) not shown in the second embodiment.

[0053] This concludes the description of each modified example of the second embodiment. Next, the effects of the second embodiment will be described. For example, when newly acquired biometric information has not been registered, the information processing system 2 registers new customer ID information and biometric information in association with each other. On the other hand, for example, when newly acquired biometric information has been registered, the information processing system 2 does not register new customer ID information and biometric information in association with each other. This allows the information processing system 2 to aggregate behavioral information of the same customer. Therefore, the information processing system 2 can more easily grasp the behavioral history of a customer before it was linked to membership information.

[0054] (Embodiment 3) Next, the third embodiment will be described in detail with reference to the drawings. In the third embodiment, an example will be described in which, when member ID information is acquired, a face image is displayed to confirm whether the face image is a customer or not, and then the customer ID information is updated to the member ID information. Below, as long as the explanation of the third embodiment is not unclear, the explanation of the contents that overlap with the above explanation will be omitted. Also, in the third embodiment, an example will be described in which face features are used as biometric information.

[0055] 10 is a block diagram showing a configuration example of an information processing system according to the third embodiment. The information processing system 3 includes, as basic components, the units of the information processing systems 1 and 2 according to the first and second embodiments. The information processing system 3 includes an information processing device 30, a terminal device 31, an imaging device 32, and a reading device 33. The information processing system 3 further includes the terminal device 31 from the second embodiment.

[0056] Each device may be the same device. For example, the information processing device 30 and the terminal device 31 may be the same device. For example, the terminal device 31 and the reading device 33 may be the same device. For example, the imaging device 32 and the reading device 33 may be the same device. There is no particular limitation on which combination of devices is the same device.

[0057] The information processing device 30 has the functions of the information processing device 20 according to the second embodiment. The imaging device 32 has the functions of the imaging device 22 according to the second embodiment. The reading device 33 has the functions of the reading device 23 according to the second embodiment.

[0058] The information processing device 30 includes a first acquisition unit 301, a determination unit 305, an identification information registration unit 302, a behavior information registration unit 303, an update unit 304, a second acquisition unit 307, a display control unit 308, and a first reception unit 309. The information processing device 30 includes the second acquisition unit 307, the display control unit 308, and the first reception unit 309 in addition to the information processing device 30 according to the second embodiment. The first acquisition unit 301 includes the functions of the first acquisition units 201 and 201 according to the first and second embodiments. The determination unit 305 includes the function of the determination unit 305 according to the second embodiment. The identification information registration unit 302 includes the functions of the identification information registration units 102 and 202 according to the first and second embodiments. The behavior information registration unit 303 includes the functions of the behavior information registration units 103 and 203 according to the first and second embodiments. The update unit 304 includes the functions of the update units 104 and 204 according to the second embodiment. The storage unit 306 has the function of the storage unit 206 according to the embodiment 2. The member DB 3011 and the face DB 3012 stored in the storage unit 306 may be the same as the member DB 2011 and the face DB 2012 according to the embodiment 2. Note that the information processing device 30 according to the embodiment 3 does not need to have, for example, the determination unit 305.

[0059] The terminal device 31 has, for example, an output device. Examples of the output device include a display device (display unit) that displays images and a printer that prints images. The terminal device 31 also has, for example, an input device that accepts input of operations by a user. Examples of the input device include a keyboard, a mouse, and a touch panel display. Note that the information processing device 30 may have both or either of the output device and the input device. Also, the information processing device 30 may have both or either of the output device and the input device in a device or the like different from the information processing device 30 and the terminal device 31.

[0060] The first acquiring unit 301, the determining unit 305, the identification information registering unit 302, and the behavior information registering unit 303 are unchanged from those in the second embodiment, and detailed description thereof will be omitted.

[0061] When the member ID information is acquired, the second acquisition unit 307 acquires face image information of the customer. The data format of the face image information is not particularly limited. Here, the timing at which the face image is captured by the imaging device 32 is not particularly limited. For example, the face image may be captured by the imaging device 32 after the member ID information is acquired. Or, for example, the face image may be captured by the imaging device 32 in order to obtain face feature amounts. That is, the second acquisition unit 307 may acquire a face image used to extract face feature amounts.

[0062] The display control unit 308, for example, causes the face image of the customer to be displayed on the display unit. The display unit of the terminal device 31 is taken as an example of the display unit. As described above, the display unit may be a display unit provided in the information processing device 30 or another device. Furthermore, the display control unit 308 is not limited to a display unit, and may be an output control unit that outputs the face image of the customer to another output device such as a printer. Specifically, the display control unit 308 causes the face image to be displayed on the display unit of the terminal device 31, for example.

[0063] Next, the first reception unit 309 receives, for example, an input as to whether the displayed image shows the face of the customer. The first reception unit 309 receives, for example, an input as to whether the image shows the face of the customer via an input device of the terminal device 31. Specifically, the first reception unit 309 receives an input by the customer as to whether the facial image shows the face of the customer. For example, the customer checks whether the image displayed on the display unit of the terminal device 31 shows his / her own face. Then, the customer inputs to the first reception unit 309 whether the image shows the face of the customer via the input device of the terminal device 31. In this way, the first reception unit 309 receives an input by the customer as to whether the facial image shows the face of the customer via the input device of the terminal device 31. Note that the input device may be provided in the information processing device 30, another device, or the like.

[0064] Alternatively, the first reception unit 309 receives an input by a person in charge at a specified location as to whether or not the facial image shows the face of a customer. Here, in the case of a store, the person in charge may be a store clerk. In the case of an event space, the person in charge may be an event staff member. In the case of a station, the person in charge may be a station staff member. The person in charge checks whether or not the image displayed on the terminal device 31 shows the face of a customer. Then, the person in charge inputs whether or not the image shows the face of a customer using the input device of the terminal device 31. In this way, the first reception unit 309 receives an input by the person in charge as to whether or not the facial image shows the face of a customer via the input device of the terminal device 31.

[0065] If it is accepted that the facial image shows the face of the customer, the update unit 304 updates the customer ID information to the acquired member ID information. Also, for example, if it is accepted that the facial image does not show the face of the customer, the update unit 304 does not update the customer ID information to the acquired member ID information.

[0066] FIG. 11 is an explanatory diagram showing an example of updating after confirmation of a face image. When the member ID information is acquired, the second acquisition unit 307 acquires information on the customer's face image 3013. In FIG. 11, the display control unit 308 displays, for example, the customer's name information "●●" corresponding to the member ID information "A00001" and the face image 3013 on the touch panel display of the terminal device 31. Also, in FIG. 11, the display control unit 308 displays buttons for accepting inputs of "Confirm" and "No" on the touch panel display of the terminal device 31. The terminal device 31 can accept inputs of whether or not to show the customer's face by using these buttons. In FIG. 11, for example, if "Confirm" is pressed, it is input that the face image 3013 shows the customer's face. If "No" is pressed, it is input that the face image 3013 does not show the customer's face.

[0067] Terminal device 31 accepts pressing of a "Confirm" or "No" button by the customer or store clerk. Then, first acceptance unit 309 accepts the contents of input to the touch panel display (input device) of terminal device 31. In FIG. 11, because "Confirm" was pressed, update unit 304 updates the customer ID information associated with facial feature amount X to the acquired member ID information "A00001". Although not shown, if "No" was pressed, update unit 304 does not update the customer ID information associated with facial feature amount X to the acquired member ID information "A00001".

[0068] Fig. 12 is a flowchart showing an example of an operation of the information processing system 3 according to the third embodiment. In Fig. 12, steps S301 to S305 are the same as steps S201 to S205 shown in Fig. 9. Therefore, explanations of steps S301 to S305 will be omitted. In addition, for example, the processing results of each step by the information processing system 3 in Fig. 12 are stored in the storage unit 306 or a storage device connected via a communication network.

[0069] If the answer is Yes in step S305, the second acquisition unit 307 acquires an image of the customer's face (step S306). Step S306 may be performed at the same timing as step S301, for example. Next, the display control unit 308 displays the face image on the display unit (step S307). Then, the first reception unit 309 receives an input of whether the face image shows the customer's face (step S308). If an input is received indicating that the face image does not show the customer's face (step S308: No), the information processing system 3 ends the operation of the flow. On the other hand, if an input is received indicating that the face image shows the customer's face (step S308: Yes), the update unit 304 updates the customer ID information to the member ID information (step S309). Then, after step S309, the information processing system 3 ends the series of operations.

[0070] Next, the effects of the third embodiment will be described. For example, when member ID information is acquired, the information processing system 3 displays a facial image. Then, the information processing system 3 checks whether the facial image is a customer. If the facial image shows the face of a customer, the information processing system 3 updates the customer ID information to member ID information. This allows the information processing system 3 to link correct facial feature information to member ID information.

[0071] Alternatively, the customer may check the facial image. This allows the information processing system 3 to link more accurate facial feature information to the member ID information. Alternatively, the facial image may be checked by a staff member such as a store clerk. This allows the information processing system 3 to prevent customer impersonation. This allows the information processing system 3 to link more accurate facial feature information to the member ID information.

[0072] (Embodiment 4) Next, the fourth embodiment will be described in detail with reference to the drawings. In the fourth embodiment, an example will be described in which customer ID information is updated to member ID information after confirming with the customer whether or not to register biometric information. Below, explanations of contents that overlap with the above explanation will be omitted to the extent that the explanation of the fourth embodiment is not unclear. Also, in the fourth embodiment, an example will be described in which face features are used as biometric information.

[0073] FIG. 13 is a block diagram showing a configuration example of an information processing system according to the fourth embodiment. The information processing system 4 includes the units of the information processing systems 1, 2, and 3 described in the first, second, and third embodiments as basic configurations. The information processing system 4 includes an information processing device 40, a terminal device 41, an imaging device 42, and a reading device 43. The information processing system 4 does not include any devices newly added from the third embodiment. The information processing device 40 has the functions of the information processing devices 20 and 30 according to the second and third embodiments. The imaging device 42 has the functions of the imaging devices 22 and 32 according to the second and third embodiments. The reading device 43 has the functions of the reading devices 23 and 33 according to the second and third embodiments. The terminal device 41 has the functions of the terminal device 31 according to the third embodiment.

[0074] Each device may be the same device. For example, the information processing device 40 and the terminal device 41 may be the same device. For example, the terminal device 41 and the reading device 43 may be the same device. For example, the imaging device 42 and the reading device 43 may be the same device. There is no particular limitation on which combination of devices is the same device.

[0075] The information processing device 40 includes a first acquisition unit 401, a determination unit 405, an identification information registration unit 402, a behavior information registration unit 403, an update unit 404, a second acquisition unit 407, a display control unit 408, a first reception unit 409, a second reception unit 410, and a storage unit 406. The information processing device 40 includes the second reception unit 410 in addition to the information processing device 40 according to the third embodiment. The first acquisition unit 401 has the functions of the first acquisition units 101, 201, and 301 according to the first, second, and third embodiments. The determination unit 405 has the functions of the determination units 205 and 305 according to the second and third embodiments. The identification information registration unit 402 has the functions of the identification information registration units 102, 202, and 302 according to the first, second, and third embodiments. The behavior information registration unit 403 has the functions of the behavior information registration units 103, 203, and 303 according to the first, second, and third embodiments. An update unit 404 has the functions of the update units 104, 204, 304 according to the first, second, and third embodiments. A first reception unit 409 has the functions of the first reception unit 309 according to the third embodiment. A memory unit 406 has the functions of the memory units 206, 306 according to the second and third embodiments. A member DB 4011 and a face DB 4012, which are the stored contents of the memory unit 406, may be the same as the member DBs 2011, 3011 and the face DBs 2012, 3012 according to the second and third embodiments.

[0076] The information processing device 40 according to the fourth embodiment may not have, for example, the determination unit 405. Moreover, the information processing device 40 according to the fourth embodiment may not have the second acquisition unit 407 and the display control unit 408.

[0077] The first acquisition unit 401, the determination unit 405, the identification information registration unit 402, the behavior information registration unit 403, the second acquisition unit 407, the display control unit 408, and the first reception unit 409 are unchanged from those in the third embodiment. Detailed description of these components will be omitted.

[0078] When the member ID information is acquired, the second reception unit 410 receives an input from the customer as to whether or not to register the biometric information. The second reception unit 410 receives an input from the customer as to whether or not to register via, for example, an input device of the terminal device 41. More specifically, for example, the second reception unit 410 receives an input from the customer. For example, biometric information may correspond to personal information such as face image data. Therefore, when the biometric information corresponds to personal information, the information processing device 40 can improve the customer's trust by having the customer decide whether or not to link the member ID information with the biometric information. This allows the information processing system 4 to leave the decision on whether or not to register to the customer. Furthermore, even when the information does not correspond to personal information such as face features, the information processing system 4 can leave the decision on whether or not to link the member ID information with the biometric information to the customer. Therefore, the information processing system 4 can improve the customer's trust.

[0079] Next, when it is accepted that the biometric information is to be registered, the update unit 404 updates the customer ID information associated with the biometric information to the acquired member ID information. On the other hand, when it is accepted that the biometric information is not to be registered, the update unit 404 does not update the customer ID information associated with the biometric information to the acquired member ID information.

[0080] Furthermore, when it is accepted that the biometric information is not to be registered, the update unit 404 may delete the association between the biometric information, the customer ID information, and the behavior history information from the face DB 4012. Note that there is no particular limitation as to whether or not to perform this deletion.

[0081] FIG. 14 is an explanatory diagram showing an example of accepting whether or not to register. The display control unit 408 may display, for example, a button on the terminal device 41 that allows input of whether or not to register the facial feature amount X. For example, the display unit of the terminal device 41 displays a face image 4013, the button, and a text on the screen. The text includes the name of the customer acquired from the member DB 4011. Specifically, the text is a text for confirming whether or not to register the facial feature amount. Also, in FIG. 14, a "Register" button and a "Do not register" button are displayed on the screen of the terminal device 41 together with the face image 4013. When the customer presses the "Register" button, the second reception unit 410 accepts the registration of the facial feature amount X. On the other hand, when the customer presses the "Do not register" button, the second reception unit 410 accepts the non-registration of the facial feature amount.

[0082] In FIG. 14, because the "Register" button has been pressed, the second reception unit 410 accepts that the facial feature amount X is to be registered. Then, the update unit 404 updates the customer ID information "B00001" associated with the facial feature amount X to the acquired member ID information "A00001". Although not shown, if the "Do not register" button is pressed, the second reception unit 410 accepts that the facial feature amount X is not to be registered. Then, the update unit 404 does not update the customer ID information "B00001" associated with the facial feature amount X to the acquired member ID information "A00001".

[0083] Fig. 15 is a flowchart showing an example of an operation of the information processing system 4 according to the fourth embodiment. In Fig. 15, a face feature amount is taken as an example of biometric information for explanation. In Fig. 15, steps S401 to S405 are the same as steps S201 to S205 shown in Fig. 9 and steps S301 to S305 shown in Fig. 12. Therefore, explanations of steps S401 to S405 are omitted. In addition, for example, the processing results of each step by the information processing system 4 in Fig. 15 are stored in the storage unit 406 or a storage device connected via a communication network.

[0084] If the answer is Yes in step S405, the second reception unit 410 receives an input as to whether or not to register the facial feature (step S406). If it is received that the facial feature will not be registered (step S406: No), the information processing system 4 ends the operation of the flow. If it is received that the facial feature will be registered (step S406: Yes), the update unit 404 updates the customer ID information to the member ID information (step S407). After step S407, the information processing system 4 ends the operation of the flow.

[0085] Although not shown, the third embodiment and the fourth embodiment may be combined. For example, the information processing system 4 may perform step S406 after the Yes in step S308 shown in FIG. 12. Alternatively, for example, the information processing system 4 may perform step S406 immediately before step S306 or immediately before step S308 shown in FIG. 12. For example, when it is accepted that registration will be performed (step S406: Yes) and when it is accepted that the face image shows the face of the customer (step S308: Yes), the update unit 404 updates the customer ID information to the member ID information. On the other hand, when it is accepted that registration will not be performed (step S406: No) or when it is accepted that the face image does not show the face of the customer (step S308: No), the update unit 404 does not update the customer ID information to the member ID information.

[0086] <Modification of the fourth embodiment> In the fourth embodiment, as a modified example, a mechanism may be provided whereby a customer can receive a benefit when he or she selects to register biometric information. The benefit is not particularly limited to, for example, points given to members, issuance of discount coupons, invitation to events, and the like. For example, in the fourth embodiment, when accepting whether or not to register, a message indicating that a benefit will be given may be displayed on the display unit. Also, for example, in the fourth embodiment, when accepting whether or not to register, the display control unit 408 may display the company's rules on the handling of biometric information on the display unit. To realize such a function, for example, the fourth embodiment may have a granting unit that grants a benefit, not shown. Also, although not shown, in the member DB 4011, for example, point information may be associated with member information.

[0087] Next, the effects of the fourth embodiment will be described. For example, when the information processing system 4 acquires member ID information, it confirms with the customer whether or not to register the biometric information. This allows the information processing system 4 to leave it up to the customer to decide whether or not to register the biometric information. Therefore, the information processing system 4 can improve the customer's trust in the handling of biometric information.

[0088] This concludes the description of each embodiment. Next, the hardware configuration of the information processing system will be described. FIG. 16 is an explanatory diagram showing an example of the hardware configuration of the information processing system. The information processing system 5 is an example in which the information processing systems 1, 2, 3, and 4 according to the first, second, third, and fourth embodiments are realized by a computer. The information processing system 5 includes, for example, an information processing device 50, a terminal device, an imaging device, and a reading device.

[0089] First, a description will be given of a hardware configuration of the information processing device 50. The information processing device 50 is an example in which the information processing devices 20, 30, and 40 according to the second, third, and fourth embodiments are realized by a computer.

[0090] The information processing device 50 includes a CPU (Central Processing Unit) 501, a ROM 502, a RAM 503, a storage device 504, and a communication interface 505. Each component is connected to each other via a bus 506.

[0091] The CPU 501 controls the entire information processing device 50. For example, the CPU 501 may operate an OS (Operating System) to control the entire information processing device 50. The CPU 501 may have multiple cores.

[0092] 16, the information processing device 50 has, for example, a ROM 502, a RAM 503, and a storage device 504 as a storage unit. Examples of the storage device 504 include a semiconductor memory such as a flash memory, a HDD, and an SSD. For example, the storage device 504 stores various programs such as an OS program, an application program, and the programs according to the first, second, third, and fourth embodiments. Alternatively, the ROM 502 stores an application program. The ROM 502 may also store the programs according to the first, second, third, and fourth embodiments. The RAM 503 is used as a work area for the CPU 501.

[0093] Moreover, the CPU 501 loads a program stored in the storage device 504 or the ROM 502. Then, the CPU 501 executes each process coded in the program. The CPU 501 may download various programs via the communication network 510. The CPU 501 functions as a part or the whole of the information processing device 50. Then, the CPU 501 may execute processes or instructions in the illustrated flowchart based on the program.

[0094] The communication interface 505 is connected to a communication network 510, such as a local area network (LAN) or a wide area network (WAN), via a wireless or wired communication line. This allows the information processing device 50 to be connected to an external device and an external computer via the communication network 510. The communication interface 505 serves as an interface between the communication network 510 and the inside of the information processing device 50. The communication interface 505 controls input and output of data from the external device and the external computer.

[0095] However, the hardware configuration of the information processing device 50 shown in FIG. 16 is an example. Components other than those shown in FIG. 16 may be added, or some components may not be included. For example, the information processing device 50 may have a drive device. Then, the CPU 501 may read a program or data from a recording medium attached to the drive device to the RAM 503. Examples of the recording medium include an optical disk, a flexible disk, a magneto-optical disk, and a USB (Universal Serial Bus) memory. In addition, for example, the information processing device 50 may have input devices such as a keyboard and a mouse. The information processing device 50 may have output devices such as a printer and a display.

[0096] Next, a hardware configuration of the terminal device 51 will be described. The terminal device 51 is an example in which the terminal devices 31 and 41 described in the third and fourth embodiments are realized by a computer. The terminal device 51 has, for example, a CPU 511, a ROM 512, a RAM 513, a storage device 514, a communication interface 515, and an input / output device 517. Each component is connected to each other via a bus 516.

[0097] The CPU 511 controls the entire terminal device 51. The terminal device 51 has a ROM 512, a RAM 513, and a storage device 514 as a storage unit. Examples of the storage device 514 include a semiconductor memory such as a flash memory, a HDD, and an SSD. For example, the storage device 514 stores an OS program and an application program. Alternatively, the ROM 512 stores an application program. The RAM 513 is used as a work area for the CPU 511.

[0098] Furthermore, the CPU 511 loads a program stored in the storage device 514 or the ROM 512. Then, the CPU 511 executes each process coded in the program. The CPU 511 may download various programs via the communication network 510. The CPU 511 functions as a part or the whole of the terminal device 51. Then, the CPU 511 may execute processes or instructions in the illustrated flowchart based on the program.

[0099] The communication interface 515 is connected to a communication network 510 such as a LAN or a WAN through a wireless or wired communication line. As a result, the terminal device 51 is connected to an external device or an external computer through the communication network 510. The communication interface 515 serves as an interface between the communication network 510 and the inside of the terminal device 51. The communication interface 515 controls input and output of data from the external device or the external computer. The input / output device 517 also accepts input by a customer's operation or a staff member's operation. The input / output device 517 also outputs data. The input / output device 517 may display an image. The input / output device 517 is, for example, a touch panel display.

[0100] The hardware configuration of the terminal device 51 shown in FIG. 16 is an example. Components other than those shown in FIG. 16 may be added, or some components may not be included. For example, the terminal device 51 may have a drive device. Then, the CPU 511 may read out programs and data from a recording medium attached to the drive device to the RAM 513. Examples of the recording medium include an optical disk, a flexible disk, a magneto-optical disk, and a USB (Universal Serial Bus) memory. Furthermore, for example, the terminal device 51 may have input devices such as a keyboard and a mouse. The terminal device 51 may have an output device such as a printer.

[0101] Next, an example of the hardware configuration of the imaging device 52 will be described. The imaging device 52 corresponds to the imaging devices 22, 32, and 42 described in the second, third, and fourth embodiments. The imaging device 52 has a camera 521 and a communication interface 522. Each component is connected by a bus 523. The communication interface 522 is connected to a communication network 510 such as a LAN or a WAN through, for example, a wireless or wired communication line. The camera 521 has a function capable of capturing an image. The camera 521 may also be capable of capturing a moving image. As described above, a plurality of imaging devices 52 may also be provided. The hardware configuration of the imaging device 52 is an example. Components other than those shown in FIG. 16 may be added.

[0102] Next, an example of the hardware configuration of the reading device 53 will be described. The reading device 53 corresponds to the reading devices 23, 33, and 43 described in the second, third, and fourth embodiments. The reading device 53 includes, for example, a reader 531 and a communication interface 532. Each component is connected by a bus 533. The communication interface 532 is connected to a communication network 510 such as a LAN or a WAN through, for example, a wireless or wired communication line. The reader 531 reads a membership card or a membership code. For example, if the membership code is a barcode, the reader 531 is a barcode reader. If the membership code is a two-dimensional code, the reader 531 may be a camera. For example, if the membership card is a contactless IC card, the reader 531 is a reader for the IC card. The hardware configuration of the reading device 53 is an example. Components other than those shown in FIG. 16 may be added.

[0103] This concludes the description of the hardware configuration of the information processing system 5. There are various modified examples of the methods for realizing the information processing systems 1, 2, 3, and 4 described in the first, second, third, and fourth embodiments. For example, the information processing system 5 may be realized by any combination of different computers and programs for each component. Furthermore, multiple components provided in each device may be realized by any combination of a single computer and program.

[0104] Also, a part or all of each component of the information processing systems 1, 2, 3, 4 may be realized by a circuit for a specific application. Also, a part or all of the information processing device 50 may be realized by a general-purpose circuit including a processor such as an FPGA (Field Programmable Gate Array). Also, a part or all of the information processing systems 1, 2, 3, 4 may be realized by a combination of circuits for a specific application, a combination of general-purpose circuits, or a combination of a circuit for a specific application and a general-purpose circuit. Also, these circuits may be a single integrated circuit. Or, these circuits may be divided into multiple integrated circuits. And, the multiple integrated circuits may be configured by being connected via a bus.

[0105] Furthermore, when some or all of the components of each device are realized by a plurality of computers and circuits, the plurality of computers and circuits may be arranged in a centralized manner or in a distributed manner.

[0106] The information processing method described in each embodiment is realized by an apparatus such as the information processing device 50 executing the information processing method. Also, the information processing method is realized by a computer such as the information processing device 50 executing a program prepared in advance. The program described in each embodiment is recorded on a computer-readable recording medium such as an HDD, SSD, flexible disk, optical disk, flexible disk, magneto-optical disk, or USB memory. Then, the program is executed by being read from the recording medium by the computer. Also, the program may be distributed via the communication network 510.

[0107] The functions of each of the components of the information processing systems 1, 2, 3, and 4 in each of the embodiments described above may be realized by hardware, such as a computer information processing system 5 shown in Fig. 16. Alternatively, each component may be realized by a computer device or firmware based on program control.

[0108] Although the present invention has been described above with reference to each embodiment, the present invention is not limited to the above embodiment. The configuration and details of each invention may include embodiments to which various modifications that a person skilled in the art can grasp within the scope of the present invention are applied. The present invention may include embodiments in which the matters described in this specification are appropriately combined or replaced as necessary. For example, matters described using a specific embodiment may be applied to other embodiments to the extent that no contradiction occurs. For example, although multiple operations are described in the form of a flowchart in order, the order of description does not limit the order in which the multiple operations are performed. Therefore, when implementing each embodiment, the order of the multiple operations can be changed to the extent that the content is not impaired. [Explanation of symbols]

[0109] 1,2,3,4,5 Information Processing System 20, 30, 40, 50 Information processing equipment 22,32,42,52 Imaging device 23,33,43,53 Reading device 31, 41, 51 Terminal equipment 101,201,301,401 1st acquisition part 102, 202, 302, 402 Identification Information Registration Department 103, 203, 303, 403 Behavioral Information Registration Department 104,204,304,404 Update section 205,305,405 Judgment section 206,306,406 Storage section 307,407 2nd Acquisition Department 308,408 Display control unit 309,409 First Reception Section 410 Second Reception Section 2011,3011,4011 Member DB 2012,3012,4012 Face DB

Claims

1. A first acquisition means for acquiring biometric information of a customer; an identification information registration means for registering the acquired biometric information and the customer's identification information in association with each other; a behavior information registration means for registering the behavior information of the customer in association with the identification information; a second receiving means for receiving an input by the customer as to whether or not to register the biometric information when the member identification information of the customer is acquired; an update means for updating the identification information associated with the behavioral information and the biometric information of the customer to the acquired member identification information when it is accepted that the biometric information will be registered, and deleting the association between the behavioral information, the biometric information, and the identification information of the customer when it is accepted that the biometric information will not be registered; An information processing system comprising:

2. a determination means for determining whether there is identification information associated with the acquired biometric information, the identification information registration means, when there is no identification information associated with the biometric information, registers the biometric information and the identification information in association with each other; The information processing system according to claim 1 .

3. The biometric information includes information on facial features obtained from an image of the customer's face.

3. The information processing system according to claim 1 or 2.

4. a second acquisition means for acquiring the image when the member identification information is acquired; A display control means for displaying the acquired image; a first receiving means for receiving an input as to whether the displayed image shows the face of the customer; Equipped with the updating means updates the identification information associated with the biometric information to the acquired member identification information when it accepts that the biometric information shows the face of the customer. The information processing system according to claim 3 .

5. The first reception means receives an input by a person in charge or the customer as to whether or not to show the face of the customer.

5. The information processing system according to claim 4.

6. The biometric information of the customer is obtained from an image of the customer captured by an imaging device installed near an entrance of the store; the first acquisition means, when acquiring the member identification information of the customer at the time of payment, further acquires biometric information of the customer from an image of the customer captured by an imaging device capable of capturing an image of the customer making the payment; the updating means updates the identification information associated with the biometric information that matches the biometric information of the customer making the payment to the acquired member identification information; 6. An information processing system according to claim 1.

7. A first acquisition means for acquiring biometric information of a customer; an identification information registration means for registering the acquired biometric information and the customer's identification information in association with each other; a behavior information registration means for registering the behavior information of the customer in association with the identification information; a second receiving means for receiving an input by the customer as to whether or not to register the biometric information when the member identification information of the customer is acquired; an update means for updating the identification information associated with the behavioral information and the biometric information of the customer to the acquired member identification information when it is accepted that the biometric information will be registered, and deleting the association between the behavioral information, the biometric information, and the identification information of the customer when it is accepted that the biometric information will not be registered; An information processing device comprising:

8. A computer comprising: Acquire the customer's biometric information, registering the acquired biometric information and the customer's identification information in association with each other; registering the behavioral information of the customer in association with the identification information; When the member identification information of the customer is acquired, an input as to whether or not the customer wishes to register the biometric information is accepted; when it is accepted that the biometric information will be registered, the identification information associated with the behavioral information and the biometric information of the customer is updated to the acquired member identification information, and when it is accepted that the biometric information will not be registered, the association between the behavioral information, the biometric information, and the identification information of the customer is deleted. How to perform the action.

9. On the computer, Acquire the customer's biometric information, registering the acquired biometric information and the customer's identification information in association with each other; registering the behavioral information of the customer in association with the identification information; When the member identification information of the customer is acquired, an input as to whether or not the customer wishes to register the biometric information is accepted; when it is accepted that the biometric information will be registered, the identification information associated with the behavioral information and the biometric information of the customer is updated to the acquired member identification information, and when it is accepted that the biometric information will not be registered, the association between the behavioral information, the biometric information, and the identification information of the customer is deleted. A program that executes a process.

Images