Information processing system

Abstract

To provide an information processing system capable of improving security.SOLUTION: In the information processing system, the encryption means encrypts the information on the face image of the user transmitted from the user terminal into a homomorphic encryption at the time of registration of the user, and the storage means stores the homomorphic encryption of the information on the face image of the user in the storage unit, and encrypts each of the information on the face image of the user and the usage history information transmitted from the authentication engine into a homomorphic encryption at the time of authentication of the user. The authentication means authenticates the user by comparing the encrypted homomorphic encryption of the information related to the face image of the user with the homomorphic encryption of the information related to the face image of the user already stored in the storage unit. The recommendation means calculates recommendation information on the basis of the homomorphic encryption of the authenticated use history information of the user and the homomorphic encryption of another use history information of the user already stored in the storage unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system. [Background technology]

[0002] Conventionally, as a face authentication system that performs authentication based on face data, a face authentication system that performs identity authentication by comparing input face data with pre-registered face data has been known. For example, the face authentication system described in Patent Document 1 authenticates a target person using an image of the face part in an image captured by a camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-197426 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional face authentication systems have a risk of information leakage, such as a user's facial image. Even if the user's facial image is encrypted, there is a risk of information leakage if the image is decrypted.

[0005] The present invention has been made in consideration of these points, and aims to improve security. The present invention aims to provide an information processing system that can [Means for solving the problem]

[0006] The information processing system of the present invention comprises: a face storage unit in which face data for comparison of a user is registered; An imaging means for capturing an image of a user's face; an authentication means for extracting features from the photographed face of the user and comparing the extracted features with the comparison face data; an encryption means for converting the user's face authentication usage history information into homomorphic encryption; a history storage unit in which the encrypted usage history information is stored; a recommendation unit that calculates the usage history information while it is still encrypted and generates recommendation information for the user; The history storage unit is characterized by being a data storage provided by a cloud service provider. [Effects of the Invention]

[0007] According to the information processing system of the present invention, security can be improved. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an information processing system according to an embodiment of the present invention. [Figure 2] 4 is a flowchart showing an operation when a user is registered by the information processing system shown in FIG. [Figure 3] 4 is a flowchart showing an operation when user authentication is performed by the information processing system shown in FIG. [Figure 4] FIG. 10 is a diagram schematically illustrating an information processing system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the accompanying drawings, in which: Figures 1 to 3 show an information processing system according to the present embodiment.

[0010] The information processing system 1 according to this embodiment includes a face recognition platform 2 and a plurality of authentication engines 3 (authentication engines A to C). The face recognition platform 2 and each authentication engine 3 are connected to each other so as to be able to communicate with each other via a network such as the Internet. The face recognition platform 2 is also connected to a user terminal 5 carried by a user so as to be able to communicate with each other via a network such as the Internet. Each component of such an information processing system 1 will be described below.

[0011] Each authentication engine 3 is installed in various service facilities and includes an imaging unit 31, such as a camera, that captures a user's facial image. Information about the user's facial image captured by the imaging unit 31 is transmitted from the authentication engine 3 to the facial recognition platform 2. Furthermore, when the facial recognition platform 2 authenticates a user, various processes are performed for the authenticated user at the service facility where the authentication engine 3 is installed. For example, if the authentication engine 3 is installed at an entrance to an office building, apartment complex, or the like, the doors at the entrances or exits of the office building or apartment complex are unlocked when the facial recognition platform 2 authenticates the user. Furthermore, if the authentication engine 3 is installed in a taxi, restaurant, hotel, public transportation facility, convenience store, or the like, cashless payment is enabled when paying a fare at these service facilities when the facial recognition platform 2 authenticates the user. In this case, payment information is transmitted from the facial recognition platform 2 to a server of a financial institution or credit card company, and the payment amount is automatically debited from the user's bank account or added to the credit card statement. Furthermore, instead of payment being made by the facial recognition platform 2, cashless payment may be made by a service facility where the authentication engine 3 is installed. Specifically, when the facial recognition platform 2 authenticates the user, information that the user has been authenticated is transmitted from the facial recognition platform 2 to the service facility where the authentication engine 3 is installed, and cashless payment is then made at the service facility. For example, if the authentication engine 3 is installed inside a taxi and a face image of the user is registered in the facial recognition platform 2, the image capturing unit 31 of the authentication engine 3 captures an image of the user's face, and the face recognition platform 2 authenticates the user, allowing payment by a credit card or the like registered in the taxi app.

[0012] The facial recognition platform 2 is a computer or the like installed at a system company. Specifically, the facial recognition platform 2 includes a control unit 20, a storage unit 22, and a communication interface 24. The storage unit 22 and the communication interface 24 are each connected to the control unit 20 via a bus. The control unit 20 is a CPU (Central Processing Unit) or the like, and functions as a reception unit 201, an encryption unit 202, an authentication unit 203, a storage unit 204, a recommendation unit 205, and a transmission unit 206 by executing a program stored in the storage unit 22. The storage unit 22 stores homomorphic encryption of the user's registration information, homomorphic encryption of information related to the user's facial image, and homomorphic encryption of the user's usage history information, linked to the user's identification information. The user's registration information includes the user's name, date of birth, username, email address, telephone number of the user terminal 5, credit card information, password, etc. The information related to the user's facial image is the user's facial image data or a hash value extracted from the user's facial image data using a predetermined hash function. The user's usage history information includes identification information of the authentication engine 3, location information of the authentication engine 3, information on the date and time when the user's face image was captured by the imaging unit 31 of the authentication engine 3, and information on the service facility where the authentication engine 3 is installed (e.g., store code, type of service, etc.). The communication interface 24 transmits and receives signals to and from other devices wirelessly or via a wired connection via a network such as the Internet.

[0013] The storage unit 22 also stores, in association with each other, identification information of the authentication engine 3, information about the service facility where the authentication engine 3 is installed (e.g., store code, business hours, opening hours, type of service, etc.), and location information about the authentication engine 3. The storage unit 22 also stores a program executed by the control unit 20. The storage unit 22 may also store information about stores where the authentication engine 3 is not installed, in addition to information about stores where the authentication engine 3 is installed.

[0014] The programs executed by the control unit 20 are not limited to those stored in the storage unit 22. The control unit 20 may function as the reception means 201, the encryption means 202, the authentication means 203, the storage means 204, the recommendation means 205, and the transmission means 206 by executing a program stored in a recording medium attached to the face recognition platform 2 or a program transmitted to the face recognition platform 2 from an external device.

[0015] The accepting means 201 accepts information from various external devices such as each authentication engine 3 and the user terminal 5. Specifically, when a user registers, the accepting means 201 accepts information related to the user's facial image and the user's registration information from the user terminal 5. Furthermore, when a user is authenticated, the accepting means 201 accepts information related to the user's facial image captured by the imaging unit 31 and the user's usage history information from the authentication engine 3.

[0016] When a user is registered, the encryption means 202 encrypts information related to the user's facial image and the user's registration information transmitted from the user terminal 5, respectively, to generate homomorphic encryption. Furthermore, when a user is authenticated, the encryption means 202 encrypts information related to the user's facial image and the usage history information transmitted from each authentication engine 3, respectively, to generate homomorphic encryption. Here, homomorphic encryption refers to encryption that allows operations on encrypted data without decryption, and when two ciphertexts are given, binary operations such as addition and multiplication can be performed without plaintext or a secret key.

[0017] The authentication means 203 authenticates the user by comparing the homomorphic encryption of the encrypted user's face image with the homomorphic encryption of the user's face image stored in the storage unit 22. Here, as described above, homomorphic encryption is an encryption method that allows binary operations such as addition and multiplication to be performed when two ciphertexts are given, without the need for plaintext or a secret key. Therefore, the authentication means 203 can authenticate the user by comparing the homomorphic encryption of the encrypted user's face image with the homomorphic encryption of the user's face image stored in the storage unit 22, without the need to decrypt the homomorphic encryption of the user's face image.

[0018] The storage means 204 stores, in the storage unit 22, the homomorphic encryption of the user's facial image and the user's registration information encrypted by the encryption means 202 at the time of user registration. The storage means 204 also stores, in the storage unit 22, the homomorphic encryption of the usage history information for which the user has been authenticated.

[0019] The recommendation means 205 calculates recommendation information based on the homomorphic encryption of the usage history information for which the user has been authenticated and the homomorphic encryption of the user's usage history information stored in the storage unit 22. The recommendation information includes information useful to the user, such as service campaign information (e.g., information about discount coupons and free coupons), service availability date and time information, and special service provision information. As described above, homomorphic encryption is an encryption method that allows binary operations such as addition and multiplication to be performed when two ciphertexts are given, even without plaintext or a secret key. Therefore, the recommendation means 205 can calculate recommendation information based on the homomorphic encryption of the encrypted user's usage history information and the homomorphic encryption of the user's usage history information stored in the storage unit 22, without decrypting the homomorphic encryption of the user's usage history information. The recommendation means 205 may also calculate recommendation information by referring to the homomorphic encryption of the user's registration information stored in the storage unit 22.

[0020] The transmission means 206 transmits the recommendation information calculated by the recommendation means 205 to the user terminal 5 of the user authenticated by the authentication means 203 .

[0021] In this embodiment, when a user registers, the control unit 20 issues user identification information upon receiving the user's facial image and registration information from the user terminal 5. The storage unit 204 associates the homomorphic encryption of the user's facial image and registration information with the user's identification information and stores them in the storage unit 22. The authentication unit 203 authenticates the user by comparing the homomorphic encryption of the encrypted user's facial image with the homomorphic encryption of the user's facial image stored in the storage unit 22, and acquires the user's identification information associated with the homomorphic encryption of the authenticated user's facial image. The recommendation unit 205 calculates recommendation information based on the homomorphic encryption of the usage history information used to authenticate the user and the homomorphic encryption of the usage history information corresponding to the user's identification information stored in the storage unit 22. The transmission unit 206 transmits the calculated recommendation information to the user terminal 5 corresponding to the authenticated user's identification information.

[0022] The user terminal 5 is owned by a user. Examples of the user terminal 5 include a personal computer, a PC tablet terminal, a smartphone, etc. The user terminal 5 transmits and receives signals to and from other devices (specifically, for example, the facial recognition platform 2) wirelessly or via a wired connection via a network such as the Internet. Furthermore, the user terminal 5 is configured to allow a facial recognition app to be installed from an online store, the website of the company managing the facial recognition platform 2, or the like. Once such a facial recognition app is installed, the user can use the user terminal 5 to register facial image data and input various registration information (e.g., name, gender, age, date of birth, username, email address, telephone number of the user terminal 5, credit card information, password, etc.). The user terminal 5 also includes an imaging unit 51 for capturing an image of the user's face to obtain a facial image.

[0023] Next, a description will be given of a processing flow by the information processing system 1 of this embodiment. Fig. 2 is a flowchart showing the operation when a user is registered by the information processing system 1 shown in Fig. 1, and Fig. 3 is a flowchart showing the operation when a user is authenticated by the information processing system 1 shown in Fig. 1.

[0024] First, the operation when a user is registered by the information processing system 1 of this embodiment will be described.

[0025] First, a user who intends to use the service of the information processing system 1 of this embodiment registers a facial image using the user terminal 5. Specifically, the user first installs a facial recognition application on the user terminal 5. Then, when the user launches the facial recognition application on the user terminal 5, a user registration screen is displayed on the touch panel. On this user registration screen, the user enters registration information such as the user's name (kanji), name (kana), gender, age, date of birth, username, telephone number of the user terminal 5, email address of the user terminal 5, and password, checks the box indicating agreement to the terms of use, and then presses the registration button. Then, a facial image capture screen is displayed on the touch panel. When the user captures a facial image on this capture screen using the camera of the user terminal 5, the various registration information entered on the user registration screen and information on the user's facial image are transmitted from the user terminal 5 to the facial recognition platform 2 (step St101). As a result, the receiving means 201 of the facial recognition platform 2 receives the user's registration information and facial image information from the user terminal 5. Additionally, identification information of the user terminal 5 (for example, the serial number of the user terminal 5) is also transmitted from the user terminal 5 to the face authentication platform 2, and is accepted by the accepting means 201 of the face authentication platform 2. When the accepting means 201 accepts the user's registration information and facial image information from the user terminal 5, the control unit 20 of the face authentication platform 2 issues the user's identification information.

[0026] The encryption means 202 encrypts the information related to the user's facial image and the user's registration information, both of which have been accepted by the acceptance means 201, to generate homomorphic encryption (step St102). Here, the encryption means 202 may encrypt the user's facial image itself accepted by the acceptance means 201 to generate homomorphic encryption. Alternatively, the control unit 20 may extract a hash value from the user's facial image accepted by the acceptance means 201 using a predetermined hash function, and the encryption means 202 may encrypt this hash value to generate homomorphic encryption. In this embodiment, the information related to the user's facial image and the user's registration information accepted by the acceptance means 201 are first converted into 512-dimensional vector data, and the encryption means 202 converts the 512-dimensional vector data related to the information related to the user's facial image and the user's registration information into homomorphic encryption. The storage means 204 then associates the information related to the user's facial image and the homomorphic encryption of the user's registration information encrypted by the encryption means 202 with the user's identification information and stores them in the storage unit 22 (step St103). In this way, the user registration is completed.

[0027] Next, an operation when the information processing system 1 of this embodiment performs user authentication will be described.

[0028] For example, when a user unlocks a door located at an entrance or exit of an office building or an apartment complex at a certain service facility or enables cashless payment, the user causes the imaging unit 31 of an authentication engine 3 (e.g., authentication engine A) installed at the service facility to capture an image of the user's face. Information related to the user's facial image captured by the imaging unit 31 of the authentication engine 3 (authentication engine A) and usage history information (specifically, identification information of authentication engine A, location information of authentication engine A, information on the date and time when the user's facial image was captured by the imaging unit 31 of authentication engine A, information on the service facility where authentication engine A is installed, etc.) are transmitted from the authentication engine 3 to the face recognition platform 2 (step St201). At this time, the user's facial image captured by the imaging unit 31 may itself be transmitted from the authentication engine 3 to the face recognition platform 2, or a hash value may be extracted from the user's facial image captured by the imaging unit 31 in the authentication engine 3 using a predetermined hash function, and the extracted hash value may be transmitted from the authentication engine 3 to the face recognition platform 2. As a result, the accepting means 201 of the face authentication platform 2 accepts information about the user's face image and usage history information from the authentication engine 3 (authentication engine A).

[0029] The encryption means 202 encrypts the information related to the user's facial image and the user's usage history information accepted by the acceptance means 201, respectively, to generate homomorphic encryption (step St202). Here, the encryption means 202 may encrypt the user's facial image itself accepted by the acceptance means 201 to generate homomorphic encryption. Alternatively, the control unit 20 may extract a hash value from the user's facial image accepted by the acceptance means 201 using a predetermined hash function, and the encryption means 202 may encrypt this hash value to generate homomorphic encryption. Furthermore, when a hash value related to the user's facial image is transmitted from the authentication engine 3 to the face authentication platform 2, the encryption means 202 may encrypt this hash value to generate homomorphic encryption. Furthermore, as described above, the information related to the user's facial image and the user's registration information accepted by the acceptance means 201 are first converted into 512-dimensional vector data, and the encryption means 202 converts the 512-dimensional vector data of the information related to the user's facial image and the user's registration information into homomorphic encryption.

[0030] Then, the authentication means 203 authenticates the user by comparing the homomorphic encryption of the user's face image encrypted by the encryption means 202 with the homomorphic encryption of the user's face image stored in the storage unit 22, and acquires the user's identification information linked to the homomorphic encryption of the authenticated user's face image. Specifically, the authentication means 203 calculates the Euclidean distance between the homomorphic encryption of the user's face image sent from the authentication engine 3 and encrypted by the encryption means 202 and the homomorphic encryption of each user's face image stored in the storage unit 22 for each user stored in the storage unit 22. Here, the Euclidean distance refers to the square root of the sum of the squares of the difference between two points in a 512-dimensional Euclidean space. The homomorphic encryption of the user's face image sent from the authentication engine 3 and encrypted by the encryption means 202 is calculated as (a1, a2, a3, . . . , a 512 ), and the homomorphic encryption of the face images of each user stored in the storage unit 22 is (b1, b2, b3, . . . , b 512 ), the Euclidean distance L is expressed by the following formula:

[0031]

number

[0032] Then, the authentication means 203 determines the user's identification information corresponding to the homomorphic encryption having the smallest calculated Euclidean distance and smaller than a predetermined threshold as the user's identification information captured by the imaging unit 31 of the authentication engine 3. When the user is authenticated in this manner, the acquired user's identification information is returned from the face authentication platform 2 to the service facility where the authentication engine 3 (authentication engine A) is installed. This allows the service facility where the authentication engine 3 (authentication engine A) is installed to perform various processes for the authenticated user. Specifically, it is possible to unlock doors installed at entrances and exits of office buildings and apartment complexes, and enable cashless payments. Furthermore, when the user is authenticated, the storage means 204 stores the homomorphic encryption of the user's usage history information encrypted by the encryption means 202 (specifically, the identification information of the authentication engine A, the location information of the authentication engine A, information on the date and time when the user's face image was captured by the imaging unit 31 of the authentication engine A, information on the service facility where the authentication engine A is installed, etc.) in the storage unit 22 (step St203).

[0033] If the Euclidean distance between the homomorphic encryption of the user's face image transmitted from the authentication engine 3 and encrypted by the encryption means 202 and the homomorphic encryption of each user's face image stored in the storage unit 22 is greater than a predetermined threshold for all users stored in the storage unit 22, it is determined that the user's face image data captured by the imaging unit 31 of the authentication engine 3 is not stored in the storage unit 22, and authentication of the user is not performed. In this case, the user's identification information is not transmitted from the face authentication platform 2 to the authentication engine 3, and the authentication engine 3 does not perform any processing on the user.

[0034] Next, when the user moves to another service facility and unlocks a door located at an entrance or exit of an office building or an apartment complex or enables cashless payment at the other service facility, the user causes the imaging unit 31 of an authentication engine 3 (e.g., authentication engine B) installed at the other service facility to capture an image of the user's face. Information related to the user's facial image captured by the imaging unit 31 of the authentication engine 3 (authentication engine B) and usage history information (specifically, identification information of authentication engine B, location information of authentication engine B, information on the date and time when the user's facial image was captured by the imaging unit 31 of authentication engine B, information on the service facility where authentication engine B is installed, etc.) are transmitted from the authentication engine 3 to the face authentication platform 2 (step St204). As a result, the receiving means 201 of the face authentication platform 2 receives information related to the user's facial image and usage history information from the authentication engine 3 (authentication engine B).

[0035] The encryption means 202 encrypts the information related to the user's facial image and the user's usage history information accepted by the acceptance means 201 into homomorphic encryption (step St205). Then, the authentication means 203 authenticates the user by comparing the homomorphic encryption of the user's facial image encrypted by the encryption means 202 with the homomorphic encryption of the user's facial image stored in the storage unit 22, and acquires the user's identification information linked to the homomorphic encryption of the authenticated user's facial image (step St206). Once the user is authenticated, the face authentication platform 2 returns the acquired user's identification information to a service facility in which an authentication engine 3 (authentication engine B) is provided. This allows the service facility in which the authentication engine 3 (authentication engine B) is provided to perform various processes for the authenticated user.

[0036] Furthermore, the recommendation means 205 calculates recommendation information based on homomorphic encryption of usage history information used to authenticate the user (e.g., identification information of authentication engine B, location information of authentication engine B, information on the date and time when the user's face image was captured by the imaging unit 31 of authentication engine B, information on the service facility where authentication engine B is installed, etc.), and homomorphic encryption of other usage history information corresponding to the user's identification information already stored in the storage unit 22 (e.g., identification information of authentication engine A, location information of authentication engine A, information on the date and time when the user's face image was captured by the imaging unit 31 of authentication engine A, information on the service facility where authentication engine A is installed, etc.) (step St207). As described above, the recommendation information includes information useful to the user, such as campaign information for a service (e.g., information on discount coupons or free coupons), information on the date and time when the service is available, and information on the provision of special services. Furthermore, the recommendation means 205 may calculate recommendation information by referring not only to the homomorphic encryption of the user's usage history information stored in the storage unit 22, but also to the homomorphic encryption of the user's registration information stored in the storage unit 22. The recommendation information calculated by the recommendation means 205 is transmitted by the transmission means 206 to the user terminal 5 of the authenticated user from the face authentication platform 2 (step St208). Specifically, the transmission means 206 transmits the calculated recommendation information to the user terminal 5 corresponding to the identification information of the authenticated user. Then, the recommendation information transmitted from the face authentication platform 2 is displayed on a touch panel or the like of the user terminal 5. This allows the user to make use of the recommendation information.

[0037] A specific example of the use of recommendation information will be described below. For example, if information that a certain user frequently visits a particular type of store (e.g., a beef bowl restaurant) is stored as homomorphic encryption of usage history information of the user stored in the storage unit 22, when the user, for example, rides in a taxi and the user's face is captured by the imaging unit 31 of the authentication engine 3 installed in the taxi so that the user can pay the taxi fare cashlessly when getting out, information about the user's facial image and usage history information (specifically, identification information of the authentication engine 3, location information of the authentication engine 3, information on the date and time when the user's facial image was captured by the imaging unit 31 of the authentication engine 3, and information about the taxi in which the authentication engine 3 is installed) are transmitted from the authentication engine 3 to the face authentication platform 2. Then, the face authentication platform 2 authenticates the user by the authentication means 203. Furthermore, the recommendation means 205 calculates recommendation information based on homomorphic encryption of the usage history information for which the user has been authenticated (i.e., the usage history information transmitted from the taxi's authentication engine 3 to the face recognition platform 2) and homomorphic encryption of other usage history information corresponding to the user's identification information already stored in the storage unit 22 (i.e., information that the user frequently visits a specific type of store (e.g., a beef bowl restaurant)). Specifically, the recommendation information calculated is information about specific types of stores (i.e., beef bowl restaurants) that are open on the date and time when the user's face image is captured by the imaging unit 31 of the authentication engine 3 and that are close to the location information of the authentication engine 3. The transmission means 206 then transmits this recommendation information from the face recognition platform 2 to the user terminal 5 corresponding to the user's identification information authenticated by the authentication means 203. This allows the user to learn information about specific types of stores close to the location where he or she got off the taxi by viewing the recommendation information displayed on the user terminal 5.

[0038] In this way, when the recommendation means 205 calculates the recommendation information, for example, the service facility where the authentication engine 3 where the user was authenticated is installed substantially matches the service facility in the usage history information corresponding to the user's identification information already stored in the storage unit 22, and the date and time when the user's face image was captured by the imaging unit 31 of the authentication engine 3 where the user was authenticated is within the business hours of the authentication engine 3 or service facility in the usage history information corresponding to the user's identification information stored in the storage unit 22, and information about the service facility that is close to the location information of the authentication engine 3 where the user was authenticated is output as the recommendation information. Note that the calculation method of the recommendation information by the recommendation means 205 is not limited to this method, and various other methods may be used as the method of calculating the recommendation information. For example, when the recommendation 205 calculates the recommendation information, the recommendation information may be outputted from a service facility in the same industry where the authentication engine 3 where the user authentication was performed is installed but is not the same as the service facility in the usage history information corresponding to the user's identification information already stored in the storage unit 22, and the date and time when the user's face image was captured by the imaging unit 31 of the authentication engine 3 where the user authentication was performed is within the business hours of the service facility in the same industry, and the service facility is close to the location information of the authentication engine 3 where the user authentication was performed. Furthermore, when the recommendation 205 calculates the recommendation information, the recommendation information may be outputted from a service facility in the same industry where the date and time when the user's face image was captured by the imaging unit 31 of the authentication engine 3 where the user authentication was performed is within the business hours of the authentication engine 3 or the service facility in the usage history information corresponding to the user's identification information stored in the storage unit 22, and the service facility is close to the location information of the authentication engine 3 where the user authentication was performed. In this way, various methods can be considered for calculating recommendation information by the recommendation means 205 based on homomorphic encryption of other usage history information corresponding to the user's identification information already stored in the memory unit 22.

[0039] Furthermore, homomorphic encryption of the user's registration information may be used when calculating the recommendation information. Specifically, the recommendation means 205 calculates the recommendation information by also referencing homomorphic encryption of at least one of the gender and age in the user's registration information stored in the storage unit 22. More specifically, for example, for a user whose registration information indicates that he or she is a man in his or her 40s, when the image capture unit 31 of the authentication engine 3 installed in the taxi captures an image of the user's face so that the user can pay the taxi fare cashlessly when getting off the taxi, information about fast food restaurants that are close to the location of the authentication engine 3 and are open for business is displayed as the recommendation information. On the other hand, for a user whose registration information indicates that he or she is a woman in his or her 20s, information about nail salons and beauty salons that are close to the location of the authentication engine 3 and are open for business is displayed as the recommendation information, rather than information about fast food restaurants.

[0040] Furthermore, campaign information available at a specific store (for example, information on discount coupons or free coupons) may be calculated as recommendation information by the recommendation means 205, and the calculated campaign information may be transmitted as recommendation information by the transmission means 206 from the face recognition platform 2 to the user terminal 5 corresponding to the identification information of the user authenticated by the authentication means 203. This allows the authenticated user to enjoy various services such as discounts on fees.

[0041] According to the information processing system 1, program, and information processing method of the present embodiment configured as described above, when a user is registered, the encryption means 202 encrypts information related to the user's facial image transmitted from the user terminal 5 into a homomorphic cipher, and the storage means 204 stores the homomorphic cipher of the information related to the user's facial image in the storage unit 22. When a user is authenticated, the encryption means 202 encrypts information related to the user's facial image and usage history information transmitted from the authentication engine 3 having the imaging unit 31 that captures the user's facial image into a homomorphic cipher, respectively, and the authentication means 203 authenticates the user by comparing the homomorphic cipher of the encrypted information related to the user's facial image transmitted from the authentication engine 3 with the homomorphic cipher of the information related to the user's facial image already stored in the storage unit 22, and the storage means 204 stores the homomorphic cipher of the usage history information used for the user's authentication in the storage unit 22. Furthermore, the recommendation means 205 calculates recommendation information based on the homomorphic encryption of the usage history information for which the user has been authenticated and the homomorphic encryption of other usage history information of this user that has already been stored in the storage unit 22, and the transmission means 206 transmits the calculated recommendation information to the user terminal 5 of the authenticated user. This reduces the risk of information leakage of the user's face image and the user's usage history information, thereby improving security.

[0042] In particular, homomorphic encryption is an encryption method that allows binary operations such as addition and multiplication to be performed when two ciphertexts are given, without the need for plaintext or a secret key, and therefore, without decrypting the homomorphic encryption of the user's face image, the authentication means 203 can authenticate the user by comparing the homomorphic encryption of the encrypted user's face image with the homomorphic encryption of the user's face image stored in the memory unit 22. Furthermore, without decrypting the homomorphic encryption of the user's usage history information, the recommendation means 205 can calculate recommendation information based on the homomorphic encryption of the encrypted user's usage history information and the homomorphic encryption of the user's usage history information stored in the memory unit 22.

[0043] In the information processing system 1, program, and information processing method according to the present embodiment, user identification information is issued when a user registers. The storage unit 204 associates information about the user's facial image and the homomorphic encryption of the user's registration information with the user's identification information and stores them in the storage unit 22. The authentication unit 203 authenticates the user by comparing the homomorphic encryption of the encrypted information about the user's facial image with the homomorphic encryption of information about the user's facial image already stored in the storage unit 22, and acquires the user's identification information associated with the homomorphic encryption of the information about the authenticated user's facial image. The recommendation unit 205 calculates recommendation information based on the homomorphic encryption of the usage history information used to authenticate the user and the homomorphic encryption of other usage history information corresponding to the user's identification information already stored in the storage unit 22. The transmission unit 206 transmits the calculated recommendation information to the user terminal 5 corresponding to the authenticated user's identification information. Using the user's identification information in this way makes it possible to more reliably transmit recommendation information to the user terminal 5 of the authenticated user.

[0044] In addition, when comparing the homomorphic encryption of the information relating to the user's face image that has been encrypted and sent from the authentication engine 3 with the homomorphic encryption of the information relating to the user's face image that has already been stored in the memory unit 22, the authentication means 203 calculates the Euclidean distance for each piece of user identification information, and identifies the user identification information whose Euclidean distance is the smallest and is smaller than a predetermined threshold as the user identification information corresponding to the user's face image information received from the authentication engine 3.

[0045] Furthermore, in the information processing system 1, program, and information processing method of this embodiment, when a user registers, the encryption means 202 also encrypts the user's registration information transmitted from the user terminal 5 into homomorphic encryption, the storage means 204 stores the homomorphic encryption of the user's registration information in the storage unit 22, and the recommendation means 205 calculates recommendation information by also referencing the homomorphic encryption of the user's registration information stored in the storage unit 22. This makes it possible to provide the user with recommendation information that better suits the user's needs. Furthermore, it is possible to reduce the risk of information leakage of the user's registration information, thereby improving security.

[0046] The information processing system, program, and information processing method according to the present embodiment are not limited to the above-described aspects, and various modifications can be made.

[0047] For example, in the above description, the information on the user's facial image, the user's registration information, and the user's usage history information are all encrypted by the encryption means 202 to generate homomorphic encryption, but the present embodiment is not limited to this. One or two of the information on the user's facial image, the user's registration information, and the user's usage history information may be stored in the storage unit 22 without being encrypted. For example, when the reception means 201 of the face authentication platform 2 receives the user's registration information and facial image information from the user terminal 5 during user registration, the information on the user's facial image may be encrypted by the encryption means 202 to generate homomorphic encryption, but the user's registration information may be linked to the user's identification information and stored in the storage unit 22 without being encrypted by the encryption means 202.

[0048] Furthermore, when the receiving means 201 of the face authentication platform 2 receives the user's usage history information and facial image information from the user terminal 5 during user authentication, the information regarding the user's facial image is encrypted by the encryption means 202 to become homomorphic encryption, but the user's usage history information may be linked to the user's identification information as is without being encrypted by the encryption means 202 and stored in the storage unit 22. In this case, when the user's facial image and usage history information transmitted from the authentication engine 3 are received during user authentication, the encryption means 202 encrypts the information regarding the user's facial image transmitted from the authentication engine 3 to become homomorphic encryption, the authentication means 203 authenticates the user by comparing the homomorphic encryption of the encrypted information regarding the user's facial image transmitted from the authentication engine 3 with the homomorphic encryption of the information regarding the user's facial image already stored in the storage unit 22, and the storage means 204 stores the usage history information used to authenticate the user in the storage unit 22. In addition, the recommendation means 205 calculates recommendation information based on the usage history information of the authenticated user and other usage history information of this user that has already been stored in the memory unit 22, and the transmission means 206 transmits the calculated recommendation information to the user terminal 5 of the authenticated user.

[0049] Furthermore, in the above explanation, an aspect has been described in which the encryption means 202 that encrypts information about a user's facial image and the like to generate homomorphic encryption is provided in the face recognition platform 2, but the present embodiment is not limited to such an aspect. An information processing system 1a according to a modified example will be explained using Fig. 4. With regard to the components of the information processing system 1a shown in Fig. 4, components that are substantially the same as those in the information processing system 1 shown in Fig. 1 will be assigned the same reference numerals and their explanation will be omitted.

[0050] 4, an encryption server 4 is communicatively connected to each authentication engine 3, and each encryption server 4 is communicatively connected to the face authentication platform 2. Furthermore, when a user is registered, information relating to the user's face image and the user's registration information to be transmitted from the user terminal 5 to the face authentication platform 2 are transmitted to the encryption server 4.

[0051] The encryption server 4 has encryption means 41 that encrypts information related to a user's facial image and the user's registration information transmitted from the user terminal 5, or information related to a user's facial image and the user's usage history information transmitted from the authentication engine 3, to generate homomorphic encryption. The encryption means 41 generally has the same function as the encryption means 202 in the information processing system 1 shown in FIG. 1 . The information related to a user's facial image and the user's registration information transmitted from the user terminal 5 to the encryption server 4 at the time of user registration are each encrypted by the encryption means 41 to generate homomorphic encryption, and the homomorphic encryption of the information related to the user's facial image and the homomorphic encryption of the user's registration information are each transmitted from the encryption server 4 to the face authentication platform 2. The information related to a user's facial image and the user's usage history information transmitted from the authentication engine 3 to the encryption server 4 at the time of user authentication are each encrypted by the encryption means 41 to generate homomorphic encryption, and the homomorphic encryption of the information related to the user's facial image and the homomorphic encryption of the user's usage history information are each transmitted from the encryption server 4 to the face authentication platform 2.

[0052] 1 , in such information processing system 1a, at the time of user registration, encryption means 41 encrypts information relating to the user's facial image transmitted from user terminal 5 into homomorphic encryption, and storage means 204 stores the homomorphic encryption of the information relating to the user's facial image in storage unit 22. Furthermore, at the time of user authentication, encryption means 41 encrypts information relating to the user's facial image transmitted from authentication engine 3 having imaging unit 31 that captures the user's facial image and usage history information into homomorphic encryption, respectively, authentication means 203 authenticates the user by comparing the homomorphic encryption of the encrypted information relating to the user's facial image with the homomorphic encryption of information relating to the user's facial image already stored in storage unit 22, and storage means 204 stores the homomorphic encryption of the usage history information used for user authentication in storage unit 22. Furthermore, the recommendation means 205 calculates recommendation information based on the homomorphic encryption of the usage history information for which the user has been authenticated and the homomorphic encryption of other usage history information of this user that has already been stored in the storage unit 22, and the transmission means 206 transmits the calculated recommendation information to the user terminal 5 of the authenticated user. This reduces the risk of information leakage of the user's face image and the user's usage history information, thereby improving security.

[0053] 4, the control unit 20 of the face authentication platform 2 may also function as an encryption means, and information relating to the user's face image captured by the imaging unit 51 of the user terminal 5 and the user's registration information input to the user terminal 5 may be sent to the face authentication platform 2 without being sent to the encryption server 4. In this case, the information relating to the user's face image transmitted from the user terminal 5 to the face authentication platform 2 is encrypted by the encryption means in the control unit 20 to become a homomorphic cipher, and the homomorphic cipher of this information relating to the user's face image is stored in the memory unit 22 by the memory means 204.

[0054] Furthermore, in the above description, the facial recognition platform 2 mainly functions as a program, but this is not limiting. Some of the receiving means 201, encryption means 202, authentication means 203, storage means 204, recommendation means 205, and transmission means 206 may be executed by a device separate from the facial recognition platform 2. Furthermore, in the above description, the storage unit 22 is provided in the facial recognition platform 2, but the present embodiment is not limited to this. Instead of providing the storage unit 22 in the facial recognition platform 2, the homomorphic encryption of the user's registration information, the homomorphic encryption of information related to the user's facial image, and the homomorphic encryption of the user's usage history information may be stored in a storage device or cloud data service separate from the facial recognition platform 2, in association with the user's identification information.

[0055] Furthermore, the information processing system 1, 1a according to this embodiment may not be provided with the recommendation means 205, and may not provide recommendation information to the user. In this case, the face authentication platform 2 only authenticates the user. That is, when registering a user, the encryption means 202 encrypts information related to the user's facial image transmitted from the user terminal 5 into homomorphic encryption, and the storage means 204 stores the homomorphic encryption of the information related to the user's facial image in the storage unit 22. Furthermore, when authenticating a user, the encryption means 202 encrypts information related to the user's facial image and usage history information transmitted from the authentication engine 3 into homomorphic encryption, and the authentication means 203 authenticates the user by comparing the homomorphic encryption of the encrypted information related to the user's facial image with the homomorphic encryption of the information related to the user's facial image already stored in the storage unit 22. [Explanation of symbols]

[0056] 1, 1a Information Processing System 2. Facial Recognition Platform 20 Control Unit 22 Memory section 24 Communication Interface 201 Reception method 202 Encryption method 203 Authentication Methods 204 Memory means 205 Recommendation Methods 206 Transmission Method 3 Authentication Engine 31 Imaging unit 4 Encryption Server 41 Encryption methods 5. User terminal 51 Imaging unit

Claims

1. a face storage unit in which face data for comparison of a user is registered; An imaging means for capturing an image of a user's face; an authentication means for extracting features from the photographed face of the user and comparing the extracted features with the comparison face data; an encryption means for converting the user's face authentication usage history information into homomorphic encryption; a history storage unit in which the encrypted usage history information is stored; a recommendation unit that calculates the usage history information while it is still encrypted and generates recommendation information for the user; The history storage unit is a data storage provided by a cloud service provider. Information processing system.

2. A personal information storage unit is provided as data storage provided by a cloud service provider, The personal information storage unit stores user registration information in an encrypted form. The information processing system according to claim 1 .

Images