Terminal device, information processing system, data update method, and program

Abstract

Provided is a terminal device comprising a processor and a camera. The processor acquires first face image data obtained by imaging the face of a user by the camera, acquires preliminarily registered second face image data or third face image data held in identify verification certificate data certifying the identity of the user, checks the first face image data against the second face image data or the third face image data, and updates the third face image data on the basis of the first face image data when the check is successful.

Description

Terminal device, information processing system, data update method and program

[0001] The present disclosure relates to a terminal device, an information processing system, a data update method, and a program.

[0002] In recent years, digital wallets that electronically integrate functions such as payment, asset management, and certification have been studied. Known technologies related to digital wallets include devices and methods that use biometric technology to ensure secure transactions using blockchain technology. This method uses biometric authentication when a user uses a digital wallet to verify that the user is the same person who uses the service as the digital wallet holder (see, for example, Patent Document 1).

[0003] Japan Special Table No. 2022-508773

[0004] In the terminal of Patent Document 1, the accuracy of user identity verification using certificate data may be reduced, and there is room for improvement.

[0005] The present disclosure provides a terminal device, an information processing system, a data update method, and a program that can improve the accuracy of identity verification using facial image data of identity verification certificate data that electronically certifies the identity of a person.

[0006] One aspect of the present disclosure is a terminal device comprising a processor and a camera, wherein the processor acquires first facial image data obtained by capturing an image of a user's face with the camera, acquires pre-registered second facial image data or third facial image data stored in identity verification certificate data that proves the user is the user, compares the first facial image data with the second facial image data or the third facial image data, and if the comparison is successful, updates the third facial image data based on the first facial image data.

[0007] One aspect of the present disclosure is an information processing system comprising a terminal device and a server device, wherein the terminal device acquires first facial image data obtained by capturing an image of a user's face with a camera, acquires pre-registered second facial image data or third facial image data stored in personal identification certificate data that proves the user is the user, compares the first facial image data with the second facial image data or the third facial image data, and if the comparison is successful, updates the third facial image data based on the first facial image data.

[0008] One aspect of the present disclosure is a data update method comprising: acquiring first facial image data obtained by capturing an image of a user's face with a camera; acquiring pre-registered second facial image data or third facial image data stored in identity verification certificate data that proves the user is the user; comparing the first facial image data with the second facial image data or the third facial image data; and, if the comparison is successful, updating the third facial image data based on the first facial image data.

[0009] One aspect of the present disclosure is a program that causes a computer to execute the above-described data update method.

[0010] According to the present disclosure, it is possible to improve the accuracy of identity verification using facial image data of identity verification certificate data that electronically certifies the identity of a person.

[0011] FIG. 1 shows an example of the configuration of a terminal device in a first embodiment of the present disclosure. FIG. 2 shows an example of the display of a menu screen of a digital wallet application. FIG. 3 shows an example of the display of a wallet list screen of a digital wallet application. Flowchart (1) showing a first example of the operation of a terminal device. Flowchart (2) showing a first example of the operation of a terminal device. Flowchart (1) showing a second example of the operation of a terminal device. Flowchart (2) showing a second example of the operation of a terminal device. FIG. 1 shows a first example of the display of an update reminder. FIG. 2 shows a second example of the display of an update reminder. FIG. 2 shows an example of the configuration of an information processing system in a second embodiment of the present disclosure. Sequence diagram showing an example of the operation of an information processing system when issuing data for an official identification card. When issuing data for a private certificate. FIG. 1 is a sequence diagram showing an example of the operation of the information processing system when the expiration date of face image data is approaching. FIG. 2 is a sequence diagram showing an example of the operation of the information processing system when the face matching accuracy is reduced when logging in to a wallet application. FIG. 1 is a sequence diagram showing an example of the operation of the information processing system when the face matching accuracy is reduced when logging in to a wallet application. FIG. 1 is a sequence diagram showing an example of the operation of the information processing system when an expiration date alert for a certificate is issued. FIG. 2 is a sequence diagram showing an example of the operation of the information processing system when an expiration date alert for a certificate is issued.

[0012] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0013] (Background to the development of the embodiment of the present disclosure) Certificate data stored in a digital wallet may contain a facial image (face photo) of the user who uses the certificate data. In real space, even for the same person (user), facial features may change over time. As a result, the facial image at the time of registering the certificate data may not match the facial photo at the time the certificate data is used. The method of Patent Document 1 does not take into consideration updating the facial image of the certificate data stored in the digital wallet when a facial image is used as facial information for biometric authentication. In this case, it is difficult to perform appropriate face matching using the facial image when using the certificate data in the digital wallet, which may result in insufficient identity verification accuracy.

[0014] In the following embodiments, a terminal device, an information processing system, a data update method, and a program that can improve the accuracy of personal identification using face image data of personal identification certificate data that electronically certifies the identity of a person will be described.

[0015] 1 is a diagram showing an example of the configuration of a terminal device according to a first embodiment of the present disclosure. The terminal device 10 executes a digital wallet application.

[0016] The terminal device 10 includes a processor 11 , a memory 12 , a communication unit 13 , an input unit 14 , a display unit 15 , and a camera 16 .

[0017] The processor 11 may be configured using, for example, a central processing unit (CPU), a digital signal processor (DSP), or a graphic processing unit (GPU). The processor 11 may also be configured using various integrated circuits (for example, a large scale integration (LSI) or a field programmable gate array (FPGA)). The processor 11 realizes various functions by executing programs stored in the memory 12. The processor 11 comprehensively controls each unit of the terminal device 10 and performs various processes.

[0018] For example, the processor 11 executes a digital wallet application, and in cooperation with the camera 16, performs processing related to updating face image data of various certificate data stored in the digital wallet.

[0019] The memory 12 includes, for example, Random Access Memory (RAM) or Read Only Memory (ROM). The memory 12 may include either volatile memory or non-volatile memory. The memory 12 may include, for example, a Hard Disk Drive (HDD), a Solid State Drive (SSD), an optical disk, an SD card, etc. The memory 12 may also be an external storage medium that is detachable from the terminal device 10. The memory 12 stores various data, information, programs, etc.

[0020] The memory 12 stores, for example, a digital wallet (digital ID wallet). The digital wallet can store various certificate data (identification certificate data) that electronically certify the identity of a user. The various certificate data stores facial image data of users who use the certificate data. The facial image data of the certificate data can be updated. The certificate data may store multiple pieces of facial image data acquired at multiple times, or may store facial image data in which old facial image data has been updated (e.g., overwritten) with new facial image data. The memory 12 may also store one or more pieces of facial image data acquired at a predetermined time (e.g., registered facial image data, described later).

[0021] The communication unit 13 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication unit 13 may include, for example, a local area network (LAN), a wide area network (WAN), a mobile phone network, or power line communication.

[0022] The communication unit 13 communicates with, for example, an external communication device or a network.

[0023] The input unit 14 may include various buttons, keys, a mouse, a keyboard, a touch panel, a microphone, or other input devices. The input unit 14 accepts input of various data or information. The input unit 14 may be operated by a user of the terminal device 10. The user here is, for example, a user of a digital wallet, who owns and uses the certificate data.

[0024] The input unit 14 receives, for example, general operations for the terminal device 10 and operations related to the execution of applications.

[0025] The display unit 15 is, for example, a liquid crystal display or an organic EL display. The display unit 15 displays various data or information. The display on the display unit 15 may be confirmed by, for example, a user.

[0026] The display unit 15 displays, for example, a screen used in a digital wallet application, certificate data, face image data, and the like.

[0027] <Digital Wallet Application> Next, a digital wallet application will be described.

[0028] The digital wallet application performs processes related to, for example, displaying, updating, changing, adding, etc., certificate data stored in the digital wallet or certificate data to be stored.

[0029] The digital wallet stores various types of certificate data. The certificate data includes, for example, certificate data corresponding to official certificates issued by public institutions and certificate data corresponding to private certificates issued by private institutions. Official certificates include, for example, licenses (e.g., driver's licenses), My Number cards, and international driver's licenses. Private certificates include, for example, employee ID cards, qualification certificates, and student ID cards. Some certificates have expiration dates. Furthermore, the certificate data in this embodiment broadly includes data that can be used to authenticate a user's identity using facial image data, and may also include, for example, electronic tickets with facial image data. The digital wallet may also have a credit card function, an electronic money function, or a points function that can be used for electronic payments.

[0030] FIG. 2 is a diagram showing an example of the menu screen G1 of the digital wallet application.

[0031] The menu screen G1 has a plurality of buttons BT (icons). The buttons BT include an add wallet button, an online authentication button, an identity verification button, an age authentication button, a face image update button BT1, a wallet button, and a settings button.

[0032] The add wallet button is a button for adding and storing new certificate data in the digital wallet. When the add wallet button is selected (e.g., tapped) via the input unit 14, the processor 11 activates the camera 16 and instructs it to capture an image. In accordance with the image capture instruction, the camera 16 captures an image of the identification information (e.g., a two-dimensional code (e.g., a QR code (registered trademark))) of the certificate that the digital wallet user wishes to add. The processor 11 reads the certificate identification information based on the captured information and adds the certificate data identified by this identification information as one of the digital wallet data. Note that the processor 11 may add new certificate data to the digital wallet using a method other than the above.

[0033] The wallet button is a button for checking various certificate data stored in the digital wallet. When the wallet button is selected (e.g., tapped) via the input unit 14, the processor 11 displays the certificate data stored in the digital wallet. In this case, the processor 11 may display each certificate data individually or as a list.

[0034] FIG. 3 is a diagram showing an example of a display of the wallet list screen G2 of the digital wallet application.

[0035] The wallet list screen G2 displays a list of certificate data stored in the digital wallet. In this embodiment, the various certificate data are certificate data SM to which facial image data KG is added. Figure 3 shows multiple different types of certificate data SM1, SM2, and SM3 each having facial image data KG.

[0036] When the user selects (one example of designation) the certificate data SM to be displayed on the wallet list screen G2 via the input unit 14, the processor 11 may display the selected certificate data SM in a predetermined selection screen area. The display of the certificate data on the selection screen display may be enlarged. In FIG. 3, the selected certificate data SM0 is enlarged and displayed in the upper area of ​​the wallet list screen G2. This allows the user to easily check the details of the desired certificate data.

[0037] 3 illustrates an example in which all certificate data SM includes facial image data KG, but this is not limited to this. There may be certificate data SM that does not include facial image data KG. Since updates to facial image data are performed for certificate data SM that includes facial image data, there may also be certificate data SM for which updating of facial image data is not expected.

[0038] Returning to FIG. 2 , the identity verification button is a button for disclosing predetermined information about the user's identity contained in the certificate data. As an example, assume that driver's license data is stored in the digital wallet. For example, suppose that the user of the terminal device 10 is requested by a police officer or the like to verify their identity (e.g., disclose their driver's license). In this case, when the processor 11 selects the identity verification button via the input unit 14, the processor 11 may select the driver's license data from the certificate data in the digital wallet and display the driver's license data on the display unit 15. The processor 11 may also selectively display predetermined information from the driver's license data (e.g., information requested by the police officer). In this case, the processor 11 may specify the requested information via the input unit 14 and display it.

[0039] The age confirmation button is a button for confirming the age of the user of the terminal device 10. When the age confirmation button is selected via the input unit 14, the processor 11 displays information about the user's age. In this case, the processor 11 may extract and display age information contained in any certificate data. For example, when a user receives a request for age disclosure (e.g., age information from My Number card data or driver's license data) when purchasing licensed goods (e.g., alcoholic beverages or tobacco) at a store, the user taps the age confirmation button via the input unit 14. In response to this tap, the processor 11 may obtain and display information about the user's age from the My Number card data or driver's license data.

[0040] The online authentication button is a button for authenticating a user online. When the online authentication button is selected via the input unit 14, the processor 11 activates the camera 16 and instructs it to capture an image. In accordance with the image capture instruction, the camera 16 captures an image of the two-dimensional code on the certificate storing the information to be disclosed. The processor 11 may read identification information of the certificate data based on the captured information, and display the certificate data identified by this identification information from the certificate data stored in the digital wallet.

[0041] The facial image update button BT1 is a button for updating facial image data of the certificate data stored in the digital wallet. When the facial image update button is selected via the input unit 14, the processor 11 activates the camera 16 and instructs it to capture an image. The camera 16 captures an image of a subject including the user's face in accordance with the image capture instruction, and obtains a captured image in which the user's face is captured. The processor 11 updates the facial image data of at least one piece of certificate data, for example, based on the captured image.

[0042] The settings button is a button for making various settings related to the digital wallet application.

[0043] 4A and 4B are flowcharts showing a first example of the operation of the terminal device 10.

[0044] The terminal device 10 performs processing related to updating the facial image data of the certificate data stored in the digital wallet. At this time, the terminal device 10 performs identity verification processing at least once at a predetermined timing. The identity verification processing is processing for verifying the identity of the user using the terminal device 10. The identity verification processing is processing for verifying (determining) whether the user is the real user by comparing multiple sets of facial image data to be compared. Also, Figures 4A and 4B illustrate identity verification processing A to D as examples of identity verification processing.

[0045] First, the processor 11 executes a process for logging in to the terminal device 10 (step S11). The processor 11 may perform an identity verification process A when logging in at step S11 (e.g., logging in). If identity verification (face matching) in the identity verification process A is successful, login to the terminal device 10 is permitted. In the identity verification process A, the camera 16 captures an image of the user's face, and the processor 11 may acquire facial image data including the user's face. This facial image data is one piece of captured facial image data captured for the identity verification process.

[0046] The processor 11 launches a digital wallet application (wallet app) (step S12). The processor 11 may perform identity verification processing B when (e.g., before) launching the wallet app in step S12. If identity verification (face matching) in identity verification processing B is successful, launch of the wallet app is permitted. In identity verification processing B, the camera 16 captures an image of the user's face, and the processor 11 may acquire facial image data including the user's face. This facial image data is one piece of captured facial image data captured for identity verification processing.

[0047] The processor 11 detects a tap (an example of selection or designation) on the face image update button BT1 via the input unit 14 (step S13).

[0048] In response to the tapping of the face image update button BT1, the processor 11 activates the camera 16 and sends an image capture instruction to the camera 16 (step S14).

[0049] The camera 16 receives an image capture instruction from the processor 11, captures an image of a subject including the face of the user of the terminal device 10 in accordance with the image capture instruction, and obtains facial image data including the user's face (step S15). This facial image data is one of the captured facial image data captured for identity verification processing.

[0050] When capturing an image of the user's face, the processor 11 may perform liveness determination such as "Please look to the left" or "Please close your eyes." The processor 11 may also display guide information for capturing an image of the user's face on the display unit 15. The guide information may include, for example, information such as "Please take the image in a location where the background is a light color such as white," "Are your bangs out of the way?" or "Please fit the image into the face frame." By performing loudness determination and displaying guide information, the terminal device 10 can reduce the need to re-capture the user's face.

[0051] The processor 11 determines whether or not a face has been captured by the camera 16 (step S16). For example, the processor 11 may determine whether or not a face has been captured by acquiring face image data captured by the camera 16 and analyzing the captured image to determine whether or not a human face is included in the face image data. The processor 11 may also determine whether or not a face has been captured by using a method other than the above.

[0052] If it is determined that the face has been captured (Yes in step S16), the processor 11 selects the certificate data whose face image data is to be updated (step S17). In this case, the processor 11 may manually select the certificate data to be updated via the input unit 14, for example, or may automatically select the certificate data that satisfies a predetermined update target condition (for example, within a predetermined number of days from the expiration date) as the certificate data to be updated. The processor 11 may perform identity verification processing C when selecting the certificate data to be updated in step S17 (for example, before the selection), that is, after capturing the image of the user's face in step S15. If identity verification in the identity verification processing C is successful, the selection of the certificate data to be updated is permitted, or the selection result is validated.

[0053] The processor 11 determines whether the captured facial image data conforms to the predetermined update conditions for the facial image data required by the certificate data to be updated (step S18). These update conditions may include conditions related to the resolution of the facial image data, the background, the angle and brightness of the face, the presence or absence of accessories (such as a hat), bangs (whether the eyes are covered), the capture range (whether the face is cut off in the middle), etc. The update conditions may differ depending on the type of certificate data (e.g., passport data, driver's license data). The predetermined update conditions may be stored in the memory 12, for example.

[0054] The facial image data captured in step S18 may be facial image data (captured facial image data) used in identity verification processing performed at each timing. Specifically, for example, the facial image data here may include at least one of the facial image data captured in step S15, the facial image data captured at login, and the facial image data captured at launch of the wallet application.

[0055] If it is determined that the captured facial image data does not satisfy the update condition (No in step S18), the processor 11 instructs the camera 16 to capture an image again (step S19), and then proceeds to step S15.

[0056] If it is determined that the captured face image data satisfies the update condition (Yes in step S18), the processor 11 may perform identity verification processing D after this determination.

[0057] After identity verification process D, the processor 11 determines whether identity verification has been successful, for example, by at least one of identity verification processes A to D (step S20). That is, the processor 11 determines whether face matching (authentication) of the user of the terminal device 10 (user of the wallet application) has been successful through face matching in the identity verification process.

[0058] For example, if the identity of the person cannot be confirmed by any of the identity confirmation processes A to D (No in step S20), the processor 11 instructs the camera 16 to capture an image again (step S19), and then proceeds to step S15.

[0059] For example, if the identity of the user is confirmed by at least one of the identity confirmation processes A to D (Yes in step S20), the processor 11 processes (edits) the facial image data in accordance with the standard of the selected certificate data (step S21). For example, the standard of the certificate data prescribes information that defines the size of the facial image data, background color, etc. Processing of the facial image data may involve, for example, trimming the facial image data.

[0060] The processor 11 determines whether the processed face image data conforms to the standards of the certificate data (step S22).

[0061] If it is determined that the processed facial image data does not conform to the standard of the certificate data (No in step S22), the processor 11 instructs the camera 16 to re-image (step S19), and then proceeds to step S15.

[0062] If it is determined that the processed facial image data conforms to the standards of the certificate data (Yes in step S22), the processor 11 updates the facial image data (stored facial image data) stored in the certificate data to be updated by using the processed facial image data as new facial image data for the certificate data to be updated (step S23).

[0063] 4A and 4B, the terminal device 10 can preferably update the facial image data stored in the certificate data using facial image data captured during each of identity verification processes A to C, which are performed at predetermined timings. Therefore, compared to when the facial image data is not changed from the time when the predetermined certificate data is first stored in the digital wallet, the terminal device 10 can update the facial image data in response to changes in the user's appearance in real space. Therefore, the terminal device 10 can improve the accuracy of identity verification using the facial image data of the certificate data when using the certificate data.

[0064] It should be noted that the identity verification process D does not have to be performed between step S18 and step S20. In this case, the processor 11 may simultaneously determine whether the update conditions are met and whether the certificate data meets the standards (step S21) in step S18. In this case, step S20 may be omitted. In this case, for example, the identity verification process D may be performed after step S22.

[0065] <Details of Identity Verification Processing> Next, details of identity verification processing will be described. Note that identity verification processing A to D below corresponds to identity verification processing A to D shown in Figures 4A and 4B.

[0066] The identity verification process A is a process for verifying identity for logging in to the terminal device 10. The target for verification in the identity verification process A (also referred to as the target for verification A) includes (A1) facial image data of the user's face captured by the camera 16 at the time of login, and may include, for example, feature data from the facial image data. The target for verification A may include (A2) facial image data registered when setting up login to the terminal device 10. Therefore, for example, the facial image data registered at the time of login setup may be stored in the memory 12 in advance, or facial image data captured by the camera 16 at a predetermined timing may be stored in the memory 12. (A3) The target for verification A may be facial image data of any certificate data stored in the digital wallet.

[0067] The facial image data of any of the certificate data in (A3) is preferably facial image data of certificate data whose expiration date is close to the timing of performing the identity verification process A, and a predetermined update reminder has been issued for the certificate data. Also, the facial image data of any of the certificate data is preferably facial image data of certificate data whose latest update date and time is old.

[0068] The capture date and time of facial image data in certificate data that is close to its expiration date, certificate data for which an update reminder has been issued, and certificate data with an old update date and time has a large time difference from the time when identity verification process A was performed, and the user's facial features may have changed significantly. If matching using such facial image data is successful, it can be said that there is a very high possibility that the user is the actual person. Conversely, the facial image data of any of the certificate data may be facial image data of the certificate data with the most recent update date and time, or facial image data of certificate data with a different update date and time.

[0069] The combination of multiple matching targets used in the identity verification process A is, for example, a combination of (A1) and (A2), a combination of (A1) and (A3), or a combination of (A1), (A2), and (A3). Note that other data may be used as a matching target and in the combination of multiple matching targets.

[0070] Identity verification process B is a process for verifying identity when launching a wallet application. The object of verification in identity verification process B (also referred to as verification object B) includes (B1) facial image data of the user's face captured by camera 16 when the wallet application is launched, and may include, for example, feature data from the facial image data. Verification object B may be (B2) facial image data of any of the certificate data stored in the digital wallet. In other words, (B2) is the same as (A2).

[0071] The combination of multiple matching targets used in the identity verification process B is, for example, (B1) and (B2). Note that other data may be used as a matching target and in the combination of multiple matching targets.

[0072] The identity verification process C is a process for verifying identity performed when the user's face is captured during the update of facial image data. The matching target (also referred to as matching target C) in the identity verification process C includes (C1) facial image data of the user's face captured by the camera 16 in step S15 of FIG. 4B, and may include, for example, feature data from the facial image data. The matching target C may be (C2) facial image data of any certificate data stored in the digital wallet. In other words, (C2) is the same as (A2) and (B2). The matching target C may be (C3) facial image data (registered facial image data) registered as facial image data for updating the facial image. The registered facial image data may be facial image data stored (registered) in the memory 12 at a predetermined timing.

[0073] The combination of multiple matching targets used in the identity verification process C is, for example, a combination of (C1) and (C2) or a combination of (C2) and (C3). Note that other data may be used as a matching target and in the combination of multiple matching targets.

[0074] Therefore, when the terminal device 10 uses the registered face image data (C3) for face matching, it does not need to use the captured face image data (C1) for face matching, so it is possible to omit capturing images using the camera 16 during matching, and the face image can be updated quickly.

[0075] The identity verification process D is an identity verification process that is performed when certificate data to be updated is selected during the updating of facial image data. The comparison target (also referred to as comparison target D) in the identity verification process D includes (D1) facial image data of the user's face captured by the camera 16 after the selection of the certificate data to be updated, and may include, for example, feature data from the facial image data. The comparison target D may be (D2) facial image data of the certificate data to be updated stored in the digital wallet. The comparison target C may be (D3) registered facial image data. In other words, (D3) is the same as (C3).

[0076] The combination of multiple matching targets used in the identity verification process D is, for example, a combination of (D1) and (D2) or a combination of (D2) and (D3). Note that other data may be used as a matching target and as a combination of multiple matching targets.

[0077] Incidentally, some physical certificates, such as passports, are not updated frequently. Therefore, if a long period of time passes since the certificate was issued, facial features (especially of children) may change from the original facial image data registered in the certificate data, increasing the likelihood that facial matching will fail (cause an error) when it comes time to renew the certificate, even if the user is the actual person. To avoid this situation, it is preferable that the processor 11 periodically performs face matching using facial image data that is preferred as a matching target, as shown in (A3). For example, if the matching accuracy falls below a predetermined level after matching the features of the facial image data, the processor 11 may capture an image of the user's face using the camera 16 during the matching process, obtain facial image data, and store the captured facial image data in the memory 12 as registered facial image data.

[0078] It is not essential that all four identity verification processes A to D be performed. For example, it is sufficient that at least one of identity verification processes A to D is performed. Furthermore, if identity verification process C verifies (authenticates) that the user is the real person, identity verification process D may be unnecessary.

[0079] The identity verification process includes at least the above-described matching using the facial image data of the plurality of matching targets, but other authentications may also be performed. For example, the identity verification process may include authentication by entering a PIN in addition to the matching using the facial image data, thereby further improving the accuracy of identity authentication.

[0080] In addition, if the user is verified as the actual user in the identity verification processes A to D (i.e., if the verification is successful), the facial image data captured by the camera 16 used as the subject of verification may be stored in memory 12 as registered facial image data.

[0081] <Personal Identification Processing Taking into Account Verification Errors> Next, a description will be given of personal identification processing taking into account verification errors.

[0082] In the identity verification process, when updating the facial image data of each certificate data, if a long time has passed since the last update, the face cannot be identified as the user in face matching (face authentication) using image data, and a matching error is likely to occur. Taking such a matching error into consideration, the processor 11 of the terminal device 10 may perform the following process.

[0083] For example, the processor 11 may repeatedly acquire facial image data by capturing an image of the user's face with the camera 16 at a predetermined timing (e.g., periodically) separate from the timing of updating the facial image data in the certificate data. The processor 11 may use this facial image data to check the accuracy of matching the user. The processor 11 may store the acquired facial image data in the memory 12 as registered facial image data. The predetermined timing may be, for example, the timing of logging in to the terminal device 10 (i.e., the timing of identity verification process A), the timing of launching the wallet app (i.e., the timing of identity verification process B), or another timing.

[0084] For example, if the matching check is successful, that is, if it is determined that the user is a legitimate user, the processor 11 may store the captured face image data that has been successfully matched as registered face image data in the memory 12. Since logging in to the terminal device 10 and launching the wallet app are performed relatively frequently, the processor 11 can store registered face image data that is close to the state of the user in the real space by storing the captured face image data as registered face image data when logging in to the terminal device 10 or launching the wallet app.

[0085] For example, the identity verification process may include an electronic identity verification (eKYC) process, where eKYC stands for electronic Know Your Customer.

[0086] For example, the identity verification process may include a process of visual verification by a person. The visual verification process may include a process in which the communication unit 13 transmits facial image data of the user's face captured by the camera 16 of the terminal device 10 to the administrator device, and a process in which the administrator device receives the facial image data from the terminal device 10 and displays it on a display or the like. The visual verification process may also include a process in which the administrator, who is the user of the administrator device, checks the display of the facial image data and compares it with, for example, a facial image on a certificate stored on a physical medium to determine whether the person appearing in the facial image data is the user (whether matching is successful or not). The visual verification process may also include a process in which the administrator server inputs information on the matching result from the administrator via the input unit and transmits the information on the matching result to the terminal device 10, and a process in which the terminal device 10 obtains information on the matching result from the administrator server via the communication unit 13. As a result, even when the identity verification process is a visual verification process, the processor 11 can obtain the matching result and use it for subsequent processing.

[0087] Next, an example of the operation of the terminal device 10 taking into account registered face image data will be described.

[0088] 5A and 5B are flowcharts showing a second example of the operation of the terminal device 10. Figures 5A and 5B show an example of the operation of the terminal device 10 taking into account registered face image data. In Figures 5A and 5B, the same steps as those in Figures 4A and 4B are assigned the same step numbers, and their descriptions are omitted or simplified.

[0089] The terminal device 10 updates the facial image data in the certificate data stored in the digital wallet. At this time, the terminal device 10 performs identity verification processing at least once at a predetermined timing. Then, if the identity verification processing results in verification of the identity of the person, the terminal device 10 stores (registers) the facial image data used for verification in the memory 12 so that it can be used for identity verification processing at a later timing.

[0090] First, the processor 11 executes a process for logging in to the terminal device 10 (step S11). The processor 11 may perform identity verification process A when logging in at step S11. If identity verification in identity verification process A is successful, login to the terminal device 10 is permitted. Furthermore, if identity verification in identity verification process A is successful, the processor 11 stores the face image data that was successfully matched in identity verification process A as registered face image data in, for example, the memory 12 (step SH1).

[0091] The facial image data may be stored in a memory provided in an external device or system, instead of in the memory 12 in the terminal device 10. This also applies to subsequent storage of facial image data.

[0092] The processor 11 launches a digital wallet application (wallet app) (step S12). The processor 11 may perform identity verification process B when launching the wallet app in step S12. If identity verification in identity verification process B is successful, the launch of the wallet app is permitted. Furthermore, if identity verification in identity verification process A is successful, the processor 11 stores the facial image data that was successfully matched in identity verification process B as a registered facial image, for example, in memory 12 (step SH2).

[0093] The processor 11 detects a tap on the face image update button BT1 via the input unit 14 (step S13).

[0094] The processor 11 determines whether or not there is registered face image data stored in the memory 12, for example (step S31).

[0095] If the registered face image data does not exist (No in step S31), the processor 11 activates the camera 16 and sends an image capture instruction to the camera 16 (step S14).

[0096] The camera 16 receives an image capture instruction from the processor 11, and captures an image of a subject including the face of the user of the terminal device 10 in accordance with the image capture instruction, thereby obtaining face image data including the user's face (step S15).

[0097] When capturing an image of a subject including the user's face in step S15 (for example, after capturing the image), the processor 11 may perform an identity verification process C.

[0098] The processor 11 determines whether or not the identity of the person has been verified by the identity verification process C, that is, whether or not the verification is successful (step S32).

[0099] If the identity of the person is verified by the identity verification process C (Yes in step S32), that is, if the identity verification in the identity verification process C is successful, the use of the facial image data captured in step S15 is permitted. In this case, the processor 11 also determines whether or not to save the facial image data used in the identity verification process C (step S33). For example, if an operation to save facial image data is received via the input unit 14, it may be determined that this facial image data is to be saved. Furthermore, if the number of pieces of facial image data saved in the memory 12 is equal to or less than a predetermined number, it may be determined that this facial image data is to be saved.

[0100] If it is determined that the facial image data used in the identity verification process C should be saved (Yes in step S33), the processor 11 saves (registers) the facial image data used in the identity verification process C as registered facial image data, for example, in the memory 12 (step SH3). After the processing of step SH3 is completed, the process proceeds to step S13 in FIG. 5A.

[0101] If it is determined in step S33 that the data should not be saved, the facial image data captured in step S15 can be used to update the facial image data in the certificate data, but is not saved as registered facial image data.

[0102] In step S31, if registered face image data exists (Yes in step S31), the processor 11 acquires the stored registered face image data (step S34). That is, the processor 11 uses the registered face image data stored during any of the identity verification processes A to C as face image data for updating.

[0103] In step S34, after the saved registered face image data is acquired, or if it is determined that the face image data used in the identity verification process C will not be saved (No in step S33), the processor 11 selects the certificate data to update the face image data for (step S17).

[0104] The processor 11 determines whether the acquired facial image data meets the predetermined update conditions for facial image data required by the certificate data to be updated (step S18).

[0105] The facial image data acquired in step S18 may be registered facial image data or facial image data (captured facial image data) captured for identity verification processing performed at each timing. This captured facial image data may include, for example, at least one of the facial image data captured in step S15, the facial image data captured at login, and the facial image data captured at launch of the wallet app.

[0106] If it is determined that the acquired facial image data does not satisfy the update condition (No in step S18), the processor 11 instructs the camera 16 to re-image (step S19), and then proceeds to step S15.

[0107] If it is determined that the acquired facial image data meets the update conditions (Yes in step S18), the processor 11 processes the facial image data in accordance with the standard of the selected certificate data (step S21).

[0108] The processor 11 determines whether the processed face image data conforms to the standards of the certificate data (step S22).

[0109] If it is determined that the processed facial image data does not conform to the standard of the certificate data (No in step S22), the processor 11 instructs the camera 16 to re-image (step S19), and then proceeds to step S15.

[0110] If it is determined that the processed facial image data conforms to the standards of the certificate data (Yes in step S22), the processor 11 updates the facial image data (stored facial image data) stored in the certificate data to be updated by using the processed facial image data as new facial image data for the certificate data to be updated (step S23).

[0111] Furthermore, if the verification by the identity verification process C fails in step S32, that is, if the identity of the person is not verified by the identity verification process C (No in step S32), the processor 11 instructs the camera 16 to capture an image again (step S19). Then, the process proceeds to step S15.

[0112] 5A and 5B, the terminal device 10 can preferably update the facial image data stored in the certificate data using facial image data captured during each of the identity verification processes A to C, which are performed at predetermined timings. Therefore, compared to when the facial image data is not changed from the time when the predetermined certificate data is first stored in the digital wallet, the terminal device 10 can update the facial image data in response to changes in the user's appearance in real space. Therefore, the terminal device 10 can improve the accuracy of identity verification using the facial image data of the certificate data when using the certificate data.

[0113] Furthermore, the terminal device 10 stores facial image data that has been successfully matched in the identity verification process as registered facial image data, which can be used when updating the facial image data of any certificate data later. This allows facial image data that is close to the user's current state to be registered, improving the accuracy of identity verification. Furthermore, because the terminal device 10 performs verification using the registered facial image data, it is possible to omit capturing images using the camera 16, and facial image data can be updated quickly.

[0114] 5A and 5B illustrate an example in which the personal identification process D is not performed, but this is not limiting. The personal identification process D may be performed at the same timing as in FIGS. 4A and 4B, that is, when selecting face image data to be updated.

[0115] <Facial Image Data Update Reminder> Next, the function of the facial image data update reminder will be described.

[0116] FIG. 6 is a diagram showing a first example of a display of an update reminder.

[0117] On the menu screen G1, update reminder information RI related to the update reminder may be displayed as a pop-up together with various buttons BT. That is, the processor 11 may perform the update reminder by displaying the update reminder information RI on the display unit 15, or by displaying the update reminder information RI on the menu screen G1 in FIG. 6 .

[0118] Furthermore, on the menu screen G1, the display mode DM1 of the face image update button BT1 is different from the display modes of the other buttons. The face image update button BT1 may be displayed larger than the other buttons BT, in a different color than the other buttons BT, in a different display pattern (e.g., hatched) than the other buttons BT, flashing differently than the other buttons BT, or in a different flashing pattern (e.g., fast flashing) than the other buttons BT, thereby indicating that the face image data needs to be updated. In other words, the processor 11 may issue an update reminder by displaying the face image update button BT1 in a different display mode from the other buttons BT.

[0119] FIG. 7 is a diagram showing a second example of the display of the update reminder.

[0120] On the wallet list screen G2, update reminder information RI related to the update reminder is displayed as a pop-up along with various certificate data SM. That is, the processor 11 may issue an update reminder by causing the display unit 15 to display the update reminder information RI on the wallet list screen G2. More specifically, the processor 11 may display the update reminder information RI for the certificate data SM to be updated (e.g., certificate data SM1). The certificate data SM to be updated may be specified via the input unit 14 or may be automatically specified by the processor 11.

[0121] Furthermore, when performing the identity verification process, the processor 11 may derive (e.g., calculate) the matching accuracy of the identity verification process, and if the derived matching accuracy is equal to or lower than a predetermined matching accuracy, may issue a reminder to update the facial image data as illustrated in Figures 6 and 7. In other words, the processor 11 may update the facial image data at a practical timing when the matching accuracy based on the facial image data has decreased, rather than attempting to uniformly update the facial image data based on the expiration date of the certificate data, for example.

[0122] In this way, the terminal device 10 of this embodiment can suitably update the facial image data stored in the certificate data using facial image data captured during the identity verification process and registered facial image data. Therefore, compared to when the facial image data is not changed from the time when the specified certificate data is first stored in the digital wallet, the terminal device 10 can update the facial image data in response to changes in the user's appearance in real space. Therefore, the terminal device 10 can improve the accuracy of identity verification using the facial image data of the certificate data when using the certificate data.

[0123] In the present embodiment, the terminal device 10 performs the process related to updating the facial image data by itself, but this is not limiting. For example, the process related to updating the facial image data may be distributed and performed by a system including the terminal device 10 and a server device. That is, part of the process executed by the terminal device 10 may be performed by the server device. The server device may be configured as an on-premise server device, or may be configured as a cloud server device on a network. The server device may be configured as a single computer, or may be configured as a distributed server device including multiple computers.

[0124] Second Embodiment In a second embodiment, an information processing system including a terminal device 10 and a server device performs processes related to updating face image data and updating personal identification certificate data.

[0125] 8 is a diagram illustrating a configuration example of an information processing system 5 according to the second embodiment of the present disclosure. The information processing system 5 includes a terminal device 10 and a server device 20. The terminal device 10 and the server device 20 are connected to each other so as to be able to communicate with each other via a wired or wireless connection.

[0126] The configuration of the terminal device 10 is the same as that of the terminal device 10 of the first embodiment shown in Fig. 1, and therefore a description thereof will be omitted. The terminal device 10 has at least the same components and functions as those of the first embodiment, and performs the same operations as those of the first embodiment. The terminal device 10 also has the functions of this embodiment, which will be described later, and performs the operations of this embodiment, which will be described later.

[0127] The server device 20 may be a server device of a public institution, a server device of a private business, or any other server device. The server device 20 performs various processes such as authentication, verification, and judgment, and issues various certificate data (digital certificates).

[0128] The server device 20 may be configured as an on-premise server device or as a cloud server device on a network. The server device 20 may be configured as a single computer or may be configured as a distributed server device including multiple computers.

[0129] The server device 20 includes a processor 21 , a memory 22 , and a communication unit 23 .

[0130] The processor 21 may be configured using, for example, a CPU, a DSP, or a GPU. The processor 21 may also be configured using various integrated circuits (for example, an LSI or an FPGA). The processor 21 realizes various functions by executing programs stored in the memory 22. The processor 21 comprehensively controls each part of the terminal device 10 and performs various processes.

[0131] For example, the processor 21 functions as a verification unit 211 and a certificate generation unit 212. The verification unit 211 performs various verification-related processes, such as verifying the validity of various certificate data. The certificate generation unit 212 generates predetermined certificate data.

[0132] The memory 22 includes, for example, RAM or ROM. The memory 22 may include either volatile memory or non-volatile memory. The memory 22 may include, for example, an HDD, an SSD, an optical disk, an SD card, etc. The memory 22 may also be an external storage medium, and may be detachable from the server device 20. The memory 22 stores various types of data, information, programs, etc.

[0133] The communication unit 23 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication unit 23 may include, for example, a LAN, a WAN, a mobile phone network, or power line communication. The communication unit 23 communicates with, for example, an external communication device (for example, the terminal device 10) or a network.

[0134] Next, the timing of face matching using face image data and the target of matching will be described.

[0135] The timing of face matching may include the time of logging in to the terminal device 10. Face matching at the time of logging in to the terminal device 10 is used, for example, to determine terminal authority. The terminal authority determination is a determination regarding the authority to use the terminal device 10. Face matching at the time of logging in to the terminal device 10 may be performed based on face image data captured by the terminal device 10 and face image data (an example of second face image data) registered in the terminal device 10.

[0136] The timing of face matching may include the time of logging in to the wallet app. Face matching at the time of logging in to the wallet app may be performed based on face image data captured by the terminal device 10 and face image data (an example of second face image data) registered when setting up the wallet app.

[0137] The timing of face matching may include the time when the certificate data is actually used. Face matching when the certificate data is used may be performed based on face image data captured by the terminal device 10 and face image data (an example of third face image data) stored in the selected certificate data (e.g., qualification data).

[0138] The timing of face matching may include when the face image data contained in various certificate data is updated. When the face image is updated, face matching may be performed based on the face image data captured by the terminal device 10 or the face image data captured at the time of login, and the face image data (an example of third face image data) stored in the certificate data to be updated.

[0139] Next, the operation of the information processing system 5 will be described.

[0140] FIG. 9 is a sequence diagram showing an example of the operation of the information processing system 5 when issuing data for an official identification card.

[0141] The data of the official identification card is an example of official certificate data, such as a digital My Number card (electronic My Number card) that is an electronic My Number card. In Fig. 9, the server device 20 is a server device that performs official verification and issuance.

[0142] In this embodiment, the wallet application executed by the processor 11 of the terminal device 10 is also referred to as a wallet application 111 (see FIG. 8 ). The My Number Card is also referred to as an MNC. MNC is an abbreviation for My Number Card.

[0143] It is also assumed that the server device 20 holds information about the user's physical My Number card at the start of the processing of FIG. 9 . The information about the My Number card here includes, for example, the four pieces of user information described below and facial image data. Specifically, the My Number card (MNVC) includes information such as name, address, date of birth, gender, facial image data, expiration date, and MNC electronic certificate, i.e., the My Number card electronic certificate. The digital My Number card may include the same information as the physical My Number card (MNC). Note that the information about the My Number card here includes at least some of the information included in the My Number card, and may not include facial image data.

[0144] First, the wallet application 111 of the terminal device 10 activates the camera 16 (step S101). The wallet application 111 sends an image capture request to the camera 16 (step S102). The camera 16 captures an image of the face of the user of the terminal device 10 (step S103). The wallet application 111 acquires facial image data captured by the camera 16 (also referred to as captured facial image data) from the camera 16 (step S104).

[0145] The wallet application 111 instructs the user to place the physical My Number Card (MNC) in a predetermined position on the terminal device 10, for example, by displaying predetermined information via the display unit 15 (step S105).

[0146] The user places his / her own My Number card in a predetermined position on the terminal device 10. The input unit 14 also accepts input of PIN (Personal Identification Number) information from the user (step S106).

[0147] The wallet application 111 acquires information stored in the chip of the My Number card (chip information) (step S107). The chip information includes four pieces of information held by the My Number card (specifically, address, name, date of birth, and gender) and facial image data. Note that the chip information does not have to include facial image data. The wallet application 111 may acquire the chip information from the My Number card, for example, by contactless wireless communication (NFC: Near Field Communication) or contact wireless communication of the communication unit 13.

[0148] The wallet application 111 performs face matching based on the captured face image data and the face image data included in the chip information (step S108). This face matching checks whether the two sets of face image data are of the same person. The wallet application 111 determines whether the face matching is successful (face matching OK) (step S109).

[0149] The wallet application 111 transmits the acquired chip information to the server device 20 via the communication unit 13 (step S110). The server device 20 receives the chip information from the terminal device 10.

[0150] The server device 20 performs a predetermined validity determination based on the in-chip information (step S111). For example, the server device 20 compares information about the My Number card stored in advance as information for each user with the acquired in-chip information. The server device 20 then determines whether or not there is information about the My Number card that matches the in-chip information. If there is information about the My Number card that matches the in-chip information, the card may be determined to be valid. If there is no information about the My Number card that matches the in-chip information, the card may be determined to be invalid.

[0151] Based on the above determination, the server device 20 determines whether the user's My Number card that holds the chip information is valid (step S112). If it is determined to be valid (Yes in step S113), the server device 20 generates an electronic My Number card (electronic MNC) by digitizing the My Number card of the user that was determined to be valid (step S113). The electronic My Number card may contain the same information as a physical My Number card, such as name, address, date of birth, gender, facial image data, expiration date, MNC electronic certificate, etc. In other words, expiration date information is attached to this electronic My Number card.

[0152] The server device 20 issues an electronic My Number card with expiration date information attached thereto, and transmits this electronic My Number card to the terminal device 10 (step S114).

[0153] The wallet application 111 of the terminal device 10 receives the electronic My Number card from the server device 20 via the communication unit 13 and stores it in the wallet application 111 (step S115). The wallet application 111 also refers to the expiration date information attached to the electronic My Number card and manages the expiration date of the electronic My Number card (step S115).

[0154] According to the operation example of Figure 9, the information processing system 5 acquires information such as the four information and facial image data from the My Number card, generates and issues an electronic My Number card whose validity is guaranteed by the server device 20, and stores it in the wallet application 111 of the terminal device 10.

[0155] If the information in the chip of the My Number Card does not include facial image data, steps S108 and S109 may be omitted, i.e., the process related to face matching may be omitted. In this case, when the wallet application 111 recognizes that facial image data has not been acquired in step S107, it proceeds to step S110. Then, if the server device 20 determines that the information in the chip other than the facial image data is valid, the wallet application 111 of the terminal device 10 stores the electronic My Number Card in the wallet application.

[0156] Fig. 10 is a sequence diagram showing an example of the operation when private certificate data is issued by the information processing system 5. In Fig. 10, the description of the same processes as those in the example of operation shown in Fig. 9 will be omitted or simplified.

[0157] The private certificate data is an example of private certificate data, such as a digital employee ID card (electronic employee ID card) that is an electronic version of an employee ID card. In Fig. 10, the server device 20 is a server device that performs private verification and issuance.

[0158] It is also assumed that, at the start of the processing of Figure 10, the server device 20 is managing and registering (retaining) information related to the user's physical employee ID card, i.e., employee information related to employees of a specific company. The employee information here includes, for example, the employee's name, address, date of birth, gender, facial image data, expiration date, employee number, etc. Note that the employee information here may not include facial image data. The employee information here also includes information on the expiration date of the employee ID card, etc.

[0159] First, the wallet application 111 of the terminal device 10 activates the camera 16 (step S201). The wallet application 111 sends an image capture request to the camera 16 (step S202). The camera 16 captures an image of the face of the user of the terminal device 10 (step S203). The wallet application 111 acquires facial image data captured by the camera 16 (captured facial image data) from the camera 16 (step S204).

[0160] The wallet application 111 instructs the user to input necessary information, for example, by displaying predetermined information via the display unit 15. The input unit 14 accepts the input of the necessary information from the user (step S205). The necessary information here is, for example, information necessary for employee verification, and includes the user's name and employee number. The wallet application 111 acquires the necessary information input from the input unit 14 (step S206).

[0161] The wallet application 111 transmits the captured face image data, the acquired necessary information, and issuance request information requesting the issuance of a digital employee ID card to the server device 20 via the communication unit 13 (step S207). The server device 20 receives the captured face image data, the necessary information, and the issuance request information.

[0162] The server device 20 collates the employee information for the user of the terminal device 10 based on the captured facial image data and the necessary information in accordance with the acquired issuance request information (step S208). For example, the server device 20 compares the employee information for each employee with the acquired captured facial image data and necessary information. The server device 20 then determines whether or not there is facial image data and necessary information (name, employee number, and other information) that matches the employee information.

[0163] Based on the above determination, the server device 20 determines whether the user corresponding to the facial image data and necessary information is a legitimate employee (matching OK) (step S209). If it is determined that the user is a legitimate employee (Yes in step S209), the server device 20 generates a digital employee ID card by digitizing the employee ID card of the user determined to be valid (step S210). The digital employee ID card may contain the same information as a physical employee ID card, such as the employee's name, address, date of birth, gender, facial image data, expiration date, employee number, etc. In other words, expiration date information is attached to this digital employee ID card.

[0164] The server device 20 issues a digital employee ID card with expiration date information attached thereto, and transmits this digital employee ID card to the terminal device 10 (step S211).

[0165] The wallet application 111 of the terminal device 10 receives the digital employee ID card from the server device 20 via the communication unit 13 and stores it in the wallet application 111 (step S212). The wallet application 111 also references the expiration date information attached to the digital employee ID card and manages the expiration date of the digital employee ID card (step S212).

[0166] According to the operation example of Figure 10, the information processing system 5 obtains the necessary information required to issue a digital employee ID card from the input of the employee user, generates and issues a digital employee ID card whose validity is guaranteed by the server device 20, and stores it in the wallet application 111 of the terminal device 10.

[0167] 11A and 11B are diagrams showing an example of the operation of the information processing system 5 when the expiration date of the facial image data is approaching. In Fig. 11A and Fig. 11B, the description of the same processes as those in the example of operation shown in Fig. 9 or Fig. 10 will be omitted or simplified.

[0168] 11A and 11B, it is assumed that the facial image data held by various certificate data has an expiration date. The expiration date information is included in the certificate data, for example. In FIG. 11A and 11B, the server device 20 is a server device that performs public or private verification and issuance.

[0169] First, the wallet application 111 of the terminal device 10 references the expiration date information of each stored credential data (an example of certificate data), measures the current time, and determines whether the expiration date of the facial image data for any credential data will arrive (step S301). In other words, the wallet application 111 determines whether the predetermined period from the current time to the expiration date of the facial image data is equal to or less than threshold value th1 (e.g., within six months). This determination of whether the expiration date will arrive may be made continuously or discontinuously. It may also be made, for example, at a predetermined time every day.

[0170] If the expiration date is approaching (Yes in step S301), the wallet application 111 issues an alert indicating that the expiration date is approaching (step S302) by, for example, displaying predetermined information on the display unit 15. Note that at the timing when the expiration date arrives or when the alert is issued, the expiration date of the credential data has not yet arrived or the credential data is still valid.

[0171] The user checks the alert regarding the expiration date. The input unit 14 accepts input from the user to update the facial image data. In this case, for example, when the expiration date alert is issued, the display unit 15 displays the menu screen G1 of the wallet application 111 illustrated in FIG. 2. The input unit 14 accepts pressing of the facial image update button BT1 on this menu screen G1 (step S303).

[0172] The wallet application 111 activates the camera 16 (step S304). The wallet application 111 sends an image capture request to the camera 16 (step S305). The camera 16 captures an image of the face of the user of the terminal device 10 (step S306). The wallet application 111 acquires face image data captured by the camera 16 (captured face image data) from the camera 16 (step S307).

[0173] The wallet application 111 performs face matching of the user's face. In this case, the wallet application 111 performs face matching based on, for example, captured face image data and registered face image data (step S308). The registered face image data may be acquired from the memory 12. Alternatively, the wallet application 111 may perform face matching based on the captured face image data and face image data included in the certificate data.

[0174] The wallet application 111 may send captured facial image data via the communication unit 13, and the server device 20 may acquire the captured facial image data, perform facial matching, and transmit the facial matching result to the terminal device 10. In this case, the server device 20 may perform facial matching based on the captured facial image and facial image data included in information related to the My Number card or employee information stored in the server device 20.

[0175] The wallet application 111 determines whether the face matching is successful (face matching OK) (step S309). If the face matching is unsuccessful (No in step S309), the wallet application 111 proceeds to step S304 and controls the camera 16 to re-image the user's face.

[0176] The wallet application 111 acquires information on processing conditions for the facial image data (step S310). The information on processing conditions may be stored in advance in, for example, the memory 12 and acquired from the memory 12. The information on processing conditions may include, for example, information specifying the size and background color of the facial image data according to the certificate data standard. The processing conditions for the facial image data here correspond to the update conditions for the facial image data described in the first embodiment.

[0177] The wallet application 111 processes the face image data based on the acquired processing conditions (step S311). In this case, the wallet application 111 performs, for example, trimming of the face image data.

[0178] The information regarding the processing in this embodiment may be the same as the processing described in the first embodiment.

[0179] After step S311, the wallet application 111 associates the captured face image data (processed captured face image data) with the certificate data (also referred to as stored certificate data) stored in the wallet application 111 (step S312). The certificate data here is, for example, an electronic My Number card or a digital employee ID card. The certificate data here is existing old certificate data. The stored certificate data in step S312 is stored certificate data for the certificate data whose face image data is about to expire.

[0180] The wallet application 111 transmits a verification request for verifying the stored credential data to the server device 20 via the communication unit 13 (step S313). This verification request may include, for example, all data included in the stored credential data and the processed captured face image data. Alternatively, this verification request may include the electronic certificate included in the stored credential data and the processed captured face image data, but may not include other data. The electronic certificate may include an electronic signature.

[0181] The server device 20 receives the verification request from the terminal device 10. The server device 20 verifies the validity of the stored credential data in accordance with the verification request (step S314). For example, the server device 20 stores in advance information about the My Number card and employee information for each user and employee, as exemplified in Figures 9 and 10.

[0182] When the verification request includes all data included in the stored credential data and the processed captured facial image data, the server device 20 may perform verification based on all data in the stored credential data and the processed captured facial image data, and information previously stored by the server device 20 (e.g., information related to the My Number card and employee information). For example, the server device 20 determines whether data matching all data in the stored credential data and the processed captured facial image data included in the verification request is included in the information previously stored by the server device 20. For example, the server device 20 determines that the verification was successful if the matching data is included in the information previously stored by the server device 20. For example, the server device 20 determines that the verification was unsuccessful if the matching data is not included in the information previously stored by the server device 20.

[0183] When the verification request includes the digital certificate and processed captured facial image data included in the stored credential data, the server device 20 may perform verification based on the digital certificate and captured facial image data and information previously stored by the server device 20 (e.g., information related to the My Number card and employee information). For example, the server device 20 determines whether data matching the digital certificate and processed captured facial image data of the stored credential data included in the verification request is included in the information previously stored in the server device 20. For example, the server device 20 determines that the verification is successful if the matching data is included in the information previously stored in the server device 20. For example, the server device 20 determines that the verification is unsuccessful if the matching data is not included in the information previously stored in the server device 20.

[0184] Furthermore, the digital certificate acquired by the server device 20 from the terminal device 10 includes information in the stored credential data that has been encrypted using a hash value or a key. Therefore, the server device 20 can acquire all data in the stored credential data by decrypting the information in the digital certificate.

[0185] The server device 20 determines whether the verification of the stored credential data has been successful (step S315).

[0186] If it is determined that the verification in step S314 has failed (No in step S315), the information processing system 5 proceeds to step S304 or step S311. That is, if the verification has failed, the server device 20 transmits verification failure information indicating that the verification has failed to the terminal device 10. In the terminal device 10, the wallet application 111 acquires the verification failure information via the communication unit 13. Then, the wallet application 111 may proceed to step S304 and re-image the user's face. Alternatively, the wallet application 111 may proceed to step S311 and re-process the captured face image data. The terminal device 10 aims to succeed in the verification by performing such re-imaging or re-processing.

[0187] On the other hand, if it is determined that the verification in step S314 is successful (Yes in step S315), the server device 20 generates updated credential data (updated credential data, new credential data) by updating the facial image data of the stored credential data (old credential data) with the processed captured facial image data (step S316). In other words, the server device 20 generates updated credential data in which the processed captured facial image data and the credential data are associated (bound). Expiry date information is attached to the updated credential data. Expiry date information is also attached to the updated, processed captured facial image data.

[0188] The server device 20 updates the held credential data with the generated updated credential data, and stores the updated credential data in the memory of the server device 20 (step S317).

[0189] The server device 20 issues updated credential data with information about the expiration date attached thereto, and transmits this updated credential data to the terminal device 10 (step S318).

[0190] Wallet application 111 of terminal device 10 receives the updated credential data from server device 20 via communication unit 13 and stores it in wallet application 111 (step S319). Wallet application 111 also refers to the expiration date information attached to the updated credential data and manages the expiration date of the updated credential data (step S319).

[0191] The wallet application 111 stores the processed captured face image data as processed face image data within the wallet application 111 or in the memory 12 (step S320). Note that the wallet application 111 may refer to the expiration date information in the update credential data, and delete the stored processed face image data if the current time measured by the wallet application 111 has passed the expiration date.

[0192] Furthermore, in step S301, if the expiration date of the facial image data has not arrived for any of the credential data (No in step S301), in other words, if the expiration date of the facial image data is longer than a predetermined period, the input unit 14 accepts an input from the user to select a desired wallet (step S321). In this case, the wallet application 111 may display a wallet list screen G2 on the display unit 15 and accept an input from the user via the input unit 14.

[0193] 11A and 11B, when the expiration date of the facial image data of each credential data is approaching, the information processing system 5 can generate and issue updated credential data in which the facial image data has been updated using captured facial image data (for example, the latest facial image data) obtained at this timing, and store the updated credential data in the wallet application 111 of the terminal device 10. Note that in this case, the facial image data is updated and a new expiration date is set, but the credential data itself is not updated, and the expiration date of the credential data remains the same as it was previously.

[0194] In steps S310 and S311, the standard of the certificate data may be changed, and the processing conditions for the facial image data may be changed. Therefore, the verifying party (e.g., the server device 20) that verifies the certificate data may process the captured facial image data in accordance with the changeable processing conditions.

[0195] 12A and 12B are sequence diagrams showing an example of the operation of the information processing system 5 when the face matching accuracy is reduced when logging in to the wallet app. In Fig. 12A and 12B, the description of the same processes as those in the operation examples shown in Fig. 9, 10, 11A, or 11B will be omitted or simplified. In Fig. 12A and 12B, the server device 20 is a server device that performs public or private verification or issuance.

[0196] First, input unit 14 of terminal device 10 accepts an input from the user selecting wallet application 111 and sends selection information of wallet application 111 to processor 11 (step S401). Processor 11 acquires the selection information of wallet application 111 and starts wallet application 111 in accordance with this selection information (step S402). Wallet application 111 displays wallet list screen G2 on display unit 15 (step S403).

[0197] The user of the terminal device 10 checks the display of the wallet list screen G2. The input unit 14 accepts an input from the user to select the desired credential data (step S404). Based on the accepted input information, the input unit 14 sends a credential use request to the wallet application 111 requesting the use of the selected credential data (step S405).

[0198] In response to the certificate usage request, the wallet application 111 displays a screen (face matching prompt screen) on the display unit 15 to prompt the user to perform face matching (step S406). The wallet application 111 activates the camera 16 (step S407). The wallet application 111 sends an image capture request to the camera 16 (step S408).

[0199] The camera 16 captures an image of the face of the user of the terminal device 10 (step S409). The wallet application 111 acquires the face image data captured by the camera 16 (captured face image data) from the camera 16 (step S410). The wallet application 111 temporarily stores the captured face image data within the wallet application 111 (step S411).

[0200] The wallet application 111 calls and acquires the face image data included in the selected certificate data (also referred to as face image data in the certificate) (step S412).

[0201] The wallet application 111 performs face matching based on the captured face image data and the face image data in the certificate (step S413). As a result of the face matching, the wallet application 111 determines whether the matching accuracy is equal to or greater than threshold th2 (step S414). The matching accuracy corresponds to, for example, the similarity of the feature amounts between the captured face image data to be compared and the face image data in the certificate, and indicates the likelihood that the person shown in the two face image data is the same person.

[0202] If the matching accuracy is equal to or greater than the threshold th2 (Yes in step S414), the wallet application 111 determines that face matching has been successful (face matching OK). Then, the wallet application 111 logs in to the wallet application 111 (step S415). Then, the wallet application 111 ends the processing of FIGS. 12A and 12B.

[0203] On the other hand, if the matching accuracy is less than the threshold th2 (No in step S414), the wallet application 111 determines that the face matching has failed (face matching NG). Then, the wallet application 111 displays information regarding the update timing of the face image data on the display unit 15 (step S416). In addition, the wallet application 111 displays the menu screen G1 of the wallet application 111 on the display unit 15.

[0204] The user checks the information regarding the update timing. The input unit 14 accepts an input to select (e.g., press) the face image update button BT1 using the menu screen G1 of the wallet application 111 (step S417).

[0205] The wallet application 111 acquires selection information indicating that the facial image update button BT1 has been selected from the input unit 14. The wallet application 111 calls and acquires the captured facial image data that has been temporarily stored (step S418).

[0206] The wallet application 111 determines whether the acquired captured face image data is usable (step S419). For example, if the captured face image data meets the predetermined conditions, such as size, orientation, and brightness, required for the captured face image data of the qualification certificate data, the wallet application 111 may determine that the captured face image data is usable, and if the captured face image data does not meet the above conditions, the wallet application 111 may determine that the captured face image data is unusable.

[0207] If the captured face image data is not available (No in step S419), the wallet application 111 activates the camera 16 (step S420) and sends an image capture request to the camera 16 (step S421).

[0208] The camera 16 captures an image of the face of the user of the terminal device 10 (step S422). The wallet application 111 acquires the face image data captured by the camera 16 (captured face image data) from the camera 16 (step S423).

[0209] The wallet application 111 calls and acquires the face image data in the certificate from the selected certificate data (step S424).

[0210] After step S424, or if the captured face image data is available (Yes in step S419), the wallet application 111 acquires information on processing conditions for the captured face image data. The wallet application 111 processes the captured face image data based on the acquired processing conditions (step S425).

[0211] The wallet application 111 associates the processed captured face image data with the credential data stored in the wallet application 111 (stored credential data) (step S426). The stored credential data in step S412 is the stored credential data for the selected credential data.

[0212] The wallet application 111 transmits a verification request for verifying the stored credential data to the server device 20 via the communication unit 13 (step S427). This verification request may be the same as the verification request shown in FIG. 11B.

[0213] The server device 20 receives the verification request from the terminal device 10. The server device 20 verifies the validity of the stored credential data in accordance with the verification request (step S428). The verification here may be the same as the verification shown in Fig. 11B. The server device 20 determines whether the verification of the stored credential data has been successful (step S429).

[0214] If it is determined that the verification in step S428 has failed (No in step S429), the information processing system 5 proceeds to step S420 or step S425.

[0215] On the other hand, if it is determined that the verification in step S428 is successful (Yes in step S429), the server device 20 updates the facial image data of the stored credential data to the processed captured facial image data, thereby generating updated credential data (updated credential data, new credential data) (step S430). Expiration date information is attached to the updated credential data. In addition, expiration date information is also attached to the updated, processed captured facial image data.

[0216] The server device 20 updates the held credential data with the generated updated credential data, and stores the updated credential data in the memory of the server device 20 (step S431).

[0217] The server device 20 issues updated credential data with expiration date information attached thereto, and transmits this updated credential data to the terminal device 10 (step S432).

[0218] Wallet application 111 of terminal device 10 receives the updated credential data from server device 20 via communication unit 13 and stores it in wallet application 111 (step S433). Wallet application 111 also refers to the expiration date information attached to the updated credential data and manages the expiration date of the updated credential data (step S433).

[0219] The wallet application 111 stores the processed face image data as processed captured face image data within the wallet application 111 or in the memory 12 (step S434). Note that the wallet application 111 may refer to the expiration date information in the update credential data, and delete the stored processed face image data if the current time measured by the wallet application 111 has passed the expiration date.

[0220] That is, the processes in steps S425 to S434 in FIG. 12B are the same as the processes in steps S311 to S320 in FIGS. 11A and 11B.

[0221] 12A and 12B , if the matching accuracy of the face matching performed when logging in to the wallet application 111 has decreased, the information processing system 5 can generate and issue updated credential data in which the face image data has been updated using captured face image data obtained when the login attempt failed, and store the updated credential data in the wallet application 111 of the terminal device 10. Note that in this case, the face image data is updated and a new expiration date is set, but the credential data itself is not updated, and the expiration date of the credential data remains the same as it was previously.

[0222] Note that, although Figures 12A and 12B illustrate an example in which the determination of whether the facial matching accuracy has decreased is performed when logging in to the wallet application 111, this is not limited to this and may be performed at other times.

[0223] 13A and 13B are diagrams showing an example of the operation of the information processing system 5 when an expiration date alert for credential data is issued. In Fig. 13A and 13B, the description of the same processes as those in the operation examples shown in Fig. 9, Fig. 10, Fig. 11A, Fig. 11B, Fig. 12A and Fig. 12B will be omitted or simplified.

[0224] 13A and 13B assume that the expiration date of the certificate data arrives after the facial image data of the certificate data is updated in FIGS. 12A and 12B. It is assumed that, at the start of the processing in FIGS. 13A and 13B, the wallet application 111 of the terminal device 10 has already stored processed facial image data. This facial image data is, for example, the processed facial image data stored in step S434 of FIG. 12B and is based on facial image data captured within a predetermined period (e.g., within six months). In FIGS. 13A and 13B, the server device 20 is a server device that performs public or private verification and issuance.

[0225] First, input unit 14 of terminal device 10 accepts an input from a user to select wallet application 111, and sends selection information of wallet application 111 to processor 11 (step S501). Processor 11 acquires the selection information of wallet application 111, and starts wallet application 111 in accordance with this selection information (step S502).

[0226] The wallet application 111 displays the wallet list screen G2 on the display unit 15 (step S503). The wallet application 111 receives input from the user via the input unit 14 and specifies the desired credential. The wallet application 111 displays at least a portion of the information contained in the stored credential data (also referred to as target credential data) for the specified credential on the display unit 15 (step S504).

[0227] The wallet application 111 references the expiration date information of the credential data included in the target credential data, measures the current time, and determines whether the expiration date will arrive. In other words, the wallet application 111 determines whether the period from the current time to the expiration date of the target credential data is equal to or less than threshold value th3 (for example, within six months). If the expiration date will arrive, that is, if the period is equal to or less than threshold value th3, the wallet application 111 displays alert information indicating the expiration date is approaching on the display unit 15 (step S504).

[0228] The user checks the displayed target credential data and expiration date alert information. The display unit 15 displays the menu screen G1 of the wallet application 111. The input unit 14 accepts the user's pressing of the face image update button BT1 (step S505). For example, if the data contained in the target credential data (e.g., name, address, employee number) has not been changed, the user presses the face image update button BT1 using the input unit 14.

[0229] The wallet application 111 retrieves and acquires the processed captured face image data that is associated with the target credential data and stored (step S506).

[0230] The wallet application 111 determines whether the acquired processed face image data is within the expiration date of the face image data (step S507). In this case, for example, the wallet application 111 determines whether the current time is before the expiration date of the face image data, that is, whether the current time is before the expiration date.

[0231] If the facial image data is not within the expiration date (No in step S507), that is, if the current time has passed the expiration date of the facial image data, the wallet application 111 causes the camera 16 to recapture the facial image and associates the recaptured, processed facial image data for the target credential with the target stored credential data (step S508). Specifically, the wallet application 111 deletes the stored facial image data whose expiration date has passed. The wallet application 111 activates the camera 16 and sends an image capture request to the camera 16. In response to the image capture request, the camera 16 captures (recaptures) the face of the user of the terminal device 10. The wallet application 111 acquires the facial image data (captured facial image data) captured (recaptured) by the camera 16 from the camera 16. In other words, the terminal device 10 performs the same processes as, for example, steps S420 to S423 described above. The wallet application 111 then compares the acquired captured facial image data with the facial image data contained in the target credential data. If the match is successful, the wallet application 111 processes the captured facial image data based on the processing conditions and associates the processed facial image data (i.e., processed facial image data for the target certificate) with the target stored certificate data.

[0232] On the other hand, if the facial image data is within the expiration date (Yes in step S507), the wallet application 111 associates the acquired processed facial image data (i.e., processed facial image data for the target certificate) with the stored target certificate data (step S509).

[0233] The wallet application 111 transmits a verification request to verify the stored credential data of interest to the server device 20 via the communication unit 13 (step S510). The verification request here may be the same as the verification requests shown in FIGS. 11B and 12B.

[0234] The server device 20 receives a verification request from the terminal device 10. The server device 20 verifies the validity of the stored credential data in accordance with the verification request (step S511). The server device 20 determines whether the verification is successful (step S512). The verification here may be similar to the verification shown in Figures 11B and 12B.

[0235] For example, the stored credential data in steps S427 and S428 in Fig. 12B is the stored credential data of the target in steps S510 and S511 in Fig. 13B. For example, the processed captured face image data in steps S427 and S428 in Fig. 12B is the processed face image data for the target credential in steps S510 and S511 in Fig. 13B.

[0236] If it is determined that the verification in step S511 has failed (No in step S512), the server device 20 ends the processing in FIGS. 13A and 13B.

[0237] On the other hand, if it is determined that the verification in step S511 is successful (Yes in step S512), the server device 20 updates the facial image data of the target stored credential data to processed facial image data for the target credential, thereby generating updated credential data (updated credential data, new credential data) (step S513). The updated credential data is provided with expiration date information.

[0238] The server device 20 updates the held target credential data to the generated target updated credential data, and stores the target updated credential data in the memory of the server device 20 (step S514).

[0239] The server device 20 issues the target update credential data with expiration date information attached thereto, and transmits this target update credential data to the terminal device 10 (step S515).

[0240] Wallet application 111 of terminal device 10 receives the target update credential data from server device 20 via communication unit 13 and stores it in wallet application 111 (step S516). Wallet application 111 also references the expiration date information attached to the target update credential data and manages the expiration date of the target update credential data (step S516).

[0241] After the facial image update button BT1 is pressed in step S505, the input unit 14 accepts input from the user to select a desired wallet (step S517). In this case, the wallet application 111 may display the wallet list screen G2 on the display unit 15 and accept input from the user via the input unit 14. In other words, the information processing system 5 can update the credential data to include facial image data as necessary based on the expiration date of the credential data and issue updated credential data, and also allow the user to select the wallet they want to use.

[0242] Note that this is the same as the processing in steps S510 to S517 in FIG. 13B, the processing in steps S427 to S434 in FIG. 12B, and the processing in steps S313 to S320 in FIGS. 11A and 11B.

[0243] 13A and 13B, when the expiration date of the credential data is approaching, the information processing system 5 generates and issues updated credential data including processed facial image data that was registered (stored) before the expiration date, and stores the updated credential data in the wallet application 111 of the terminal device 10. Here, the facial image data has the most recently updated and set expiration date, and the credential data itself is updated and a new expiration date is set.

[0244] Therefore, when the information processing system 5 updates the facial image data of the certificate data (e.g., an electronic My Number card) and the content of the certificate data itself within a specified period of time, the updated facial image data can be used as the facial image data of the certificate data after the content itself has been updated.

[0245] As described above, the information processing system 5 of this embodiment, like the first embodiment, can improve the accuracy of identity verification using facial image data of identity verification certificate data that electronically certifies the identity of a user. Furthermore, the information processing system 5 can issue updated certificate data whose authenticity is verified by the server device 20 and in which the facial image data has been updated, and can store this updated certificate data in the terminal device 10. Therefore, the user can easily use the terminal device 10 to use updated certificate data that includes facial image data with high facial matching accuracy.

[0246] In the present embodiment, the captured facial image data is processed in each operation example, but this is not limiting. For example, if facial image data in a desired state is obtained as captured facial image data from the beginning, the captured facial image data can be used as facial image data for renewal certificate data without being processed.

[0247] (Summary of the embodiment) As described above, the present disclosure describes at least the following matters. Note that, in parentheses, examples of components corresponding to the above-described embodiment are shown, but the present disclosure is not limited to these.

[0248] (Item 1) A terminal device (terminal device 10) comprising a processor (processor 11) and a camera (camera 16), wherein the processor: acquires first facial image data (captured facial image data) obtained by capturing an image of a user's face with the camera; acquires second facial image data (registered facial image data) that has been registered in advance, or third facial image data (stored proof facial image data) that is stored in personal identification certificate data that proves the user is the user; compares the first facial image data with the second facial image data or the third facial image data; and if the comparison is successful, updates the third facial image data based on the first facial image data.

[0249] This allows the terminal device to change the facial image used to compare the face of the user of the certificate data in real space at a predetermined timing. The updated facial image data is used to compare faces when using the certificate data that stores this updated facial image data. Therefore, even if the characteristics of the person certified by the certificate data have changed, appropriate face comparison using the facial image can be achieved when using the certificate data, improving the accuracy of identity verification.

[0250] (Item 2) The terminal device according to item 1, further comprising a memory, wherein the memory holds a digital wallet, and the digital wallet stores at least one piece of personal identification certificate data.

[0251] This allows the terminal device to improve the accuracy of personal identification using facial image data of each personal identification certificate data while retaining various personal identification certificate data in the digital wallet.

[0252] (Item 3) The terminal device according to Item 2, wherein the processor performs the verification when logging in to the terminal device or when launching an application for the digital wallet.

[0253] As a result, since the terminal device frequently logs in to the terminal device and launches the digital wallet application, face matching is also performed frequently, and the frequency of updating face image data can be increased. Therefore, the terminal device can perform face matching using updated face image data that more closely reflects facial features in real space.

[0254] (Item 4) The terminal device according to item 2 or 3, wherein the camera captures an image of the user's face when logging in to the terminal device or launching the digital wallet application, and the processor acquires the first facial image data of the user's face from the camera, and stores the first facial image data in the memory as the second facial image data.

[0255] This allows the terminal device to frequently register the second facial image data because logins to the terminal device and launching of digital wallet applications are frequently performed, and therefore the terminal device can quickly update the third facial image data using the second facial image data that more closely reflects facial features in real space.

[0256] (Item 5) The terminal device according to any one of items 1 to 4, wherein the processor processes the first facial image data based on an update condition of the personal identification certificate data to be updated, and updates the third facial image data with the processed first facial image data.

[0257] This allows the terminal device to store the facial image data (first facial image data) that is updated and stored in the certificate data in a state that matches the update conditions such as the format of each certificate data, the imaging conditions, etc. Therefore, the terminal device can easily use the certificate data that stores the updated facial image data.

[0258] (Item 6) An information processing system (information processing system 5) comprising a terminal device and a server device (server device 20), wherein the terminal device acquires first facial image data obtained by capturing an image of a user's face with a camera, acquires pre-registered second facial image data or third facial image data stored in personal identification certificate data that proves the user is the user, compares the first facial image data with the second facial image data or the third facial image data, and if the comparison is successful, updates the third facial image data based on the first facial image data.

[0259] As a result, the information processing system can achieve the same effect as item 1.

[0260] (Item 7) The information processing system described in Item 6, wherein the terminal device acquires the personal identification certificate data (credentials data), associates the first facial image data with the personal identification certificate data, and transmits the associated data to the server device, the server device receives the associated first facial image data and the personal identification certificate data, verifies the validity of the personal identification certificate data, and if the verification of the validity of the personal identification certificate data is successful, updates the third facial image data held in the personal identification certificate data with the first facial image data, thereby generating updated certificate data (updated credential data) in which the personal identification certificate data is updated, and transmits the updated certificate data to the terminal device, and the terminal device receives and holds the updated certificate data.

[0261] As a result, the information processing system can issue updated certificate data whose authenticity is verified by the server device and whose facial image data has been updated, and can store this updated certificate data in the terminal device, allowing users to easily use the updated certificate data that has facial image data with high facial matching accuracy using the terminal device.

[0262] (Item 8) The information processing system described in Item 7, wherein the terminal device acquires information on processing conditions for the facial image data, processes the first facial image data based on the processing conditions, associates the processed first facial image data with the personal identification certificate data and transmits them to the server device, and stores the processed first facial image data.

[0263] This allows the information processing system to issue updated certificate data using the first facial image data that has been subjected to a predetermined process. The information processing system can also store the processed facial image data used at this time, making the processed facial image data available for use at a later time.

[0264] (Item 9) The information processing system according to Item 7 or 8, wherein the terminal device measures the current time, and if the period from the current time to the expiration date of the third facial image data is equal to or less than a first threshold (threshold th1), associates the first facial image data with the personal identification certificate data and transmits them to the server device.

[0265] As a result, when the expiration date of the facial image data of the personal identification certificate data is approaching, the information processing system can generate and issue personal identification certificate data in which the third facial image data is updated using the first facial image data (e.g., the latest facial image data) obtained at this time, and store it in the terminal device.

[0266] (Item 10) The information processing system according to Item 7 or 8, wherein the terminal device performs the matching between the first facial image data and the second facial image data, and if the matching accuracy obtained as a result of the matching is equal to or less than a second threshold (threshold th2), associates the first facial image data with the personal identification certificate data and transmits them to the server device.

[0267] As a result, if the accuracy of facial matching decreases, for example, when logging in to a specified application, the information processing system can generate and issue identity verification certificate data in which the third facial image data is updated using the first facial image data obtained when the login fails, and store the data in the wallet application 111 of the terminal device 10.

[0268] (Item 11) The information processing system according to Item 9, wherein the terminal device acquires the stored processed first facial image data when the period from the current time to the expiration date of the personal identification certificate data is equal to or less than a third threshold, and when the current time is before the expiration date of the processed first facial image data, associates the processed first facial image data with the personal identification certificate data and transmits them to the server device.

[0269] As a result, when the expiration date of the identification certificate data is approaching, the information processing system can generate and issue updated identification certificate data using the registered (stored) processed face image data and store it in the terminal device, thereby reducing the effort required to newly capture an image of the user's face.

[0270] (Item 12) A data update method comprising: acquiring first facial image data obtained by capturing an image of a user's face with a camera; acquiring pre-registered second facial image data or third facial image data stored in personal identification certificate data that proves the user is the user; comparing the first facial image data with the second facial image data or the third facial image data; and, if the comparison is successful, updating the third facial image data based on the first facial image data.

[0271] As a result, the data update method can achieve the same effect as item 1.

[0272] (Item 13) A program that causes a computer to execute the data update method according to Item 12.

[0273] This allows the program to achieve the same effect as item 1.

[0274] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0275] In addition, the above embodiment may also be applicable to a program that realizes the functions of the data update method, which is supplied to a computer (e.g., terminal device 10) via a network or various storage media, and which is read and executed by the processor of this computer, as well as to the storage media on which this program is stored.

[0276] This disclosure is based on a Japanese patent application filed on June 28, 2024 (application number: Patent application 2024-105297) and a Japanese patent application filed on February 21, 2025 (application number: Patent application 2025-027303), the contents of which are incorporated by reference into this disclosure.

[0277] The present disclosure is useful for a terminal device, an information processing system, a data update method, a program, etc. that can improve the accuracy of identity verification using facial image data of identity verification certificate data that electronically certifies the identity of a person.

[0278] 5 Information processing system 10 Terminal device 11 Processor 12 Memory 13 Communication unit 14 Input unit 15 Display unit 16 Camera 20 Server device 21 Processor 21 22 Memory 23 Communication unit 111 Wallet application 211 Verification unit 212 Certificate generation unit KG Facial image data G1 Menu screen G2 Wallet list screen SM Certificate data RI Update reminder information

Claims

1. A terminal device comprising a processor and a camera, wherein the processor: acquires first facial image data obtained by capturing an image of a user's face with the camera; acquires pre-registered second facial image data or third facial image data stored in personal identification certificate data that proves the user is the user; compares the first facial image data with the second facial image data or the third facial image data; and if the comparison is successful, updates the third facial image data based on the first facial image data.

2. The terminal device according to claim 1, further comprising a memory, said memory holding a digital wallet, said digital wallet storing at least one identity verification certificate data.

3. The terminal device according to claim 2, wherein the processor performs the verification when logging in to the terminal device or when launching the digital wallet application.

4. The terminal device according to claim 2, wherein the camera captures an image of the user's face when logging in to the terminal device or launching the digital wallet application, and the processor acquires the first facial image data of the user's face from the camera, and stores the first facial image data in the memory as the second facial image data.

5. The terminal device according to claim 1 or 2, wherein the processor processes the first facial image data based on the update conditions of the personal identification certificate data to be updated, and updates the third facial image data with the processed first facial image data.

6. An information processing system comprising a terminal device and a server device, wherein the terminal device acquires first facial image data obtained by capturing an image of a user's face with a camera, acquires pre-registered second facial image data or third facial image data stored in personal identification certificate data that proves the user is the user, compares the first facial image data with the second facial image data or the third facial image data, and if the comparison is successful, updates the third facial image data based on the first facial image data.

7. The information processing system according to claim 6, wherein the terminal device acquires the personal identification certificate data, associates the first facial image data with the personal identification certificate data, and transmits the associated data to the server device; the server device receives the associated first facial image data and the personal identification certificate data, verifies the authenticity of the personal identification certificate data, and if the authenticity of the personal identification certificate data is successfully verified, updates the third facial image data held in the personal identification certificate data with the first facial image data, thereby generating updated certificate data in which the personal identification certificate data is updated, and transmits the updated certificate data to the terminal device; and the terminal device receives and stores the updated certificate data.

8. The information processing system according to claim 7, wherein the terminal device acquires information on processing conditions for the facial image data, processes the first facial image data based on the processing conditions, associates the processed first facial image data with the personal identification certificate data and transmits them to the server device, and stores the processed first facial image data.

9. The information processing system of claim 7, wherein the terminal device measures the current time, and if the period from the current time to the expiration date of the third facial image data is less than or equal to a first threshold, associates the first facial image data with the personal identification certificate data and transmits them to the server device.

10. The information processing system according to claim 7, wherein the terminal device performs the matching between the first facial image data and the second facial image data, and if the matching accuracy obtained as a result of the matching is equal to or less than a second threshold, associates the first facial image data with the personal identification certificate data and transmits them to the server device.

11. The information processing system of claim 9, wherein the terminal device acquires the stored processed first facial image data when the period from the current time to the expiration date of the personal identification certificate data is equal to or less than a third threshold, and when the current time is before the expiration date of the processed first facial image data, associates the processed first facial image data with the personal identification certificate data and transmits them to the server device.

12. A data update method comprising: acquiring first facial image data obtained by capturing an image of a user's face with a camera; acquiring pre-registered second facial image data or third facial image data stored in personal identification certificate data that proves the user is the user; comparing the first facial image data with the second facial image data or the third facial image data; and, if the comparison is successful, updating the third facial image data based on the first facial image data.

13. A program that causes a computer to execute the data updating method according to claim 12.

Images