System, program, and authentication method

Abstract

SOLUTION: To provide a system which includes a storage unit for storing user data including a plurality of feature points of the face of a user, an authentication request acquisition unit for acquiring an authentication request including data of an avatar of a user as an authentication object generated using the plurality of feature points specified by analyzing the part of the face of the user in a user image generated by imaging the user, and an authentication unit for comparing the data of the avatar included in the authentication request with the user data stored in the storage unit, and thereby executing authentication of the user of the authentication object.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a system, a program, and an authentication method. [Background technology]

[0002] Patent Document 1 describes a technique for generating an avatar of a person based on depth data and image data of the person. [Prior art document] [Patent documents] [Patent Document 1] JP 2023-094549 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided a system. The system may include a storage unit that stores user data including a plurality of feature points of a user's face. The system may include an authentication request acquisition unit that acquires an authentication request including avatar data of a user to be authenticated, the avatar data being generated using a plurality of feature points identified by analyzing portions of the user's face in a user image generated by capturing an image of the user. The system may also include an authentication unit that performs authentication of the user to be authenticated by comparing the avatar data included in the authentication request with the user data stored in the storage unit.

[0004] In the system, the authentication request acquisition unit may acquire the authentication request including data of the avatar located in a metaverse space, and the authentication unit may authenticate the user in the metaverse space by comparing a plurality of feature points of the avatar included in the authentication request with the plurality of feature points included in the user data stored in the user data storage unit. The authentication unit may periodically or irregularly authenticate the user corresponding to the avatar while the avatar is located in the metaverse space, and if authentication fails, may control to prohibit the avatar from using the metaverse space.

[0005] In any of the above systems, the user data may further include gesture data indicating body movements of the user, the authentication request acquisition unit may acquire the authentication request further including gesture data indicating body movements of the avatar, and the authentication unit may authenticate the user to be authenticated by comparing the plurality of feature points of the avatar's face and the gesture data of the avatar included in the authentication request with the plurality of feature points of the user's face and the gesture data of the user included in the user data. The authentication request acquisition unit may acquire the authentication request including the gesture data indicating body movements of the avatar performed based on body movements of the user identified by analyzing continuously captured user images of the user.

[0006] Any of the systems may further include a gesture data generation unit that identifies characteristic body movements of the user by analyzing continuously captured user images of the user, and generates the gesture data indicating the identified movements.

[0007] Any of the systems may further include a behavior pattern generation unit that generates a behavior pattern of the user by analyzing the behavior of the user, the memory unit may store the user data of the user including the behavior pattern generated by the behavior pattern generation unit, the authentication request acquisition unit may acquire the authentication request including avatar data of the user to be authenticated and the behavior pattern of the user, and the authentication unit may authenticate the user to be authenticated by comparing a plurality of feature points and the behavior pattern of the avatar included in the authentication request with the plurality of feature points and the behavior pattern included in the user data stored in the memory unit.

[0008] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the above-described system.

[0009] According to one embodiment of the present invention, there is provided an authentication method executed by a computer. The authentication method may include a person data storing step of storing person data including a plurality of feature points of a person's face in a person data storage unit. The authentication method may include an authentication request acquiring step of acquiring an authentication request including avatar data of a person to be authenticated, the avatar data being generated using a plurality of feature points identified by analyzing portions of the person's face in a person image generated by capturing an image of the person's face. The authentication method may also include an authentication step of authenticating the person to be authenticated by comparing the plurality of feature points of the avatar included in the authentication request with the plurality of feature points included in the person data stored in the person data storage unit.

[0010] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0011] [Figure 1] An example of a system 10 is shown schematically. [Figure 2] 2 shows an example of a functional configuration of the authentication server 100. [Figure 3] 2 shows an example of a functional configuration of a management server 300. [Figure 4] Some examples of how services using the system 10 are used are shown in outline. [Figure 5] Another example of the system 10 is shown schematically. [Figure 6] An example of the hardware configuration of a computer 1200 that functions as the authentication server 100, the communication terminal 200, the management server 300, or the communication device 400 is shown in schematic form. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0013] An environment in which various activities can be performed in the metaverse space is realized using an avatar generated using the user's facial features. For example, in the metaverse space, the avatar can receive specific services or enter specific areas after user authentication, thereby further activating the metaverse space. System 10 according to this embodiment focuses on the fact that a user's avatar is generated based on the user's facial features, and authenticates the user using the features extracted from the avatar. This makes it possible, for example, to securely use various services in the metaverse space by authenticating using the avatar.

[0014] 1 illustrates an example of a system 10. The system 10 includes an authentication server 100. The system 10 may include a communication terminal 200. The system 10 may include a management server 300. The system 10 may include a communication device 400.

[0015] The authentication server 100, the communication terminal 200, the management server 300, and the communication device 400 may communicate via a network 20. The network 20 may include the Internet. The network 20 may include a LAN (Local Area Network). The network 20 may include a mobile communication network. The mobile communication network may conform to any of the following communication methods: LTE (Long Term Evolution), 5G (5th Generation), 3G (3rd Generation), and 6G (6th Generation) or later.

[0016] The authentication server 100 has a function of performing face authentication of the user 40. The authentication server 100 stores in advance user data including a plurality of feature points of the face of the user 40, and performs authentication of the user 40 using the user data.

[0017] For example, the authentication server 100 receives and stores user data including multiple feature points of the face of the user 40 from the communication terminal 200 of the user 40. The communication terminal 200 may generate the user data by analyzing the facial portion of the user 40 in the user image generated by the imaging unit 202 capturing an image of the user 40. The authentication server 100 may also generate the user data. The authentication server 100 may receive the user image from the communication terminal 200 and analyze the user image to generate the user data.

[0018] The imaging unit 202 may include an imaging camera. The imaging camera may generate a captured image of the user 40. The captured image may be an RGB image. The user image may include a captured image. The imaging unit 202 may further include a depth camera. The depth camera is a camera capable of generating a depth information image including depth information, which is the distance to an imaging target. The depth information image includes depth information for each pixel. The depth camera is sometimes called a depth camera. A known technique can be used to measure the distance, and for example, LiDAR (Light Detection and Ranging), triangulation, or TOF (Time of Flight) may be used. The depth information image generated by capturing an image of the user 40 may be an image in which the depth (three-dimensional shape) of the user 40 is represented by discrete point cloud data. The user image may include the depth information image. The imaging unit 202 may include a stereo camera. The stereo camera may generate a captured image and a depth information image.

[0019] The communication terminal 200 may be a smartphone, a tablet terminal, a PC (Personal Computer), a head-mounted display, a motion capture system, or the like.

[0020] The authentication server 100 may receive and store user data including multiple feature points of the face of the user 40 from the communication device 400. The communication device 400 may be a device installed in any location. The communication device 400 may generate the user data by analyzing a user image generated by the imaging unit 402 capturing an image of the user 40. The authentication server 100 may generate the user data. The authentication server 100 may receive the user image from the communication device 400 and generate the user data by analyzing the user image.

[0021] The imaging unit 402 may include an imaging camera. The imaging camera may generate a captured image of the user 40. The captured image may be an RGB image. The user image may include a captured image. The imaging unit 402 may further include a depth camera. The depth camera is a camera capable of generating a depth information image including depth information, which is the distance to an imaging target. The depth information image includes depth information for each pixel. The depth camera is sometimes called a depth camera. A known technique may be used to measure the distance, and for example, LiDAR, triangulation, or TOF may be used. The depth information image generated by capturing an image of the user 40 may be an image in which the depth (three-dimensional shape) of the user 40 is expressed as discrete point cloud data. The user image may include the depth information image. The imaging unit 402 may include a stereo camera. The stereo camera may generate a captured image and a depth information image.

[0022] The communication device 400 may be a smartphone, a tablet terminal, a PC, or the like.

[0023] The management server 300 manages the avatar 42 of the user 40. The management server 300 may receive the avatar 42 generated by the communication terminal 200 from the communication terminal 200. The communication terminal 200 may identify multiple feature points of the user 40's face by analyzing the facial portion of the user 40 in a user image generated by the imaging unit 202 capturing an image of the user 40, and generate the avatar 42 using the identified multiple feature points. The communication terminal 200 generates the avatar 42 by, for example, generating texture data of the user 40's face using the captured image, generating a 3D model of the user 40's face using the multiple feature points of the user 40's face, and pasting the texture data onto the 3D model. The avatar 42 includes at least a face. The avatar 42 may include the entire body. If the avatar 42 includes the entire body, the communication terminal 200 may identify multiple feature points of the user 40's body by analyzing parts of the user's 40's body in the user image, and generate the avatar 42 using the identified multiple feature points.

[0024] Management server 300 may provide a metaverse space. User 40 may access management server 300 using communication terminal 200 and use the metaverse space provided by management server 300. User 40 may perform various activities in the metaverse space by operating avatar 42 of user 40 arranged in the metaverse space.

[0025] There are cases where it is desired to authenticate user 40 within the metaverse space. For example, there are cases where it is desired to provide a service to user 40 only if authentication of user 40 within the metaverse space is successful. In such cases, it is conceivable to perform facial authentication by capturing an image of the face of user 40 using imaging unit 202 and transmitting the user image via network 20, but transmitting the user image via network 20 every time authentication is performed is not desirable from a security standpoint. Therefore, in system 10 according to this embodiment, authentication of user 40 is performed using avatar 42 of user 40.

[0026] For example, when authenticating a user 40, the management server 300 sends an authentication request including an avatar 42 of the user 40 to the authentication server 100. The authentication server 100 compares the data of the avatar 42 included in the authentication request with the user data of multiple users 40 stored therein, thereby authenticating the user 40 to be authenticated.

[0027] For example, the authentication server 100 calculates the distance between each of the user data of multiple users 40 stored and the data of the avatar 42, and if there is user data for which the calculated distance is shorter than a predetermined threshold, the user 40 to be authenticated is the user 40 corresponding to the user data with the shortest distance, and the authentication is successful; if there is no such user data, the authentication is unsuccessful.

[0028] The avatar 42 according to this embodiment is generated using multiple feature points on the face of the user 40 and contains information on the multiple feature points, so that facial authentication of the user 40 can be achieved by using the data of the avatar 42. According to the system 10 according to this embodiment, facial authentication of the user 40 can be achieved without transmitting a facial image of the user 40 over the network 20, which can contribute to improved security.

[0029] In the example shown in FIG. 1 , the system 10 may not include the authentication server 100. In this case, the communication terminal 200, rather than the authentication server 100, manages the user data of the user 40, and the communication terminal 200 authenticates the user 40. For example, the communication terminal 200 may have a function of performing face authentication of the user 40. The communication terminal 200 stores user data including multiple feature points of the face of the user 40 in advance, and authenticates the user 40 using the user data. The system 10 may be the same as the example described above, except that it may not include the authentication server 100. For example, the method by which the communication terminal 200 generates user data of the user 40 and the method by which the communication terminal 200 generates the avatar 42 of the user 40 may be the same as the example described above.

[0030] When authenticating a user 40 in the metaverse space, for example, the management server 300 sends an authentication request including the avatar 42 of the user 40 to the communication terminal 200, rather than the authentication server 100. The communication terminal 200 authenticates the user 40 to be authenticated by comparing the data of the avatar 42 included in the authentication request with stored user data of the user 40 to be authenticated. For example, the communication terminal 200 calculates the distance between the stored user data of the user 40 to be authenticated and the data of the avatar 42. If the calculated distance is shorter than a predetermined threshold, the authentication is successful. If the calculated distance is longer than the predetermined threshold, the authentication is unsuccessful. The communication terminal 200 sends an authentication result to the management server 300. If the management server 300 receives an authentication result indicating successful authentication from the communication terminal 200, the management server 300 determines that the authentication of the user 40 in the metaverse space has been successful. For example, in a situation where it is necessary to determine whether or not user 40's avatar 42 can use the metaverse space, management server 300 allows user 40 to use the metaverse space with avatar 42 in response to receiving an authentication result indicating successful authentication from communication terminal 200.

[0031] 2 shows an example of the functional configuration of the authentication server 100. The authentication server 100 includes a storage unit 102, a registration unit 104, an authentication request acquisition unit 106, an authentication unit 108, an authentication result output unit 110, a gesture data generation unit 112, a data collection unit 114, and a behavior pattern generation unit 116. Note that it is not essential for the authentication server 100 to include all of these units.

[0032] The registration unit 104 registers user data including a plurality of feature points of the face of the user 40. For example, the registration unit 104 registers user data received from the communication terminal 200. For example, the registration unit 104 generates and registers user data by analyzing a user image received from the communication terminal 200. For example, the registration unit 104 registers user data received from the communication device 400. For example, the registration unit 104 generates and registers user data by analyzing a user image received from the communication device 400. The registration unit 104 stores the registered user data in the storage unit 102.

[0033] The authentication request acquisition unit 106 acquires an authentication request including data of the avatar 42 of the user 40. For example, the authentication request acquisition unit 106 receives an authentication request from the management server 300. As a specific example, the authentication request acquisition unit 106 acquires, from the management server 300, an authentication request including data of the avatar 42 located in the metaverse space provided by the management server 300.

[0034] When the authentication request acquisition unit 106 acquires an authentication request, the authentication unit 108 authenticates the user 40 to be authenticated. The authentication unit 108 compares the data of the avatar 42 included in the authentication request with the user data stored in the storage unit 102, thereby authenticating the user 40 to be authenticated.

[0035] The authentication unit 108 calculates the distance between each of the user data of multiple users 40 stored in the memory unit 102 and the data of the avatar 42, and if there is user data for which the calculated distance is shorter than a predetermined threshold, the authentication is deemed successful as the user 40 to be authenticated is the user 40 corresponding to the user data with the shortest distance, and if there is no such user data, the authentication is deemed unsuccessful.

[0036] The authentication unit 108 may calculate the distance between the feature amounts of multiple feature points included in the user data and the feature amounts of multiple feature points of the avatar 42 as the distance between the user data and the data of the avatar 42. For example, the authentication unit 108 generates vector data from the multiple feature points included in the user data, generates vector data from the multiple feature points of the avatar 42, and calculates the distance between the vector data. The authentication unit 108 may calculate the distance by determining the degree of similarity between the multiple feature points included in the user data and the multiple feature points included in the avatar 42. The authentication unit 108 may calculate the distance between the user data and the data of the avatar 42 using other methods.

[0037] When the authentication request acquisition unit 106 acquires an authentication request including data of an avatar 42 located in the metaverse space provided by the management server 300, the authentication unit 108 may authenticate the user 40 in the metaverse space by comparing the user data stored in the storage unit 102 with the data of the avatar 42 included in the authentication request. The authentication unit 108 may authenticate the user 40 corresponding to the avatar 42 periodically or irregularly while the avatar 42 is located in the metaverse space, and may control the avatar 42 to be prohibited from using the metaverse space if authentication fails. For example, if authentication fails, the authentication unit 108 transmits instruction data to the management server 300 to prohibit the avatar 42 from using the metaverse space. This allows unauthorized users 40 to be quickly detected and their use to be stopped.

[0038] The authentication result output unit 110 outputs the authentication result obtained by the authentication unit 108. For example, the authentication result output unit 110 transmits the authentication result obtained by the authentication unit 108 to the management server 300. For example, the authentication result output unit 110 causes the authentication result obtained by the authentication unit 108 to be displayed on a display provided in the authentication server 100.

[0039] The authentication server 100 may authenticate the user 40 using other data in addition to the avatar 42. For example, the authentication server 100 authenticates the user 40 using gesture data indicating the body movements of the user 40 in addition to the avatar 42.

[0040] The registration unit 104 may register user data that further includes gesture data of the user 40 in addition to a plurality of feature points on the face of the user 40.

[0041] The registration unit 104 receives and registers gesture data of the user 40, for example, from the communication terminal 200 of the user 40. The registration unit 104 stores the registered gesture data in the storage unit 102. The communication terminal 200 may generate the gesture data of the user 40 by analyzing a user image generated by the imaging unit 202 continuously capturing images of the user 40.

[0042] The registration unit 104 receives and stores gesture data of the user 40, for example, from the communication device 400. The communication device 400 may generate the gesture data of the user 40 by analyzing user images generated by the imaging unit 402 continuously capturing images of the user 40.

[0043] The gesture data generation unit 112 generates gesture data of the user 40. The gesture data generation unit 112 may generate the gesture data of the user 40 by analyzing user images generated by continuously capturing images of the user 40 and received from the communication terminal 200. The gesture data generation unit 112 may generate the gesture data of the user 40 by analyzing user images generated by continuously capturing images of the user 40 and received from the communication device 400. The registration unit 104 may register the gesture data of the user 40 generated by the gesture data generation unit 112.

[0044] The authentication request acquisition unit 106 may acquire an authentication request that further includes gesture data indicating a body movement of the avatar 42. For example, the authentication request acquisition unit 106 receives, from the management server 300, an authentication request that further includes gesture data indicating a body movement of the avatar 42.

[0045] For example, the management server 300 receives from the communication terminal 200 data on the body movements of the user 40, which data is generated by the communication terminal 200 analyzing the user images of the user 40 who is making a gesture, using the imaging unit 202, and uses the data to move the avatar 42. The management server 300 may include in the authentication request gesture data indicating the body movements of the avatar 42 that were performed based on the body movements of the user 40 identified by analyzing the user images of the user 40 that were continuously captured in this manner.

[0046] Furthermore, for example, the management server 300 receives a user image from the communication terminal 200, which captures an image of the user 40 performing a gesture using the imaging unit 202, and uses data on the body movements of the user 40 generated by analyzing the user image to move the avatar 42. The management server 300 may include, in the authentication request, gesture data indicating the body movements of the avatar 42 performed based on the body movements of the user 40 identified by analyzing the user images of the user 40 captured continuously in this manner.

[0047] The authentication unit 108 may authenticate the user 40 to be authenticated by comparing a plurality of facial feature points and gesture data of the avatar 42 included in the authentication request with a plurality of feature points and gesture data included in the user data stored in the storage unit 102. For example, the authentication unit 108 executes both an authentication process in which the plurality of facial feature points of the avatar 42 included in the authentication request are compared with a plurality of feature points included in the user data stored in the storage unit 102, and an authentication process in which the gesture data included in the authentication request is compared with the gesture data included in the user data stored in the storage unit 102. If both results indicate successful authentication, the authentication is deemed successful, and if at least one result indicates unsuccessful authentication, the authentication is deemed unsuccessful.

[0048] The authentication process using the gesture data included in the authentication request and the gesture data included in the user data may be such that authentication is successful when it is determined that the body movements indicated by the respective gesture data match, and authentication is unsuccessful when it is determined that they do not match. The authentication process using the gesture data included in the authentication request and the gesture data included in the user data may calculate a similarity between these gesture data, and determine authentication as successful when the calculated similarity is higher than a predetermined threshold, and as unsuccessful when it is lower than the threshold. Methods other than these may also be used for the authentication process using the gesture data included in the authentication request and the gesture data included in the user data.

[0049] The data collection unit 114 collects various data relating to the user 40 .

[0050] The data collection unit 114 collects, for example, data on the physical movements of the user 40. For example, the data collection unit 114 receives from the communication terminal 200 data on the daily movements of the user 40 that is generated appropriately by the communication terminal 200. The communication terminal 200 generates data on the movements of the user 40 by, for example, continuously capturing images of the user 40 using the imaging unit 202 in accordance with instructions from the user 40 and analyzing the user images. If the communication terminal 200 has a function for recording a life log of the user 40, the data on the movements of the user 40 may be generated by analyzing data from the life log. The data collection unit 114 stores the collected data in the storage unit 102.

[0051] The gesture data generation unit 112 may generate gesture data of the user 40 from data of the movements of the user 40 stored in the storage unit 102. For example, the gesture data generation unit 112 identifies characteristic movements of the body of the user 40 from data of the daily movements of the user 40, and generates gesture data indicating the identified movements.

[0052] The data collection unit 114 collects, for example, data on the behavior of the user 40. For example, the data collection unit 114 receives data on the daily behavior of the user 40 collected by the communication terminal 200 from the communication terminal 200. The communication terminal 200 collects behavioral data indicating the behavior of the user 40, for example, by analyzing the usage history of the communication terminal 200 by the user 40. As a specific example, the communication terminal 200 generates the behavioral data of the user 40 by using the usage history of apps such as a schedule management app and a payment app by the user 40. If the communication terminal 200 has a function of recording a life log of the user 40, the communication terminal 200 may generate data on the behavior of the user 40 by analyzing the data of the life log. The data collection unit 114 may receive the behavioral data of the user 40 from the communication terminal 200. The data collection unit 114 stores the collected behavioral data in the storage unit 102.

[0053] The behavior pattern generation unit 116 generates a behavior pattern of the user 40 by analyzing the behavior of the user 40. The behavior pattern generation unit 116 may generate a behavior pattern of the user 40 by analyzing behavior data of the user 40 stored in the storage unit 102. The storage unit 102 may store the user data of the user 40 including the behavior pattern of the user 40 generated by the behavior pattern generation unit 116.

[0054] When transmitting an authentication request for a user 40 to be authenticated to the authentication server 100, the management server 300 may include data of the avatar 42 of the user 40 and a behavioral pattern of the user 40 in the authentication request. For example, when authenticating the user 40, the communication terminal 200 may generate a behavioral pattern of the user 40 from user images captured continuously of the user 40 or from the user 40's usage history of the communication terminal 200. The communication terminal 200 transmits the behavioral pattern generated in this manner to the management server 300. The management server 300 transmits the authentication request to the authentication server 100, including the behavioral pattern received from the communication terminal 200. The authentication request acquisition unit 106 may acquire an authentication request including data of the avatar 42 of the user 40 to be authenticated and a behavioral pattern of the user 40 to be authenticated.

[0055] The authentication unit 108 may authenticate the user 40 to be authenticated by comparing a plurality of feature points and behavior patterns of the avatar 42 included in the authentication request with a plurality of feature points and behavior patterns included in the user data stored in the storage unit 102. For example, the authentication unit 108 executes both an authentication process in which a plurality of feature points of the face of the avatar 42 included in the authentication request is compared with a plurality of feature points included in the user data stored in the storage unit 102, and an authentication process in which a behavior pattern included in the authentication request is compared with a behavior pattern included in the user data stored in the storage unit 102. If both results indicate successful authentication, the authentication is deemed successful, and if at least one of the results indicates unsuccessful authentication, the authentication is deemed unsuccessful.

[0056] The authentication process using the behavior pattern included in the authentication request and the behavior pattern included in the user data may be such that authentication is successful if it is determined that the respective behavior patterns match, and is unsuccessful if it is determined that they do not match. The authentication process using the behavior pattern included in the authentication request and the behavior pattern included in the user data may be such that the similarity between these behavior patterns is calculated, and authentication is successful if the calculated similarity is higher than a predetermined threshold, and is unsuccessful if it is lower than the threshold. Methods other than these may also be used for the authentication process using the behavior pattern included in the authentication request and the behavior pattern included in the user data.

[0057] The authentication server 100 may authenticate the user 40 using both gesture data of the user 40 and the behavioral patterns of the user 40 in addition to the avatar 42.

[0058] 3 shows an example of the functional configuration of the management server 300. The management server 300 includes a storage unit 302, a management unit 304, a service providing unit 306, an authentication process execution unit 308, and a data providing unit 310. Note that it is not essential for the management server 300 to include all of these units.

[0059] The management unit 304 manages various data related to the user 40. For example, the management unit 304 manages the avatar 42 of the user 40. The management unit 304 may receive the avatar 42 from the communication terminal 200. The management unit 304 stores the avatar 42 of the user 40 in the storage unit 302.

[0060] The service providing unit 306 provides various services to the user 40. For example, the service providing unit 306 provides a metaverse space to the user 40. The service providing unit 306 may communicate with the communication terminal 200 to enable the user 40 to perform various activities using the avatar 42 in the metaverse space.

[0061] The authentication process execution unit 308 executes authentication processing for the user 40. For example, the authentication process execution unit 308 executes authentication processing for the user 40 in the metaverse space provided by the service provision unit 306. The authentication process execution unit 308 executes authentication processing for the user 40 when it receives an instruction from the user 40 requesting authentication of the user 40, or when the user 40 attempts to use a service that requires authentication in the metaverse space.

[0062] As part of the authentication process for the user 40 , the authentication process execution unit 308 generates an authentication request for the user 40 , transmits it to the authentication server 100 , and receives the authentication result from the authentication server 100 .

[0063] The authentication process execution unit 308 generates an authentication request including, for example, the avatar 42 of the user 40 and transmits it to the authentication server 100 .

[0064] The authentication process execution unit 308 may generate an authentication request that further includes gesture data indicating the body movement of the avatar 42 of the user 40 within the metaverse space provided by the service provision unit 306.

[0065] The authentication process execution unit 308 may receive, from the communication terminal 200, user images of the user 40 taken continuously by the communication terminal 200 when authenticating the user 40, and a behavioral pattern of the user 40 generated from the usage history of the communication terminal 200 by the user 40. The authentication process execution unit 308 may generate an authentication request that further includes the behavioral pattern.

[0066] The data providing unit 310 provides data related to the user 40 to the authentication server 100. The data providing unit 310 transmits to the authentication server 100, for example, a usage history of the metaverse space provided by the service providing unit 306 by the user 40 using the avatar 42.

[0067] The data collection unit 114 stores data about the user 40 received from the data providing unit 310 of the management server 300 in the storage unit 102. The data collection unit 114 stores, for example, the usage history of the metaverse space by the user 40 using the avatar 42 in the storage unit 102.

[0068] The behavior pattern generation unit 116 may generate a behavior pattern of the user 40 by analyzing data related to the user 40 stored in the storage unit 102. The storage unit 102 may store the user data of the user 40 including the behavior pattern of the user 40 generated by the behavior pattern generation unit 116.

[0069] 4 shows an outline of some examples of how services are used using the system 10. Service 512 is a variety of electronic payment and virtual currency wallet services, service 514 is a variety of social network services (SNS), and service 516 is an app or game with an AR (Augmented Reality) function. When using these services, user 40 may authenticate himself or herself using the system 10.

[0070] In the example of FIG. 4 , the services 518 are infrastructure services (electricity, gas, water, etc.), transportation services (trains, buses, rental cars, shared vehicles, toll roads, ships, and airplanes, etc.), and communication / terminal services. When using the services 518, the users 40 may authenticate themselves using the system 10. The users 40 may pay the fees for the services 518 using services 512, etc., linked via the system 10. The other servers 600 may be servers of public agencies, etc., and may link with the system 10 via the network 20. The system 10 may further use the credentials and ID information of the users 40 held by the other servers 600 to authenticate the users 40 when they use the services 518.

[0071] For example, the authentication server 100 authenticates the user 40 using at least data of the user 40's avatar 42, and cooperates with another server 600 to verify that the user 40 is licensed to drive a car. The authentication server 100 may permit the user 40 to drive a rental car or a shared vehicle only if the result of the authentication using at least the avatar 42 is successful and it is confirmed that the user 40 is licensed to drive a car. This prevents persons without the necessary qualifications from driving a car or the like. The authentication server 100 may also permit the user 40 to board a ferry while remaining in the car in a similar manner. This allows the user 40 to complete boarding procedures for the ferry without getting out of the car, improving the convenience of travel for the user 40. For example, when user 40 travels from the country in which he or she is currently staying to another country, system 10 may cooperate with another server 600 to check the immigration information of user 40, and may permit user 40 to travel to the other country only if it can confirm that user 40 has obtained permission corresponding to the passport or visa from the relevant government agency of the relevant country, and if the authentication server 100 has successfully authenticated user 40. This makes it possible to quickly perform some or all of the immigration inspection required when traveling across borders, improving the convenience of travel for user 40.

[0072] 4, service 522 is a service such as an online community or game that can be participated in using an avatar, service 524 is a service of a city in a virtual space that can be visited using an avatar, service 526 is a service of an AR city that projects an avatar onto a real city using AR technology and allows avatars to coexist with real people wearing head-mounted displays or the like, and service 528 is a service of an AR city that further allows AR buildings to coexist with real buildings. When using these services, user 40 may authenticate himself / herself using system 10.

[0073] When using these services, the user 40 may use his / her own avatar 42 provided by the system 10. For example, the user 40 may use the avatar 42 to converse with real people in the AR city. For example, the user 40 may use the avatar 42 to visit an AR building in the AR city, purchase goods from another avatar 42 who has also been authenticated using the system 10, and pay the purchase price from the virtual currency wallet of the service 512. By using the system 10, the security of transactions is improved because it is guaranteed that each avatar 42 is a legitimate transaction partner who has been authenticated. When using these services, the user 40 may use his / her own avatar 42 provided by the system 10. For example, the user 40 may wear a head-mounted display or the like as the communication terminal 200 and visit the AR city. The user 40 may visit the real city and conduct business with an avatar 42 operated by another user 40 who coexists there as AR. The user 40's avatar 42 may or may not be displayed to other users 40. This increases the degree of freedom of the user 40's activities in the metaverse space, and improves the user experience.

[0074] FIG. 5 schematically illustrates another example of the system 10. A description of the components of the system 10 in FIG. 5 that are common to the system 10 in FIG. 1 will be omitted, and only the components that differ from FIG. 1 will be described. In FIG. 5, the system 10 does not include a management server 300, and a communication terminal 200 manages an avatar 42 of a user 40. The method by which the communication terminal 200 generates the avatar 42 may be the same as in the case of FIG. 1. The communication device 400 may be a device installed in a store or the like that provides the service. In the example of FIG. 5, the communication device 400 does not include an imaging unit.

[0075] The communication terminal 200 communicates with the communication device 400 in accordance with an operation by the user 40. The communication between the communication terminal 200 and the communication device 400 may be NFC (Near Field Communication) communication. In this case, the user 40 may bring the communication terminal 200 close to the communication device 400. The communication terminal 200 may communicate with the communication device 400 via the network 20. The communication terminal 200 and the communication device 400 may also communicate with each other by other methods.

[0076] When authenticating user 40 before providing a service, communication device 400 may transmit an authentication request to authentication server 100 at the instruction of user 40. For example, upon receiving an instruction from user 40 and data of avatar 42 from communication terminal 200, communication device 400 transmits an authentication request for user 40, including data of avatar 42, to authentication server 100. Authentication server 100 authenticates user 40 by comparing the data of avatar 42 transmitted from communication device 400 with user data of multiple users 40 stored therein.

[0077] Specifically, the authentication request acquisition unit 106 acquires an authentication request transmitted by the communication device 400, the authentication request including data of the avatar 42 of the user 40. The authentication unit 108 compares the data of the avatar 42 included in the authentication request acquired by the authentication request acquisition unit 106 with the user data stored in the storage unit 102, thereby authenticating the user 40 to be authenticated. The authentication result output unit 110 transmits the authentication result from the authentication unit 108 to the communication device 400.

[0078] This provides an environment in which, for example, when communication device 400 is installed in a store, user 40 can be authenticated by the store using avatar 42 stored in communication terminal 200. Conventionally, authentication has been performed by capturing an image of user 40 using a device installed in the store and transmitting the captured image to a server. In this case, however, the captured image of user 40 is transmitted to the server via network 20 each time authentication is performed. In contrast, according to system 10 shown in FIG. 5, it is avatar 42 of user 40, rather than a captured image of user 40, that is transmitted to authentication server 100 via network 20, thereby improving security.

[0079] In the example shown in FIG. 5, the authentication server 100 may also authenticate the user 40 using other data in addition to the avatar 42.

[0080] For example, the authentication server 100 authenticates the user 40 using gesture data indicating the body movements of the user 40 in addition to the avatar 42. For example, the communication terminal 200 generates data on the body movements of the user 40 by analyzing user images continuously captured by the imaging unit 202 of the user 40, and uses the data to move the avatar 42. In response to an instruction from the user 40, the communication terminal 200 transmits the instruction from the user 40 and gesture data indicating the body movements of the avatar 42 to the communication device 400. In response to receiving the instruction from the user 40 and the gesture data of the avatar 42 from the communication terminal 200, the communication device 400 transmits an authentication request for the user 40, including the gesture data of the avatar 42, to the authentication server 100. The authentication unit 108 authenticates the user 40 using the multiple feature points of the face of the avatar 42 and the gesture data of the avatar 42 included in the authentication request. The method in which the authentication unit 108 authenticates the user 40 to be authenticated using a plurality of facial feature points and gesture data of the avatar 42 is the same as in the case of FIG.

[0081] For example, the authentication server 100 authenticates the user 40 using the behavioral patterns of the user 40 in addition to the avatar 42. For example, the communication terminal 200 may generate the behavioral patterns of the user 40 from user images captured continuously by the user 40 or from the user 40's usage history of the communication terminal 200. In response to an instruction from the user 40, the communication terminal 200 transmits the instruction from the user 40, data of the avatar 42, and the generated behavioral patterns to the communication device 400. In response to receiving the instruction from the user 40, the data of the avatar 42, and the behavioral patterns from the communication terminal 200, the communication device 400 transmits an authentication request including the data of the avatar 42 and the behavioral patterns to the authentication server 100. The authentication unit 108 authenticates the user 40 using the data and behavioral patterns of the avatar 42 included in the authentication request. The method by which the authentication unit 108 authenticates the user 40 to be authenticated using the data and behavioral patterns of the avatar 42 is the same as that shown in FIG. 1.

[0082] When authenticating user 40 before providing a service, communication terminal 200, rather than communication device 400, may send an authentication request to authentication server 100 in response to an instruction from user 40. For example, communication terminal 200 sends an authentication request for user 40, including data of avatar 42, to authentication server 100 in accordance with an instruction from user 40. Authentication server 100 authenticates user 40 by comparing the data of avatar 42 sent from communication terminal 200 with user data of multiple users 40 stored therein.

[0083] Specifically, authentication request acquisition unit 106 acquires an authentication request transmitted by communication terminal 200, the authentication request including data of avatar 42 of user 40. Authentication unit 108 compares the data of avatar 42 included in the authentication request acquired by authentication request acquisition unit 106 with user data stored in storage unit 102, thereby authenticating user 40 to be authenticated. Authentication result output unit 110 transmits the authentication result from authentication unit 108 to communication terminal 200.

[0084] This provides an environment in which, for example, in a store, user 40 can have communication terminal 200 authenticate user 40 with authentication server 100, and then present the authentication results received by communication terminal 200 from authentication server 100 to the store clerk, thereby demonstrating to the store clerk that he or she is an authenticated person.

[0085] In this example, the authentication server 100 may also use other data in addition to the avatar 42 to authenticate the user 40.

[0086] For example, communication terminal 200 generates data on the body movements of user 40 by analyzing user images continuously captured by imaging unit 202, and uses the data to move avatar 42. In response to an instruction from user 40, communication terminal 200 transmits the instruction from user 40 and gesture data indicating the body movements of avatar 42 to authentication server 100. Authentication unit 108 authenticates user 40 using a plurality of facial feature points of avatar 42 and the gesture data of avatar 42 included in the authentication request. The method by which authentication unit 108 authenticates user 40 to be authenticated using a plurality of facial feature points and gesture data of avatar 42 is the same as that shown in FIG. 1 .

[0087] For example, communication terminal 200 generates a behavior pattern of user 40 from user images captured continuously of user 40 and a usage history of communication terminal 200 by user 40. In response to an instruction from user 40, communication terminal 200 transmits the instruction from user 40, data of avatar 42, and the generated behavior pattern to authentication server 100. Authentication unit 108 authenticates user 40 using the data and behavior pattern of avatar 42 included in the authentication request. The method by which authentication unit 108 authenticates user 40 to be authenticated using the data and behavior pattern of avatar 42 is the same as in the case of FIG. 1.

[0088] 6 schematically illustrates an example of the hardware configuration of a computer 1200 that functions as the authentication server 100, the communication terminal 200, the management server 300, or the communication device 400. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of an apparatus according to the present embodiment, or can cause the computer 1200 to perform operations associated with the apparatus according to the present embodiment or one or more "parts," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0089] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0090] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0091] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0092] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0093] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0094] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0095] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0096] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0097] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0098] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0099] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0100] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0101] Computer-readable instructions may be provided locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc. to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or programmable circuitry, such that the processor or programmable circuitry executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0102] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0103] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0104] 10 system, 20 network, 40 user, 42 avatar, 100 authentication server, 102 memory unit, 104 registration unit, 106 authentication request acquisition unit, 108 authentication unit, 110 authentication result output unit, 112 gesture data generation unit, 114 data collection unit, 116 behavior pattern generation unit, 200 communication terminal, 202 imaging unit, 300 management server, 302 memory unit, 304 management unit, 306 service providing unit, 308 authentication processing execution unit, 310 data providing unit, 400 communication device, 402 imaging unit, 512, 514, 516, 518, 522, 524, 526, 528 service, 600 other server, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 Display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. a storage unit that stores user data including a plurality of feature points of a user's face; an authentication request acquisition unit that acquires an authentication request including data of an avatar of a user to be authenticated, the avatar data being generated using a plurality of feature points identified by analyzing a facial portion of the user in a user image generated by capturing an image of the user; an authentication unit that performs authentication of the user to be authenticated by comparing the avatar data included in the authentication request with the user data stored in the storage unit; A system comprising:

2. the authentication request acquisition unit acquires the authentication request including data of the avatar located in a metaverse space; The system of claim 1, wherein the authentication unit authenticates the user within the metaverse space by comparing multiple feature points of the avatar included in the authentication request with the multiple feature points included in the user data stored in the memory unit.

3. The system described in claim 2, wherein the authentication unit authenticates the user corresponding to the avatar periodically or irregularly while the avatar is located within the metaverse space, and controls the avatar to be prohibited from using the metaverse space if authentication fails.

4. the user data further includes gesture data indicating a body movement of the user; the authentication request acquisition unit acquires the authentication request, which further includes gesture data indicating a body movement of the avatar; 2. The system of claim 1, wherein the authentication unit authenticates the user to be authenticated by comparing the plurality of feature points of the avatar's face and the gesture data of the avatar included in the authentication request with the plurality of feature points of the user's face and the gesture data of the user included in the user data.

5. The system of claim 4, wherein the authentication request acquisition unit acquires the authentication request including the gesture data indicating body movements of the avatar performed based on body movements of the user identified by analyzing continuously captured user images of the user.

6. a gesture data generation unit that identifies a characteristic body movement of the user by analyzing continuously captured user images of the user and generates the gesture data indicating the identified movement; The system of claim 4 further comprising:

7. a behavior pattern generation unit that generates a behavior pattern of the user by analyzing the behavior of the user; Further provided with the storage unit stores the user data of the user including the behavior pattern generated by the behavior pattern generation unit; the authentication request acquisition unit acquires the authentication request including data of the avatar of the user to be authenticated and a behavior pattern of the user; 7. The system according to claim 1, wherein the authentication unit authenticates the user to be authenticated by comparing the plurality of feature points and the behavioral patterns of the avatar included in the authentication request with the plurality of feature points and the behavioral patterns included in the user data stored in the memory unit.

8. A program for causing a computer to function as the system according to any one of claims 1 to 6.

9. 1. A computer-implemented authentication method comprising: a storage step of storing user data including a plurality of feature points of the user's face in a storage unit; an authentication request acquisition step of acquiring an authentication request including data of an avatar of a user to be authenticated, the avatar data being generated using a plurality of feature points identified by analyzing a facial portion of the user in a user image generated by capturing an image of the user; an authentication step of authenticating the user to be authenticated by comparing the avatar data included in the authentication request with the user data stored in the storage unit; An authentication method comprising:

Images