Information processing apparatus and method for controlling the same, and program

Abstract

To provide a technique that can accurately authenticate a person to be authenticated, while simplifying preparation for authentication.SOLUTION: An information processing apparatus comprises: an image acquisition unit 131 that acquires a face image including the face of a person to be authenticated; a face authentication unit 135 that performs face authentication of the person to be authenticated by using the face image; an operation instruction unit 133 that instructs an operation to be executed for the person to be authenticated while holding the information processing apparatus 100-1; an operation authentication unit 136 that performs operation authentication of the person to be authenticated on the basis of the operation instructed by the operation instruction unit 133 and an actual operation of the information processing apparatus 100-1; and an authentication determination unit 137 that determines that authentication of the person to be authenticated is successful when the face authentication is successful and the operation authentication is successful, and determines that authentication of the person to be authenticated fails when at least one of the face authentication and the operation authentication fails.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a control method thereof, and a program. [Background technology]

[0002] In a facial recognition system using a camera mounted on a portable terminal device, a facial image of the person to be authenticated captured by the camera is compared with a pre-authenticated and registered facial image to determine whether the person to be authenticated is currently in front of the camera. This type of authentication is highly convenient because it can be completed simply by capturing a picture of the face with the camera, and can be applied to eKYC (electronic Know Your Customer), class attendance confirmation at schools, roll calls at work sites, etc.

[0003] However, since the person to be authenticated must carry the terminal device while performing the facial recognition process, it is difficult for others to keep an eye on them. Therefore, compared to facial recognition systems using fixed cameras, there is a problem that it is easier for someone to impersonate someone else using a photograph of their face.

[0004] A known method for detecting spoofing is to analyze whether a face in front of a camera has a flat shape, and if the face in front of the camera has a flat shape, to determine that the spoofing is using a photograph or the display of another terminal device, etc. For example, Patent Document 1 describes a method for determining whether an inappropriate operation such as spoofing is occurring by analyzing whether a face has a flat shape based on the relationship between the camera angle and the captured camera image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-204358 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, Patent Document 1 has a problem that preparation for authentication is complicated because it is necessary to register a large number of images obtained by photographing a face from many different angles as images for authentication. Also, Patent Document 1 has a problem that it is insufficient from the viewpoint of performing highly accurate authentication of a person to be authenticated, including cases of impersonation.

[0007] The present disclosure has been made in consideration of such problems, and aims to enable highly accurate authentication of a person to be authenticated while simplifying the preparation for authentication. [Means for solving the problem]

[0008] The information processing device disclosed herein is an information processing device that authenticates a person to be authenticated, and includes: an acquisition means for acquiring a facial image including the face of the person to be authenticated; a facial authentication means for performing facial authentication of the person to be authenticated using the facial image; an operation instruction means for instructing the person to be authenticated on an operation to be performed while holding the information processing device; an operation authentication means for performing operation authentication of the person to be authenticated based on the operation instructed by the operation instruction means and the actual operation of the information processing device; and a determination means for determining that the authentication of the person to be authenticated is successful if the facial authentication is successful and the operation authentication is also successful, and for determining that the authentication of the person to be authenticated is unsuccessful if at least one of the face authentication and the operation authentication fails. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to simplify the preparation for authentication while performing highly accurate authentication of the person to be authenticated. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating an example of the appearance of an information processing apparatus according to a first embodiment. [Figure 2] 1 is a diagram showing an example of the appearance when a person to be authenticated is undergoing face authentication processing using the information processing device according to the first embodiment. FIG. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of the information processing device according to the first embodiment. [Figure 4] 5 is a flowchart showing an example of a processing procedure in a control method for an information processing device according to the first embodiment. [Figure 5] FIG. 5 shows the first embodiment and is a diagram for explaining the process in step S101 of FIG. 4. [Figure 6] FIG. 5 shows the first embodiment and is a diagram for explaining the process in step S102 in FIG. 4. [Figure 7] FIG. 5 is a diagram illustrating the process in step S108 in FIG. 4 according to the first embodiment. [Figure 8] FIG. 5 shows the first embodiment and is a diagram for explaining the process in step S109 in FIG. 4. [Figure 9] 5 is a flowchart illustrating an example of a processing procedure in a method for determining whether or not to display a specific mark on a display in conjunction with the position and orientation of the information processing device in step S103 of FIG. 4 according to the first embodiment. [Figure 10] 10 is a diagram illustrating the process in step S201 of FIG. 9 according to the first embodiment. FIG. [Figure 11] 10 is a diagram illustrating the process in step S202 of FIG. 9 according to the first embodiment. FIG. [Figure 12] 10 is a diagram illustrating the process in step S202 of FIG. 9 according to the first embodiment. FIG. [Figure 13] 10 is a diagram illustrating the process in step S204 of FIG. 9 according to the first embodiment. FIG. [Figure 14] FIG. 5 shows the first embodiment and is a diagram for explaining the process in step S107 in FIG. 4. [Figure 15] FIG. 10 is a diagram illustrating an example of the appearance of an information processing device according to a second embodiment. [Figure 16] FIG. 10 is a diagram showing an example of the appearance when a person to be authenticated is undergoing face authentication processing using an information processing device according to a second embodiment. [Figure 17] FIG. 5 shows the second embodiment and is a diagram for explaining the process in step S102 in FIG. 4. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the configurations described in the following embodiments are merely examples, and the present disclosure is not limited to the configurations illustrated in the drawings.

[0012] (First embodiment) First, the first embodiment will be described.

[0013] In the first embodiment, an information processing device having a face authentication function for a person to be authenticated, a behavior authentication function for authenticating the behavior of the person to be authenticated using a specific mark, and an authentication determination function for determining the authentication of the person to be authenticated based on the results of the face authentication and behavior authentication will be described. Note that the face authentication in this embodiment is not particularly restricted in location and can be performed at any location. Therefore, it is effective, for example, when performing identity verification for eKYC at home.

[0014] 1 is a diagram showing an example of the appearance of an information processing device 100-1 according to the first embodiment. As the information processing device 100-1, it is preferable to apply a portable information terminal device that can be operated while being held (carried) by the person to be authenticated.

[0015] 1 shows the appearance of the inside of the information processing device 100-1. On the inside of the information processing device 100-1, an inside camera 111, which is a first imaging means, and a display 112, which is a display means, are provided. The inside camera 111 is a camera that captures an image in front of the inside of the information processing device 100-1, and can capture an image of a person to be authenticated who operates the information processing device 100-1 by visually recognizing the display 112 (particularly, the face of the person to be authenticated in this embodiment). The display 112 displays various images and various information.

[0016] An exterior 102 shown in FIG. 1 shows the exterior of the information processing device 100-1 (the surface opposite to the interior of the information processing device 100-1 shown by the exterior 101 in FIG. 1). An exterior camera 114, which is a second imaging means, is provided on the exterior side of the information processing device 100-1. The exterior camera 114 is a camera that captures an image in front of the exterior of the information processing device 100-1. A sensor 113 for measuring the position and orientation of the information processing device 100-1 is provided inside the information processing device 100-1. The sensor 113 is, for example, an inertial measurement unit or other sensor that can estimate a relative change from an initial position and orientation.

[0017] Fig. 2 is a diagram showing an example of the external appearance when a person to be authenticated NT is performing face authentication processing using the information processing device 100-1 according to the first embodiment. In Fig. 2, the same components as those shown in Fig. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0018] The person to be authenticated NT is located on the front side of the inner surface of the information processing device 100-1, on which the inner camera 111 and the display 112 are provided. A mark 201 (here, a virtual mark) is arranged in three-dimensional space, and a mark 202 based on the mark 201 is displayed on the display 112 based on the position and orientation of the information processing device 100-1 in the three-dimensional space. The person to be authenticated NT moves the information processing device 100-1 so that the mark 202 displayed on the display 112 in conjunction with the position and orientation of the information processing device 100-1 in the three-dimensional space continues to be displayed on the display 112, thereby performing a motion authentication process along with a face authentication process. In this way, the information processing device 100-1 is a device that also authenticates the person to be authenticated NT.

[0019] Fig. 3 is a diagram showing an example of the functional configuration of the information processing device 100-1 according to the first embodiment. The information processing device 100-1 has the functional configuration of an image acquisition unit 131, a position and orientation measurement unit 132, an action instruction unit 133, a face detection unit 134, a face authentication unit 135, an action authentication unit 136, an authentication determination unit 137, and an image generation unit 138. All of the functional configurations shown in Fig. 3 are connected to each other for communication, and can transmit and receive images and information to and from each other.

[0020] The image acquisition unit 131 is an acquisition means for acquiring a face image including the face of the person to be authenticated NT as a camera image captured by the inner camera 111.

[0021] The position and orientation measurement unit 132 is a measurement unit that uses the sensor 113 to measure the position and orientation of the information processing device 100-1.

[0022] The operation instruction unit 133 is an operation instruction means for instructing the authentication target NT on an operation to be performed while holding the information processing device 100-1 during face authentication processing.

[0023] The face detection unit 134 is a face detection means for detecting the face of the person to be authenticated NT from the face image of the person to be authenticated acquired by the image acquisition unit 131.

[0024] The face authentication unit 135 is a face authentication means that performs face authentication of the person to be authenticated NT by determining whether or not the face of the person to be authenticated NT detected by the face detection unit 134 matches (matches) any face of a person who has been authenticated and registered in advance. The face authentication unit 135 determines that face authentication of the person to be authenticated NT has been successful if the face of the person to be authenticated NT detected by the face detection unit 134 matches (matches) any face of a person who has been authenticated and registered in advance. On the other hand, the face authentication unit 135 determines that face authentication of the person to be authenticated NT has failed if the face of the person to be authenticated NT detected by the face detection unit 134 does not match (match) any face of a person who has been authenticated and registered in advance.

[0025] The action authentication unit 136 is an action authentication means that performs action authentication of the authentication target NT based on the action instructed by the action instruction unit 133 and the actual action of the information processing device 100-1. At this time, the action authentication unit 136 detects the actual action of the information processing device 100-1 described above based on the position and orientation of the information processing device 100-1 measured by the position and orientation measurement unit 132. Specifically, the action authentication unit 136 determines that the action authentication of the authentication target NT has been successful when the action instructed by the action instruction unit 133 matches the actual action of the information processing device 100-1. On the other hand, the action authentication unit 136 determines that the action authentication of the authentication target NT has failed when the action instructed by the action instruction unit 133 does not match the actual action of the information processing device 100-1.

[0026] The authentication determination unit 137 determines that the authentication of the authentication subject NT is successful when the face authentication by the face authentication unit 135 is successful and the action authentication by the action authentication unit 136 is successful. On the other hand, the authentication determination unit 137 determines that the authentication of the authentication subject NT is unsuccessful when at least one of the face authentication by the face authentication unit 135 and the action authentication by the action authentication unit 136 is unsuccessful. Furthermore, even if the face authentication by the face authentication unit 135 is successful, when the action authentication by the action authentication unit 136 is unsuccessful, the authentication determination unit 137 determines that the authentication of the authentication subject NT is unsuccessful and determines that an act of impersonation of the authentication subject NT has occurred.

[0027] The image generation unit 138 is an image generation means that generates an image including various images and various information obtained by each functional configuration (131 to 137). The image generated by the image generation unit 138 can be displayed on the display 112. The image displayed on the display 112 also includes information indicating the determination result of the authentication determination unit 137.

[0028] Fig. 4 is a flowchart showing an example of a processing procedure in the control method of the information processing device 100-1 according to the first embodiment. The authentication processing of the authentication target NT will be described below with reference to Fig. 4. Note that in the following description, an example in which the operation time (time limit) of the authentication processing is set to 10 seconds will be shown as a specific example, but the operation time (time limit) of the authentication processing may be changed and set as appropriate depending on the environment, use case, etc.

[0029] First, in step S101 of Fig. 4, the information processing device 100-1 launches an authentication application to authenticate the person to be authenticated NT. Fig. 5 shows the first embodiment and is a diagram for explaining the processing in step S101 of Fig. 4. In Fig. 5, components that are the same as those shown in Figs. 1 and 2 are given the same reference numerals, and detailed explanations thereof will be omitted. In Fig. 5, a button 500 is a button for launching the authentication application. When the person to be authenticated NT presses the button 500, the information processing device 100-1 detects this and launches the authentication application.

[0030] Next, in step S102 of FIG. 4, the operation instructing unit 133 instructs the authentication target NT on an operation to be performed while holding the information processing device 100-1 during face authentication processing. Specifically, in this embodiment, the operation instructing unit 133 instructs the authentication target NT on an operation to be performed while holding the information processing device 100-1 by displaying the operation instruction content on the display 112. FIG. 6 shows the first embodiment and is a diagram for explaining the processing in step S102 of FIG. 4. In FIG. 6, components similar to those shown in FIGS. 1, 2, and 5 are designated by the same reference numerals, and detailed description thereof will be omitted. The screen displayed on the display 112 in FIG. 6 is an example of an image displayed after the button 500 in FIG. 5 is pressed. The display 112 in FIG. 6 displays an operation instruction content display area 610, a "Start" button 620, and a "Back" button 630. The operation instruction content display area 610 shown in FIG. 6 displays the operation instruction content, "Please move the device so that a specific mark (611) continues to appear." The person to be authenticated NT can start the authentication process by pressing the "Start" button 620. The person to be authenticated NT can also terminate the authentication application by pressing the "Back" button 630. When the person to be authenticated NT presses the "Start" button 620 to start the face authentication process, the processes of steps S103, S104, and S105 in FIG. 4, which will be described later, are performed. In the example shown in FIG. 4, the processes of steps S103, S104, and S105, which will be described later, are started simultaneously and processed in parallel, and end after 10 seconds, which is the operation time (time limit) of the authentication process. The person to be authenticated NT can succeed in the authentication process by continuing to move the information processing device 100-1 so that a specific mark (611) displayed in the operation instruction content display area 610 in FIG. 6 remains visible for 10 seconds, which is the operation time (time limit) of the authentication process.

[0031] 4, in step S103, the display 112 displays a specific mark 611 corresponding to the mark 202, in accordance with the position and orientation of the information processing device 100-1 and the position of the mark 201 measured by the position and orientation measurement unit 132. Note that, depending on the position and orientation of the information processing device 100-1 and the position of the mark 201, the specific mark 611 corresponding to the mark 202 may not be displayed on the display 112.

[0032] 4, in step S104, the position and orientation measurement unit 132 measures the position and orientation of the information processing device 100-1 in order to detect the operation being performed by the person to be authenticated NT while holding the information processing device 100-1. In this embodiment, the position and orientation measurement unit 132 uses the sensor 113 to measure the position and orientation at a rate of 30 times per second while the person to be authenticated NT is operating the information processing device 100-1, and stores the measurement results. In this case, since the operation time (time limit) of the authentication process is 10 seconds in this embodiment, a total of 300 pieces of information indicating the position and orientation of the information processing device 100-1 are stored.

[0033] 4, in step S105, the face authentication unit 135 performs face authentication processing using a face image of the person to be authenticated NT acquired using the inner camera 111 while the person to be authenticated NT is performing an operation while holding the information processing device 100-1. In this embodiment, the face authentication unit 135 performs face authentication processing at a rate of twice per second. In this case, since the operation time (time limit) of the authentication processing is 10 seconds in this embodiment, the face authentication processing is performed a total of 20 times.

[0034] When 10 seconds, which is the operation time (time limit) for the authentication process, has elapsed, the process proceeds to step S106 in Fig. 4. When proceeding to step S106 in Fig. 4, the face authentication unit 135 determines whether or not face authentication of the authentication target NT has been successful, based on the result of the face authentication process in step S105 in Fig. 4. In this embodiment, the face authentication unit 135 determines that face authentication has been successful when 90% or more of the face authentication processes performed in step S105 in Fig. 4, that is, the results of 18 or more face authentication processes, are identical (match) to the face of a person who has been previously authenticated and registered. In all other cases, the face authentication unit 135 determines that face authentication has failed.

[0035] In step S106 of FIG. 4, if the face authentication unit 135 determines that face authentication of the authentication target NT has been successful (S106 / YES), the process proceeds to step S107 of FIG. 4. In step S107 of FIG. 4, the action authentication unit 136 determines whether action authentication of the authentication target NT has been successful based on the action instructed by the action instruction unit 133 and the actual action of the information processing device 100-1 based on the position and orientation measured in step S104. More specifically, the action authentication unit 136 analyzes the information indicating the position and orientation of the information processing device 100-1 saved in step S104 to detect the action content performed by the authentication target NT on the information processing device 100-1 during the authentication process. Next, the action authentication unit 136 compares the detected action content with the action content instructed in step S102 and determines whether action authentication of the authentication target NT has been successful based on whether they match. Specifically, the operation authentication unit 136 determines that the operation authentication of the authentication target NT has been successful when the operation instructed by the operation instruction unit 133 matches the actual operation of the information processing device 100-1. On the other hand, the operation authentication unit 136 determines that the operation authentication of the authentication target NT has failed when the operation instructed by the operation instruction unit 133 does not match the actual operation of the information processing device 100-1.

[0036] In step S107 of FIG. 4, if the behavior authentication unit 136 determines that the behavior authentication of the authentication target NT has been successful (S107 / YES), the process proceeds to step S108 of FIG. 4. In step S108 of FIG. 4, the authentication assessment unit 137 determines that the authentication of the authentication target NT has been successful. That is, since the authentication assessment unit 137 determined that the face authentication was successful in step S106 and that the behavior authentication was successful in step S107, the authentication assessment unit 137 determines that the authentication of the authentication target NT has been successful. Note that in the process of step S108 of FIG. 4, the authentication assessment unit 137 performs a process of displaying on the display 112 that the authentication of the authentication target NT has been successful. FIG. 7 shows the first embodiment and is a diagram for explaining the process of step S108 of FIG. 4. In FIG. 7, components similar to those shown in FIGS. 1, 2, 5, and 6 are designated by the same reference numerals, and detailed description thereof will be omitted. 7, an authentication success display area 710, a "Start" button 720, and a "Back" button 730 are displayed on the display 112 of Fig. 7. A registered image 711 corresponding to the person to be authenticated NT and information 712 indicating that authentication has been successful are displayed in the authentication success display area 710 shown in Fig. 7. When the person to be authenticated NT presses the "Start" button 720 in this state, the process returns to step S102 of Fig. 4. When the person to be authenticated NT presses the "Back" button 730, the process of the flowchart shown in Fig. 4 ends.

[0037] 4, if the face authentication unit 135 determines that face authentication of the authentication target NT has failed (S106 / NO), the process proceeds to step S109 in FIG. 4. Similarly, if the operation authentication unit 136 determines that operation authentication of the authentication target NT has failed (S107 / NO) in step S107 in FIG. 4, the process proceeds to step S109 in FIG. 4. When proceeding to step S109 in FIG. 4, the authentication determination unit 137 determines that authentication of the authentication target NT has failed. That is, the authentication determination unit 137 determines that authentication of the authentication target NT has failed because it has determined that face authentication has failed in step S106 or that operation authentication has failed in step S107. Note that even if it has determined that face authentication has succeeded in S106, if it has determined that operation authentication has failed in S107, the authentication determination unit 137 determines that authentication of the authentication target NT has failed, and determines that an act of spoofing has occurred. 4, the authentication assessment unit 137 performs processing to display on the display 112 that authentication of the authentication target NT has failed. FIG. 8 shows the first embodiment and is a diagram for explaining the processing in step S109 of FIG. 4. In FIG. 8, the same components as those shown in FIGS. 1, 2, and 5 to 7 are assigned the same reference numerals, and detailed description thereof will be omitted. In FIG. 8, an authentication failure display area 810, a "Start" button 820, and a "Back" button 830 are displayed on the display 112 of FIG. 8. Information indicating that authentication has failed is displayed in the authentication failure display area 810 shown in FIG. 8. In this state, when the authentication target NT presses the "Start" button 720, the processing returns to step S102 of FIG. 4. When the authentication target NT presses the "Back" button 730, the processing of the flowchart shown in FIG. 4 ends.

[0038] 4, the position and orientation measurement process of the information processing device 100-1 in step S104 and the face authentication process in step S105 have been described as being started simultaneously and processed in parallel. However, the present disclosure is not limited to this configuration, and may be configured such that, for example, the face authentication process in step S105 is performed after the position and orientation measurement process of the information processing device 100-1 in step S104.

[0039] Next, a method for determining whether or not to display a specific mark 611 on the display 112 in conjunction with the position and orientation of the information processing device 100-1 in three-dimensional space in step S103 of Fig. 4 will be described. Fig. 9 is a flowchart illustrating an example of the processing procedure for the method for determining whether or not to display a specific mark 611 on the display 112 in conjunction with the position and orientation of the information processing device 100-1 in step S103 of Fig. 4 according to the first embodiment.

[0040] First, in step S201 of FIG. 9, the information processing device 100-1 sets the initial position of a mark to be placed in three-dimensional space. Here, the information processing device 100-1 sets the position of the mark in three-dimensional space based on position and orientation information acquired using the sensor 113 at the start time of the authentication process so that the position of the mark in three-dimensional space is 2 m in front of the information processing device 100-1. FIG. 10 shows the first embodiment and is a diagram for explaining the processing in step S201 of FIG. 9. In FIG. 10, components similar to those shown in FIGS. 1, 2, and 5 to 8 are assigned the same reference numerals, and detailed description thereof will be omitted. In FIG. 10, point 1000 indicates the position of the information processing device 100-1 at the start time of the authentication process. Furthermore, point 1001 indicates a point located 2 m in the depth direction from point 1000. Therefore, the initial position of the mark set in step S201 of FIG. 9 is set to the position of point 1001 in FIG. 10.

[0041] 9, the information processing device 100-1 draws a mark on the display 112 according to the position and orientation of the information processing device 100-1 measured by the position and orientation measurement unit 132 and the position of the mark. FIG. 11 shows the first embodiment and is a diagram for explaining the processing in step S202 of FIG. 9. In FIG. 11, the same components as those shown in FIGS. 1, 2, 5 to 8, and 10 are denoted by the same reference numerals, and detailed description thereof will be omitted. For example, when the process transitions from step S201 of FIG. 9 to this step S202, the information processing device 100-1 draws a mark 1100 at the position of point 1000 in FIG. 10, as shown in FIG. 11. When the process transitions from step S205 of FIG. 9 to this step S202, the information processing device 100-1 determines the drawing position of the mark using the position and orientation of the mark updated in step S204 and step S205, and draws the mark 1100. However, for example, if the position of the mark 1001 is outside the field of view of the external camera 114 as viewed from the information processing device 100-1, the mark 1100 is not drawn. FIG. 12 illustrates the first embodiment and is a diagram for explaining the processing in step S202 of FIG. 9. Specifically, FIG. 12 illustrates the relationship between the field of view of the external camera 114 as viewed from the information processing device 100-1 and the position of the mark set in three-dimensional space. FIG. 12 illustrates an area 1200 within the field of view of the external camera 114 as viewed from the information processing device 100-1, a point 1201 within the area 1200, and a point 1202 outside the area 1200. If the position of the mark set in three-dimensional space is outside the field of view of the external camera 114 as viewed from the information processing device 100-1, as in the case of point 1202 illustrated in FIG. 12, the mark 1100 is not drawn.

[0042] 9, the information processing device 100-1 determines whether the time elapsed since the start time of the authentication process is within the time limit. In this embodiment, the time limit, which is the operation time of the authentication process, is 10 seconds, so if the time elapsed since the start time of the authentication process is 10 seconds or less (S203 / YES), the information processing device 100-1 proceeds to step S204.

[0043] Next, in step S204 of FIG. 9, the information processing device 100-1 (e.g., the operation instruction unit 133) updates the position of the mark set in three-dimensional space. In this embodiment, the mark position is updated using random numbers r_x, r_y, and r_z that follow a uniform distribution between −1 m and +1 m. However, the mark position may be updated using another method. For example, a straight line or curve in three-dimensional space may be set, and the mark position may be updated so as to move along the line at a constant speed. Here, if the position coordinates of the mark before update are P(x, y, z) and the position coordinates of the mark after update are P′(x′, y′, z′), then x′=x+r_x, y′=y+r_y, z′=z+r_z. FIG. 13 illustrates the first embodiment and is a diagram for explaining the processing in step S204 of FIG. 9. Specifically, FIG. 13 illustrates an example of updating the position of the mark from the position (initial position) P of the mark before update to position P′. In the example shown in Fig. 13, the initial position P of the mark is at point 1001, and the updated position P' of the mark is at point 1301. The example shown in Fig. 13 shows the process of updating the position of the mark from the initial position, but in the second and subsequent updates, a different random number is generated again to generate a new mark position and update the mark position in the same way.

[0044] 9, the information processing device 100-1 acquires the latest position and orientation of the information processing device 100-1 from the sensor 113, and updates the position and orientation of the information processing device 100-1. Then, the process returns to step S202.

[0045] Also, in step S203 of FIG. 9, if the time elapsed since the start time of the authentication process is not within the time limit (S203 / NO), specifically, if the time limit of 10 seconds is exceeded, the information processing device 100-1 terminates the processing of the flowchart shown in FIG. 9.

[0046] Next, a specific example of the action authentication process in step S107 of Fig. 4 will be described. In step S107 of Fig. 4, the action authentication unit 136 performs action authentication process for the authentication target NT based on the action instructed by the action instruction unit 133 and the actual action of the information processing device 100-1 based on the position and orientation measured in step S104. In this embodiment, the action authentication unit 136 determines whether the position and orientation of the information processing device 100-1 are appropriate at each time during the action time of the authentication process, and if the position and orientation are appropriate for 90% or more of the time, determines that the action matches the action instructed in step S102.

[0047] In this embodiment, the instruction for the operation in step S102 in FIG. 4 is, "Move the device so that a specific mark (611) continues to appear," as displayed in the instruction display area 610 in FIG. 6. Therefore, the operation authentication unit 136 determines that the position and orientation of the information processing device 100-1 are valid if the specific mark (611) appears on the display 112 at each time during the operation of the authentication process. Whether or not the mark appears on the display 112 can be determined from the position of the mark in three-dimensional space and the position and orientation of the information processing device 100-1, similar to the process in step S202 in FIG. 9. FIG. 14 illustrates the first embodiment and is a diagram for explaining the process in step S107 in FIG. 4. FIG. 14 illustrates each time (T0 to Tn) during the authentication process, the position and orientation of the information processing device (R0 to Rn), the position of the mark (P0 to Pn), whether the mark appears on the display (YES / NO), and whether the position and orientation of the information processing device is valid (YES / NO). 14, the action authentication unit 136 analyzes whether or not a mark is displayed on the display 112 at each time (T0 to Tn) during the authentication process, and determines that the position and orientation of the information processing device 100-1 are valid if the mark is displayed on the display 112. Next, the action authentication unit 136 calculates the percentage of the positions and orientations of the information processing device 100-1 that are determined to be valid, and if the calculated percentage is 90% or higher, determines that the action matches the action instructed in step S102.

[0048] In the information processing device 100-1 according to the first embodiment described above, the face authentication unit 135 performs face authentication of the person to be authenticated NT, and the operation authentication unit 136 performs operation authentication of the person to be authenticated NT. If the face authentication by the face authentication unit 135 is successful and if the operation authentication by the operation authentication unit 136 is successful, the authentication assessment unit 137 determines that the authentication of the person to be authenticated NT is successful. On the other hand, if the face authentication by the face authentication unit 135 is unsuccessful or if the operation authentication by the operation authentication unit 136 is unsuccessful, the authentication assessment unit 137 determines that the authentication of the person to be authenticated NT is unsuccessful. According to this configuration, it is possible to simplify the preparation for authentication and to perform highly accurate authentication of the person to be authenticated.

[0049] Furthermore, even if the face authentication by the face authentication unit 135 is successful, if the behavior authentication by the behavior authentication unit 136 fails, the authentication judgment unit 137 judges that the authentication of the person to be authenticated NT has failed, and judges that an act of impersonation has been committed by impersonating the person to be authenticated NT. This configuration makes it possible to reduce the risk of spoofing.

[0050] (Second embodiment) Next, a second embodiment will be described. In the following description of the second embodiment, matters common to the first embodiment will be omitted, and only matters different from the first embodiment will be described.

[0051] In the first embodiment described above, the instruction for the operation during the facial authentication process (S102 in FIG. 4) was in the form of moving the device so that a specific mark based on a virtual mark would continue to appear on the display. In the first embodiment described above, the position of the mark was set near the initial position of the device. Therefore, facial authentication can be successful at any location, and the location where the person to be authenticated NT performed facial authentication cannot be identified. For example, when facial authentication is used to check attendance at a school class, authentication may be successful even if the person to be authenticated NT is not actually in the classroom but is in another building, etc. Therefore, in the second embodiment, a form of instruction for the operation that requires the person to be authenticated NT to be in a specific location will be described. The second embodiment is particularly effective in cases where the location at the time of authentication is important, such as the above-mentioned school class attendance check.

[0052] 15 is a diagram showing an example of the appearance of an information processing device 100-2 according to the second embodiment. As the information processing device 100-2, it is preferable to apply a portable information terminal device that can be operated while being held (carried) by the person to be authenticated.

[0053] 15 shows the appearance of the inside of the information processing device 100-2. An inside camera 121, which is a first imaging means, and a display 122, which is a display means, are provided on the inside of the information processing device 100-2. The inside camera 121 is a camera that captures an image of the front of the inside of the information processing device 100-2, and can capture an image of a person to be authenticated who operates the information processing device 100-2 (particularly, the face of the person to be authenticated in this embodiment) by visually recognizing the display 122, etc. The display 122 displays various images and various information.

[0054] An external appearance 104 shown in FIG. 15 illustrates the external appearance of the information processing device 100-2 (the surface opposite to the internal surface of the information processing device 100-2 illustrated by the external appearance 103 in FIG. 15). An external camera 125, which is a second imaging unit, is provided on the external surface of the information processing device 100-2. The external camera 125 is a camera that captures an image of the area in front of the external surface of the information processing device 100-2. A sensor 123 for measuring the absolute attitude of the information processing device 100-2 and a sensor 124 for measuring the absolute position of the information processing device 100-2 are provided inside the information processing device 100-2. The sensor 123 is a sensor that can estimate the absolute attitude, such as an inertial measurement unit including a magnetic sensor. The sensor 124 is realized by a global navigation satellite system (GNSS) or the like, for example, when it is assumed that the information processing device 100-2 will be used outdoors. Furthermore, when it is assumed that the information processing device 100-2 is used indoors, the sensor 124 is, for example, a sensor that estimates a position based on the radio wave intensity of Wi-Fi, a sensor that uses UWB positioning, or the like.

[0055] The functional configuration of the information processing device 100-2 according to the second embodiment is similar to the functional configuration of the information processing device 100-1 according to the first embodiment shown in FIG.

[0056] Fig. 16 is a diagram showing an example of the appearance when a person to be authenticated NT is performing face authentication processing using an information processing device 100-2 according to the second embodiment. In Fig. 16, the same components as those shown in Fig. 15 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0057] The person to be authenticated NT is located at the front side of the inner surface of the information processing device 100-2, which is provided with the inner camera 121 and display 122. The display 122 shown in Fig. 16 displays the current position 1600 of the information processing device 100-2. The person to be authenticated NT moves to a specified position while holding the information processing device 100-2, thereby performing face authentication processing and motion authentication processing. In this way, the information processing device 100-2 is a device that also authenticates the person to be authenticated NT.

[0058] The control method of the information processing device 100-2 according to the second embodiment is basically the same as the processing in the flowchart showing an example of the processing procedure in the control method of the information processing device 100-1 according to the first embodiment shown in Fig. 4. Below, differences from the processing in the first embodiment described above will be described with reference to Fig. 4.

[0059] In step S101 of FIG. 4, the information processing device 100-2 starts an authentication application, similarly to the first embodiment described above.

[0060] Next, in step S101 of FIG. 4, the operation instructing unit 133 instructs the authentication target NT to perform an operation while holding the information processing device 100-2 during face authentication processing, similar to the first embodiment described above. Specifically, in this embodiment, similar to the first embodiment described above, the operation instructing unit 133 instructs the authentication target NT to perform an operation while holding the information processing device 100-2 by displaying the operation instruction content on the display 122. FIG. 17 shows the second embodiment and is a diagram for explaining the processing in step S102 of FIG. 4. In FIG. 17, components similar to those shown in FIGS. 15 and 16 are designated by the same reference numerals, and detailed description thereof will be omitted. The screen displayed on the display 122 in FIG. 17 is an example of an image displayed after the button 500 in FIG. 5 is pressed. An operation instruction content display area 1710, a "Start" button 1720, and a "Back" button 1730 are displayed on the display 122 in FIG. 17. 17 displays an operation instruction content display area 1710 stating, "Please move to a specific destination (1711)" from a current location 1712 of the information processing device 100-2. Here, the destination 1711 is set by, for example, registering a plurality of destination candidates in the information processing device 100-2 in advance and selecting the nearest destination from the current location 1712 of the information processing device 100-2. More specifically, in the case of a system for taking attendance in a classroom, for example, destinations are set at the four corners of the classroom, and the destination 1711 in the nearest corner is selected from the current location 1712 of the information processing device 100-2.

[0061] 4, the display 122 displays a current position 1712 and a destination 1711 of the information processing device 100-2 according to the position and orientation of the information processing device 100-2, as shown in an operation instruction content display area 1710 in Fig. 17. At this time, the authentication subject NT moves toward the destination 1711 while holding the information processing device 100-2 and looking at the display 122.

[0062] In the second embodiment, steps S104 to S106 in FIG. 4 are performed in the same manner as in the first embodiment.

[0063] 4, the action authentication unit 136 determines whether or not the action authentication of the authentication target NT has been successful, based on the action instructed by the action instruction unit 133 and the actual action of the information processing device 100-2 based on the position and orientation measured in S104. Specifically, the action authentication unit 136 determines that the action authentication of the authentication target NT has failed if the distance between the current position 1712 of the information processing device 100-2 and the destination 1711 is equal to or greater than a threshold. In other cases, the action authentication unit 136 determines that the action authentication of the authentication target NT has been successful.

[0064] In step S107 of Fig. 4, if the operation authentication unit 136 determines that the operation authentication of the authentication target NT has been successful (S107 / YES), the process proceeds to step S108 of Fig. 4. Also, in step S107 of Fig. 4, if the operation authentication unit 136 determines that the operation authentication of the authentication target NT has failed (S107 / NO), the process proceeds to step S109 of Fig. 4. In the second embodiment, the same processes as those in the first embodiment described above are performed in steps S108 and S109 of Fig. 4.

[0065] According to the second embodiment, similar to the first embodiment described above, it is possible to simplify the preparation for authentication and perform highly accurate authentication of the person to be authenticated. Furthermore, according to the second embodiment, it is possible to further reduce the risk of impersonation.

[0066] (Modification of the above-described embodiment) Next, a modification of the above-described embodiment will be described.

[0067] <Variation 1> In the first embodiment described above, when the information processing device 100-1 is moved so that the mark 611 shown in FIG. 6 continues to appear on the display 112, only the mark 611 is displayed on the display 112. Therefore, once the mark 611 disappears, it may be difficult to find the mark again. Therefore, in Modification 1, a configuration is considered in which, instead of only the mark, a composite image is displayed on the display 112 by, for example, using VR (Virtual Reality). Furthermore, in Modification 1, a configuration is considered in which, instead of only the mark, a composite image is displayed on the display 112 by, for example, using AR (Augmented Reality) by combining the mark with a real environment captured by the external camera 114. As in Modification 1, by simultaneously displaying the mark and the virtual environment or the real environment on the display 112, even if the mark disappears from the display 112, it is intuitively easy to find the position where the mark was lost. This makes it easy to find the mark again. Furthermore, in the present modified example 1, when VR is used, for example, a CG model of a wide-area environment may be rendered using the current position and orientation of the information processing device 100-1, and the result may be displayed on the display 112. Furthermore, when VR is used, a part of the virtual environment may be used as a substitute for a mark, and only the virtual environment may be displayed on the display 112. Furthermore, in the present modified example 1, when AR is used, for example, an image of the scenery ahead on the outer surface of the information processing device 100-1 may be captured by the outer camera 114, and a composite image may be generated by combining the mark and the scenery, and displayed on the display 112.

[0068] <Variation 2> In the first embodiment described above, it is not possible to identify the location where the face authentication process is performed. Therefore, in order to realize a method in which authentication is successful only in an expected location, it is preferable to use the position of the mark and the position and orientation of the information processing device 100-1 based on a certain coordinate system, without depending on the initial position and orientation of the information processing device 100-1. This modification 2 can be realized, for example, by the following procedure.

[0069] As a preliminary preparation, first, an environment is constructed in which the position and orientation of the information processing device 100-1 can be measured in real time using a motion capture system with an environmentally installed camera, with one coordinate system as the reference. Next, with the above-mentioned coordinate system as the reference, the initial position of the mark and the position of the mark at each time after the start of the authentication process are set in advance. Then, during the authentication process, the processing in the first embodiment described above can be performed using the position and orientation of the information processing device 100-1 and the position of the mark based on the above-mentioned coordinate system.

[0070] <Variation 3> In the above-described second modification, similarly to the first embodiment, authentication processing is performed using a virtual mark that does not exist in the real environment. In the present third modification, an object that actually exists in the three-dimensional space, such as a robot running on the ground or a drone flying in the air (hereinafter referred to as a "real mark"), is used instead of the virtual mark arranged in the three-dimensional space. In the present third modification, in order to display the real mark on the display 112, it is necessary to capture an image of the real environment with the external camera 114.

[0071] Furthermore, in order to determine whether the real mark is within the field of view of the external camera 114 of the information processing device 100-1, as described in Modification 2, the position of the real mark and the position and orientation of the information processing device 100-1 based on a certain coordinate system are required. Therefore, for example, as in Modification 2, it is preferable to use a motion capture system using an environmentally installed camera to measure and use the position of the real mark and the position and orientation of the information processing device 100-1 in real time. Furthermore, if the real mark is a device equipped with a camera, it is preferable to use SLAM (Simultaneous Localization and Mapping) using images. That is, if the real mark is a device equipped with a camera, it is preferable to generate a map of the environment in advance using SLAM using images, and measure and use the position of the real mark and the position and orientation of the information processing device 100-1 based on the coordinate system of the map. Furthermore, it is preferable to measure the position of the real mark using an image captured by a camera mounted on the real mark, and measure the position and orientation of the information processing device 100-1 using an image captured by the external camera 114 of the information processing device 100-1.

[0072] Alternatively, a stationary object may be used as the real mark instead of a real mark whose position changes. In the case of a stationary object, the position of the real mark based on the coordinate system described above may be measured in advance and used. This measurement may be performed using, for example, a total station.

[0073] In the present third modification, for example, the external camera 114 captures an image of an actual mark arranged in a three-dimensional space, and the display 112 displays the actual mark captured by the external camera 114. Then, the operation instructing unit 133 may take a form in which the operation instructs the information processing device 100-1 to operate so that the actual mark continues to be displayed on the display 112 for the authentication subject NT, for example.

[0074] <Variation 4> In the above-described third modification, a method of using a stationary object as a real mark has been described. However, in the case of a stationary object, it is easy to move the information processing device 100-1 so that the real mark is captured, which may lead to successful spoofing. Therefore, it is possible to provide multiple real marks and increase the difficulty of the operation of moving the real marks within the field of view of the external camera 114 of the information processing device 100-1.

[0075] For example, multiple real marks may be set discretely on an indoor wall as a preparation, and if five or more real marks come into the field of view of the exterior camera 114 within 10 seconds during the authentication process, it may be determined that there is no impersonation. In this case, the operation instructing unit 133 may take a form of instructing the information processing device 100-1 to operate so that each of the multiple real marks is displayed on the display 112 at least once within a certain period of time for the person to be authenticated NT.

[0076] Furthermore, for example, restrictions may be placed on the order in which real marks are placed within the field of view of the external camera 114, and it may be determined that no impersonation has occurred only when the real marks are placed within the field of view of the external camera 114 in a specific order. In this case, the operation instructing unit 133 may take a form in which it instructs the information processing device 100-1 to operate so that the real marks among the multiple real marks are displayed on the display 112 in a predetermined order for the person to be authenticated NT.

[0077] <Variation 5> In the second embodiment described above, an example has been described in which an action instruction such as "Please move from the current position 1712 of the information processing device 100-2 to a specific destination (1711)" is given during authentication processing. The present disclosure is not limited to this example. For example, the action instruction unit 133 may give an action instruction such as "Please pass through a specific area" to the information processing device 100-2. In this case, for example, the information processing device 100-2 sets a three-dimensional area in a specific space as a specific area as a preparation in advance. Then, the action authentication unit 136 may analyze the position and orientation of the information processing device 100-2 saved in S104 of FIG. 4, and perform action authentication depending on whether the information processing device 100-2 enters the specific area from outside the specific area and then exits the specific area.

[0078] Furthermore, in the second embodiment described above, the position and orientation measurement unit 132 measures the absolute position and orientation of the information processing device 100-2 based on one coordinate system, but the present disclosure is not limited to this configuration. For example, the position and orientation measurement unit 132 may measure the relative position and orientation of the information processing device 100-1, as in the first embodiment described above. Here, the destination and specific area may be automatically set to a value close to the initial value, for example, using the position and orientation of the information processing device 100 at the start of authentication processing as the initial value.

[0079] <Variation 6> In the second embodiment and the fifth modification, the authentication target NT needs to move from an initial position while holding the information processing device 100-2. Therefore, if it is assumed that the authentication target NT moves by walking, the fact that the authentication target NT is walking while moving may be added to the determination targets of the action authentication process to increase the reliability of the action authentication process. For example, in step S107 of FIG. 4, the action authentication unit 136 analyzes data such as acceleration and angular velocity acquired from the sensor 123, which is a detection unit, to determine whether the authentication target NT is walking. If the action authentication unit 136 detects that the authentication target NT is not walking within a predetermined section, it may determine that the action authentication has failed and proceed to step S109 of FIG. 4. Here, the predetermined section may preferably be a section from the initial position of the authentication target NT holding the information processing device 100-2 to the destination, or a section of a specific area through which the authentication target NT holding the information processing device 100-2 passes.

[0080] <Variation 7> In the first embodiment, the second embodiment, and Modification 5 described above, examples of operation instructions include "move the device so that a specific mark continues to be captured," "move to a specific destination," and "pass through a specific area." However, in the present disclosure, other operation instructions may be given. For example, operation instructions may include "walk straight ahead" or "walk around once." In this case, for example, in step S107 of FIG. 4, the acceleration and angular velocity acquired from the sensor 123 and the position and orientation data of the information processing device 100 measured in step S104 may be analyzed by Pedestrian Dead Reckoning (PDR). Then, the operation authentication in step S107 of FIG. 4 may be performed by determining whether the user walked straight ahead or whether the walking trajectory has a closed loop.

[0081] <Variation 8> In the first and second embodiments described above, face authentication processing is repeatedly performed in step S105 of FIG. 4. Furthermore, in step S106 of FIG. 4, the success / failure of face authentication is determined based on the result of face authentication processed in step S105. However, if the information processing performance of the information processing device 100 is low, it may be difficult to perform face authentication processing in step S105 of FIG. 4. Therefore, as an alternative method, only face images may be stored in step S105 of FIG. 4, and face authentication processing may be performed in step S106 using the face images stored in step S105. For example, the same processing as step S106 in the first embodiment described above may be performed using the results of face authentication processing on all stored face images. Furthermore, to further reduce the amount of calculation, some face images may be selected from the stored face images and the results of face authentication processing may be used.

[0082] (Other embodiments) The present disclosure can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. This program and a computer-readable storage medium storing the program are included in the present disclosure.

[0083] It should be noted that the above-described embodiments of the present disclosure are merely examples of specific embodiments for carrying out the present disclosure, and the technical scope of the present disclosure should not be construed as being limited by these embodiments. In other words, the present disclosure can be carried out in various forms without departing from its technical concept or main features.

[0084] The disclosure of this embodiment includes the following configuration, method, and program. [Configuration 1] An information processing device that authenticates an authentication target, an acquisition means for acquiring a face image including a face of the person to be authenticated; a face authentication means for performing face authentication of the person to be authenticated using the face image; an operation instruction means for instructing the person to be authenticated to perform an operation while holding the information processing device; an action authentication means for performing action authentication of the person to be authenticated based on the action instructed by the action instructing means and the actual action of the information processing device; a determination means for determining that authentication of the person to be authenticated is successful when the face authentication and the behavior authentication are both successful, and determining that authentication of the person to be authenticated is unsuccessful when at least one of the face authentication and the behavior authentication is unsuccessful; An information processing device comprising: [Configuration 2] When the face authentication is successful and the behavior authentication is unsuccessful, the determination means determines that an act of masquerading as the person to be authenticated has been performed. 2. The information processing device according to configuration 1, [Configuration 3] The operation authentication means determines that the operation authentication has been successful when the operation instructed by the operation instruction means matches the actual operation of the information processing device. 3. The information processing device according to configuration 1 or 2. [Configuration 4] further comprising a measuring means for measuring the position and orientation of the information processing device; The motion authentication means detects the actual motion of the information processing device based on the position and orientation of the information processing device measured by the measurement means. 4. The information processing device according to any one of configurations 1 to 3. [Configuration 5] The measurement of the position and orientation of the information processing device by the measurement means and the face authentication by the face authentication means are processed in parallel. 5. The information processing device according to configuration 4. [Configuration 6] After the measurement means measures the position and orientation of the information processing device, the face authentication means performs the face authentication. 5. The information processing device according to configuration 4. [Configuration 7] The measuring means measures a relative position and orientation of the information processing device based on an initial position and orientation of the information processing device. 7. The information processing device according to any one of configurations 4 to 6. [Configuration 8] The measuring means measures the absolute position and orientation of the information processing device based on one coordinate system. 7. The information processing device according to any one of configurations 4 to 6. [Configuration 9] further comprising a display means for displaying a mark based on the position and orientation of the information processing device; The operation instruction means instructs the information processing device to perform the operation so that the mark continues to be displayed on the display means for the person to be authenticated. 9. The information processing device according to any one of configurations 1 to 8. [Configuration 10] The mark is a virtual mark located in three-dimensional space or a mark based on a real mark. 10. The information processing device according to configuration 9. [Configuration 11] the mark is based on a real mark arranged in three-dimensional space, further comprising an imaging means for imaging the actual mark, the display means displays the actual mark captured by the imaging means, The operation instruction means instructs the information processing device to perform the operation so that the real mark continues to be displayed on the display means for the person to be authenticated. 11. The information processing device according to configuration 9 or 10. [Configuration 12] There are multiple genuine marks, The operation instruction means instructs the information processing device to perform the operation so that each of the plurality of real marks is displayed on the display means at least once within a certain period of time for the person to be authenticated. 12. The information processing device according to configuration 11. [Configuration 13] There are multiple genuine marks, The operation instruction means instructs the information processing device to perform the operation so that the real marks among the plurality of real marks are displayed in a predetermined order on the display means for the person to be authenticated. 12. The information processing device according to configuration 11. [Configuration 14] The display means displays a composite image in which a virtual environment and the mark are combined using VR (Virtual Reality). 14. The information processing device according to any one of configurations 9 to 13. [Configuration 15] further comprising an imaging means for imaging a real environment; The display means displays a composite image obtained by combining the real environment captured by the imaging means with the mark using AR (Augmented Reality). 14. The information processing device according to any one of configurations 9 to 13. [Configuration 16] further comprising a display means for displaying a current position and a destination of the information processing device; The operation instruction means instructs the authentication subject to perform the operation so that the information processing device moves to the destination. 4. The information processing device according to any one of configurations 1 to 3. [Configuration 17] The operation instruction means instructs the authentication subject to perform the operation so that the information processing device passes through a specific area. 4. The information processing device according to any one of configurations 1 to 3. [Configuration 18] further comprising a detection means for detecting whether the person to be authenticated who is holding the information processing device is walking or not; The motion authentication means determines that the motion authentication has failed when the detection means detects that the person to be authenticated is not walking in a predetermined section. 18. The information processing device according to any one of configurations 1 to 17. [Configuration 19] The predetermined section is a section from the initial position of the person to be authenticated who is holding the information processing device to a destination, or a section of a specific area through which the person to be authenticated who is holding the information processing device passes. 19. The information processing device according to configuration 18. [Method 1] A method for controlling an information processing device that authenticates an authentication target, comprising: an acquisition step of acquiring a face image including a face of the person to be authenticated; a face authentication step of performing face authentication of the person to be authenticated using the face image; an operation instruction step of instructing the person to be authenticated to perform an operation while holding the information processing device; an operation authentication step of performing operation authentication of the authentication subject based on the operation instructed in the operation instruction step and an actual operation of the information processing device; a determination step of determining that authentication of the person to be authenticated is successful if the face authentication and the behavior authentication are successful, and determining that authentication of the person to be authenticated is unsuccessful if at least one of the face authentication and the behavior authentication is unsuccessful; 1. A method for controlling an information processing device, comprising: [Program 1] 20. A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 19. [Explanation of symbols]

[0085] 100: Information processing device, 101: Appearance of the inner surface of the information processing device, 102: Appearance of the outer surface of the information processing device, 111: Inner surface camera, 112: Display, 113: Sensor, 114: Outer surface camera, 131: Image acquisition unit, 132: Position and orientation measurement unit, 133: Action instruction unit, 134: Face detection unit, 135: Face authentication unit, 136: Action authentication unit, 137: Authentication determination unit, 138: Image generation unit, NT: Person to be authenticated

Claims

1. An information processing device that authenticates an authentication target, an acquisition means for acquiring a face image including a face of the person to be authenticated; a face authentication means for performing face authentication of the person to be authenticated using the face image; an operation instruction means for instructing the person to be authenticated to perform an operation while holding the information processing device; an action authentication means for performing action authentication of the person to be authenticated based on the action instructed by the action instructing means and the actual action of the information processing device; a determination means for determining that authentication of the person to be authenticated is successful when the face authentication and the behavior authentication are both successful, and determining that authentication of the person to be authenticated is unsuccessful when at least one of the face authentication and the behavior authentication is unsuccessful; An information processing device comprising:

2. When the face authentication is successful and the behavior authentication is unsuccessful, the determination means determines that an act of masquerading as the person to be authenticated has been performed.

2. The information processing apparatus according to claim 1, wherein:

3. The operation authentication means determines that the operation authentication has been successful when the operation instructed by the operation instruction means matches the actual operation of the information processing device.

2. The information processing apparatus according to claim 1, wherein:

4. further comprising a measuring means for measuring the position and orientation of the information processing device; The motion authentication means detects the actual motion of the information processing device based on the position and orientation of the information processing device measured by the measurement means.

2. The information processing apparatus according to claim 1, wherein:

5. The measurement of the position and orientation of the information processing device by the measurement means and the face authentication by the face authentication means are processed in parallel.

5. The information processing apparatus according to claim 4,

6. After the measurement means measures the position and orientation of the information processing device, the face authentication means performs the face authentication.

5. The information processing apparatus according to claim 4,

7. The measuring means measures a relative position and orientation of the information processing device based on an initial position and orientation of the information processing device.

5. The information processing apparatus according to claim 4,

8. The measuring means measures the absolute position and orientation of the information processing device based on one coordinate system.

5. The information processing apparatus according to claim 4,

9. further comprising a display means for displaying a mark based on the position and orientation of the information processing device; The operation instruction means instructs the information processing device to perform the operation so that the mark continues to be displayed on the display means for the person to be authenticated.

2. The information processing apparatus according to claim 1, wherein:

10. The mark is a virtual mark arranged in three-dimensional space or a mark based on a real mark.

10. The information processing apparatus according to claim 9,

11. the mark is based on a real mark arranged in three-dimensional space, further comprising an imaging means for imaging the actual mark, the display means displays the actual mark captured by the imaging means, The operation instruction means instructs the information processing device to perform the operation so that the real mark continues to be displayed on the display means for the person to be authenticated.

10. The information processing apparatus according to claim 9,

12. There are multiple genuine marks, The operation instruction means instructs the information processing device to perform the operation so that each of the plurality of real marks is displayed on the display means at least once within a certain period of time for the person to be authenticated.

12. The information processing apparatus according to claim 11,

13. There are multiple genuine marks, The operation instruction means instructs the information processing device to perform the operation so that the real marks among the plurality of real marks are displayed in a predetermined order on the display means for the person to be authenticated.

12. The information processing apparatus according to claim 11,

14. The display means displays a composite image in which a virtual environment and the mark are combined using VR (Virtual Reality).

10. The information processing apparatus according to claim 9,

15. further comprising an imaging means for imaging a real environment; The display means displays a composite image obtained by combining the real environment captured by the imaging means with the mark using AR (Augmented Reality).

10. The information processing apparatus according to claim 9,

16. further comprising a display means for displaying a current position and a destination of the information processing device; The operation instruction means instructs the authentication subject to perform the operation so that the information processing device moves to the destination.

2. The information processing apparatus according to claim 1, wherein:

17. The operation instruction means instructs the authentication subject to perform the operation so that the information processing device passes through a specific area.

2. The information processing apparatus according to claim 1, wherein:

18. further comprising a detection means for detecting whether the person to be authenticated who is holding the information processing device is walking or not; The motion authentication means determines that the motion authentication has failed when the detection means detects that the person to be authenticated is not walking in a predetermined section.

2. The information processing apparatus according to claim 1, wherein:

19. The predetermined section is a section from the initial position of the person to be authenticated who is holding the information processing device to a destination, or a section of a specific area through which the person to be authenticated who is holding the information processing device passes.

19. The information processing apparatus according to claim 18,

20. A method for controlling an information processing device that authenticates an authentication target, comprising: an acquisition step of acquiring a face image including a face of the person to be authenticated; a face authentication step of performing face authentication of the person to be authenticated using the face image; an operation instruction step of instructing the person to be authenticated to perform an operation while holding the information processing device; an operation authentication step of performing operation authentication of the authentication subject based on the operation instructed in the operation instruction step and an actual operation of the information processing device; a determination step of determining that authentication of the person to be authenticated is successful if the face authentication and the behavior authentication are successful, and determining that authentication of the person to be authenticated is unsuccessful if at least one of the face authentication and the behavior authentication is unsuccessful; 1. A method for controlling an information processing device, comprising:

21. A program for causing a computer to function as each of the means of the information processing apparatus according to any one of claims 1 to 19.

Images