Information processing device, information processing method, and recording medium

Abstract

To provide an information processing device, information processing method, and recording medium, which allow for appropriately acquiring biometric information according to motion of a user.SOLUTION: An information processing device is provided, comprising a motion instruction unit configured to instruct a user to perform a motion, an information acquisition unit configured to acquire biometric information of the user from the user instructed to perform the motion, and a display control unit configured to control display for the user according to the state of the user performing the motion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a recording medium. [Background technology]

[0002] Patent Document 1 describes an authentication device that checks the existence of a user to confirm whether or not the user is a human. The authentication device described in Patent Document 1 checks the existence of a user by determining whether a response to a challenge, which is information used in authentication processing and is the basis for input by the user to be authenticated, is correct. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2015 / 194135 Summary of the Invention [Problem to be solved by the invention]

[0004] When determining whether or not impersonation has occurred using a photograph or the like based on the user's actions in response to specific instructions such as a challenge, as in the technology described in Patent Document 1, the user may not be able to act appropriately in response to the specific instructions. For this reason, the technology described in Patent Document 1 may have difficulty in appropriately collecting necessary information based on the user's actions.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide an information processing device, an information processing method, and a recording medium that are capable of appropriately acquiring necessary biometric information based on the user's movements. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information processing device having an operation instruction unit that instructs a user to perform an operation, an information acquisition unit that acquires biometric information of the user from the user to whom the operation has been instructed, and a display control unit that controls a display directed to the user depending on the operation status of the user.

[0007] According to another aspect of the present invention, there is provided an information processing method for instructing a user to perform an action, acquiring biometric information of the user from the user to whom the action has been instructed, and controlling a display directed to the user according to the user's operating status.

[0008] According to yet another aspect of the present invention, there is provided a recording medium having recorded thereon a program that causes a computer to instruct a user to perform an action, acquire biometric information of the user from the user to whom the action has been instructed, and control a display directed to the user in accordance with the user's operating status. [Effects of the Invention]

[0009] According to the present invention, necessary biometric information can be appropriately acquired based on the user's movements. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of an information processing device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart showing the operation of face authentication in the information processing device according to the first embodiment of the present invention. [Figure 3] FIG. 3 is a flowchart showing the operation of spoofing determination in the information processing device according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing the operation of face matching in the information processing device according to the first embodiment of the present invention. [Figure 5] FIG. 5 is a schematic diagram showing an example of a login screen in the information processing device according to the first embodiment of the present invention. [Figure 6]FIG. 6 is a schematic diagram showing an example of a face authentication screen at the start of acquiring a face image in the information processing device according to the first embodiment of the present invention. [Figure 7] FIG. 7 is a schematic diagram showing an example of a face authentication screen during acquisition of a face image in the information processing device according to the first embodiment of the present invention. [Figure 8] FIG. 8 is a schematic diagram showing an example of a face authentication screen when acquisition of a face image is completed in the information processing device according to the first embodiment of the present invention. [Figure 9] FIG. 9 is a schematic diagram showing an example of a logged-in screen after successful face authentication in the information processing device according to the first embodiment of the present invention. [Figure 10] FIG. 10 is a schematic diagram showing an example of a face authentication screen when face authentication fails in the information processing device according to the first embodiment of the present invention. [Figure 11] FIG. 11 is a block diagram showing the configuration of an information processing device according to the second embodiment of the present invention. [Figure 12] FIG. 12 is a block diagram showing the configuration of an information processing device according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] [First embodiment] An information processing apparatus and an information processing method according to a first embodiment of the present invention will be described with reference to FIGS.

[0012] First, the configuration of the information processing device according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device according to this embodiment.

[0013] 1, an information processing device 10 according to this embodiment includes a CPU (Central Processing Unit) 12, a RAM (Random Access Memory) 14, a storage unit 16, an input unit 18, a display unit 20, an image capturing unit 22, an audio output unit 24, and a communication unit 26. The CPU 12, RAM 14, storage unit 16, input unit 18, display unit 20, image capturing unit 22, audio output unit 24, and communication unit 26 are connected to a common bus 28.

[0014] The information processing device 10 is not particularly limited, but may be, for example, a smartphone. The information processing device 10 may also be a tablet personal computer, a mobile phone, or the like. The information processing device 10 may also be, for example, a computer device such as a laptop or desktop personal computer. The information processing device 10 can execute various application programs in response to execution instructions from a user who uses the information processing device 10.

[0015] The CPU 12 operates by executing a program stored in the storage unit 16, and functions as a control unit that controls the overall operation of the information processing device 10. The CPU 12 also executes the program stored in the storage unit 16 to perform various processes as the information processing device 10. The RAM 14 provides a memory area necessary for the operation of the CPU 12.

[0016] The information processing device 10 according to this embodiment performs face authentication on a user when logging in to a specific application program stored in the storage unit 16. Furthermore, the information processing device 10 is configured to be able to determine whether or not a non-biometric data such as a photograph or video is impersonating the user during face authentication.

[0017] The information processing device 10 can perform similar facial authentication at various times, in addition to when logging in to a specific application program. For example, the information processing device 10 can perform similar facial authentication on the user when the information processing device 10 starts to be used, such as when the system of the information processing device 10 is started or when the information processing device 10 is unlocked. Furthermore, for example, the information processing device 10 can perform similar facial authentication on the user when access to a specific resource, such as a specific file, directory, or folder stored in the storage unit 16, etc. is initiated. In this way, the information processing device 10 can be configured to perform facial authentication on the user requesting a specific process at the start of the process.

[0018] The CPU 12 executes a specific application program stored in the storage unit 16, thereby functioning as the following functional units to perform face authentication for a user: an authentication processing unit 12a, an operation instruction unit 12b, a face image acquisition unit 12c, an information providing unit 12d, an impersonation determination unit 12e, and a face matching unit 12f.

[0019] The authentication processing unit 12a accepts a login request from a user by displaying a login screen on the display unit 20, which requests facial authentication of the user. A user who is the target of facial authentication can perform touch input by, for example, pressing a specific area, such as a login button or a login icon, on the login screen displayed on the display unit 20, which is configured as a touch panel display integrated with the input unit 18. This allows the user to input a login request to the information processing device 10. The user can also input a login request to the information processing device 10 by, for example, having the imaging unit 22 capture an image of their face and inputting the facial image into the information processing device 10. When the user inputs a login request, the authentication processing unit 12a causes the display unit 20 to display a facial authentication screen for performing facial authentication.

[0020] Furthermore, the authentication processing unit 12a performs face authentication to authenticate the user based on the result of the spoofing determination by the spoofing determination unit 12e and the result of the face matching by the face matching unit 12f. That is, the authentication processing unit 12a determines whether or not the face authentication of the user has been successful based on the result of the spoofing determination and the result of the face matching, and executes processing according to the determination result.

[0021] As will be described later, when the impersonation determination unit 12e determines that there is no impersonation and the face matching unit 12f determines that the face matching matches, the authentication processing unit 12a determines that the face authentication of the user is successful. When it is determined that the face authentication of the user is successful, the authentication processing unit 12a permits the user to log in to a specific application program and executes a login process to log in the user.

[0022] On the other hand, if the impersonation determination unit 12e determines that the user is an impersonator, or if the face matching unit 12f determines that the face matching does not match, the authentication processing unit 12a determines that the user's face authentication was not successful, i.e., that the user's face authentication has failed. If it is determined that the user's face authentication has failed, the authentication processing unit 12a denies the user's login to the application program. In this case, the authentication processing unit 12a can execute failure processing, such as processing to notify the user that the face authentication has failed. For example, the authentication processing unit 12a can notify the user by displaying a message on the display unit 20 indicating that the face authentication has failed.

[0023] Upon receiving a login request from the user, the action instructing unit 12b instructs the user, who is the subject of face authentication, to perform a specific action related to the face photographed by the photographing unit 22. The action instructing unit 12b can instruct the user to perform a specific action by displaying a display instructing the specific action on the display unit 20. The action instructing unit 12b can display, on the display unit 20, for example, an instruction sentence instructing the specific action, a visual symbol that animates to perform a specific action, or the like, as a display instructing the specific action. The visual symbol is, for example, a face icon that resembles a face, a pictogram, or the like.

[0024] The specific action instructed by the action instructing unit 12b is not particularly limited, and may be, for example, an action of shaking the face left and right or up and down, an action of turning the face, etc. When instructing an action of shaking the face as the specific action, the action instructing unit 12b may, for example, display an instruction message such as "Please shake your head" on the display unit 20, or display an animated face icon that moves as if shaking the head on the display unit 20.

[0025] The action instructing unit 12b can also function as a display control unit that controls and changes the display on the display unit 20 that instructs the user to perform a specific action, in accordance with the action status of the user captured by the capturing unit 22. For example, the action instructing unit 12b can control and change the display on the display unit 20 that instructs the user to perform a specific action, in accordance with the determination result by the face image acquiring unit 12c, which will be described below, as to whether the specific action of the user is appropriate.

[0026] For example, specifically, when instructing a head shaking action as a specific action, the action instructing unit 12b can control and change the display instructing the specific action as follows.

[0027] When the face image acquisition unit 12c determines that the user's face shaking action is appropriate, the action instruction unit 12b can, for example, stop the face shaking action of the face icon displayed on the display unit 20. Furthermore, when it is determined that the action is appropriate, the action instruction unit 12b can also change the display of the face icon to, for example, a positive expression that suggests that the specific action of the user is appropriate.

[0028] On the other hand, if the face image acquisition unit 12c determines that the user's head-waving action is inappropriate, the action instructing unit 12b can continue the head-waving action of the face icon displayed on the display unit 20, or can make the head-waving action of the face icon larger. Also, if it is determined that it is inappropriate, the action instructing unit 12b can change the display of the face icon to, for example, a negative expression that suggests that the specific user action is inappropriate. Also, if it is determined that it is inappropriate, the action instructing unit 12b can highlight the instruction message "Please wave your head" on the display unit 20, or can display the instruction message "Please wave your head vigorously" on the display unit 20.

[0029] In this way, when it is determined that the specific action of the user is appropriate, the action instructing unit 12b can control and change the display instructing the specific action so as to indicate that the specific action of the user is appropriate. Also, when it is determined that the specific action of the user is inappropriate, the action instructing unit 12b can control and change the display instructing the specific action so as to indicate that the specific action of the user is inappropriate.

[0030] Additionally, the action instructing unit 12b can instruct the user to perform a specific action by outputting a voice of the action instruction from the voice output unit 24, in addition to or instead of displaying the action instruction on the display unit 20. In this case, the action instructing unit 12b can output a voice from the voice output unit 24 reading out an instruction sentence instructing the user to perform a specific action, such as "Wave your head."

[0031] The face image acquiring unit 12c is an information acquiring unit that acquires a face image, which is biometric information of the user, from a moving image captured by the imaging unit 22. The face image acquiring unit 12c operates as follows to acquire a face image.

[0032] The facial image acquisition unit 12c determines whether a specific movement of the user's face in a video captured by the image capture unit 22 is appropriate. The facial image acquisition unit 12c determines that the specific movement of the user is appropriate if the specific movement of the user is one that can acquire multiple facial images from which the three-dimensional shape of the user's face can be estimated when determining whether the user is impersonating someone. On the other hand, the facial image acquisition unit 12c determines that the specific movement of the user is inappropriate if the specific movement of the user is one from which multiple facial images cannot be acquired. For example, if the user's movement is too small to acquire multiple facial images from multiple viewpoints, or if the user's face is not even captured by the image capture unit 22, it is not possible to acquire multiple facial images from which the three-dimensional shape of the user's face can be estimated. Therefore, in these cases, the facial image acquisition unit 12c determines that the specific movement of the user is inappropriate.

[0033] Note that the facial image acquiring unit 12c may determine that the user's motion is appropriate even if the specific motion instructed by the motion instructing unit 12b is not identical to the user's motion. For example, even if the motion instructing unit 12b displays an animation of shaking one's head from side to side and the user shakes their head vertically, the user's motion may be an action that can acquire multiple facial images from which the three-dimensional shape of the user's face can be estimated when determining whether or not the user is an impersonator. In this case, the facial image acquiring unit 12c can determine that the user's motion is appropriate.

[0034] When the facial image acquisition unit 12c determines that a specific user action is appropriate, it detects a facial image of the user in the video captured by the image capture unit 22. Furthermore, the facial image acquisition unit 12c acquires the detected facial image of the user as facial information of the user. At this time, the facial image acquisition unit 12c evaluates the quality of the detected facial image and acquires a high-quality facial image having a quality equal to or higher than a predetermined quality. For example, the facial image acquisition unit 12c can evaluate the quality of the facial image based on quality values ​​calculated for a predetermined number of facial feature points set in advance to estimate the three-dimensional shape of the face. The facial image acquisition unit 12c does not necessarily need to acquire the facial image itself as facial information; it may also acquire the facial information by extracting features from the facial image.

[0035] The facial image acquisition unit 12c repeatedly acquires facial images of the user in the manner described above until a time limit, which is a predetermined time from the time when a specific action is instructed by the action instructing unit 12b, has elapsed. In this way, the facial image acquisition unit 12c attempts to acquire a specified number of facial images required for spoofing determination. The time limit for acquiring facial images is set in advance depending on, for example, the required accuracy of spoofing determination, the time required to complete authentication, etc. The specified number of facial images required for spoofing determination is also set in advance depending on, for example, the required accuracy of spoofing determination, the time required to complete authentication, etc.

[0036] The facial image acquisition unit 12c can also function as a display control unit that displays the progress of facial image acquisition on the display unit 20 as a display according to the user's operating status. The facial image acquisition unit 12c can display, for example, the facial image acquisition rate, which is the ratio of the number of acquired facial images to a specified number required for impersonation determination, as a percentage on the display unit 20 as the progress of facial image acquisition according to the user's operating status. The facial image acquisition unit 12c can display the facial image acquisition rate, for example, as a numerical value displayed as a percentage, or as a gauge or progress bar whose length changes according to the facial image acquisition rate. The shape of the gauge or progress bar indicating the facial image acquisition rate is not particularly limited, and can be, for example, a bar, annular, or frame shape. The facial image acquisition unit 12c controls and changes the display indicating the progress of facial image acquisition, which is displayed to the user, according to the facial image acquisition status. The facial image acquisition unit 12c can display a display indicating the progress of facial image acquisition, such as a gauge or progress bar, in accordance with the operating status of the user, on the display unit 20. Note that the display format indicating the progress of facial image acquisition is not limited to a gauge or progress bar, and various display formats can be adopted.

[0037] When the facial image acquisition unit 12c is unable to acquire the specified number of facial images and the time limit has elapsed, i.e., when a specified amount of biometric information has not been acquired within a specified time, the information providing unit 12d provides guidance information to the user. The guidance information is information that serves as a reference, guide, or help for the facial image acquisition unit 12c to successfully acquire the specified number of facial images when performing facial authentication again. The information providing unit 12d can provide the guidance information to the user by, for example, displaying it on the display unit 20. The information providing unit 12d can also provide the guidance information to the user by, for example, outputting it as audio from the audio output unit 24.

[0038] The information providing unit 12d can provide the user with guidance information according to the cause of the inability of the facial image acquiring unit 12c to acquire a facial image. For example, if a facial image cannot be acquired because the user's behavior is inappropriate, the information providing unit 12d can provide guidance information regarding the user's specific behavior. Furthermore, for example, if a facial image cannot be acquired because the authentication environment in which facial authentication is performed is inappropriate, the information providing unit 12d can provide guidance information regarding the authentication environment. Furthermore, the guidance information regarding the user's specific behavior and the guidance information regarding the authentication environment can be provided to the user in combination.

[0039] The information providing unit 12d can provide guidance information regarding a specific user action, which guides the user to perform the specific action appropriately. For example, when a user is instructed to perform a head-shaking action as a specific action, the information providing unit 12d can display a guidance statement regarding the head-shaking action on the display unit 20 or output it by voice from the voice output unit 24 as guidance information. In this case, examples of the guidance statement include "Shake your head widely," "Shake your head slowly," "Your movement is too fast," etc.

[0040] Furthermore, the information providing unit 12d can provide guidance information relating to the authentication environment, which guides the user to perform face authentication in an appropriate authentication environment. The authentication environment for which guidance information can be provided is not particularly limited, and examples thereof include the brightness of the authentication location where authentication is performed, and the shooting distance at which the user is photographed by the photographing unit 22.

[0041] For example, if the brightness of the authentication location is too bright or too dark and the facial image acquisition unit 12c cannot acquire a facial image of a certain quality or higher, the information providing unit 12d can provide guidance information regarding the brightness of the authentication location as the guidance information. In this case, the information providing unit 12d can display guidance statements such as "It's too bright" or "It's too dark" on the display unit 20 or output them as voice from the voice output unit 24 as the guidance information.

[0042] Furthermore, for example, when the photographing distance by the photographing unit 22 is too far or too close and the facial image acquiring unit 12c cannot acquire a facial image of a certain quality or higher, the information providing unit 12d can provide guidance information regarding the photographing distance by the photographing unit 22. In this case, the information providing unit 12d can display guidance statements such as "Too far" or "Too close" on the display unit 20 or output them as voice from the voice output unit 24 as guidance information.

[0043] The information providing unit 12d can provide guidance information by displaying a visual symbol that is associated with the content of the guidance statement on the display unit 20 instead of or together with the guidance statement. The visual symbol is, for example, an icon, a pictogram, or the like.

[0044] The spoofing determination unit 12e performs spoofing determination based on the multiple face images acquired by the face image acquisition unit 12c. In the spoofing determination, the spoofing determination unit 12e determines whether the user's face is an spoofed image based on a non-biological object such as a photograph or video.

[0045] When making a spoofing determination, the spoofing determination unit 12e estimates the three-dimensional shape of the user's face based on multiple face images acquired by the face image acquisition unit 12c. The multiple face images acquired by the face image acquisition unit 12c are multi-viewpoint images taken from different viewpoints because they are acquired by capturing images of the user's face performing a specific action using the imaging unit 22. The spoofing determination unit 12e estimates the three-dimensional shape of the user's face from the multiple face images, which are multi-viewpoint images. The method for estimating the three-dimensional shape is not particularly limited, and various methods can be used. For example, the spoofing determination unit 12e can estimate the three-dimensional shape of the user's face from the multiple face images acquired by the face image acquisition unit 12c using a bundle adjustment method.

[0046] When estimating the three-dimensional shape, the spoofing determination unit 12e can perform, for example, the following processing. That is, for each of the multiple face images acquired by the face image acquisition unit 12c, the spoofing determination unit 12e acquires facial information indicating the positions of a predetermined number of facial feature points in the face image. The facial feature points for which facial information is acquired are, for example, two or more, specifically, ten facial feature points: the right eye, the left eye, the nose, the center of the mouth, the right corner of the mouth, the left corner of the mouth, the right cheek, the left cheek, the right chin, and the left chin. Furthermore, the spoofing determination unit 12e associates the predetermined number of facial feature points among the multiple acquired face images based on the facial information of each of the multiple acquired face images. The spoofing determination unit 12e can estimate the three-dimensional shape of the user's face based on the result of the association of the facial feature points. According to this processing, the predetermined number of facial feature points can be uniquely associated among the multiple face images based on the facial information of each of the multiple face images. This correspondence does not require complicated processing, so it is possible to more easily and accurately correspond facial feature points between multiple facial images, and more easily and accurately estimate three-dimensional shapes.

[0047] The spoofing determination unit 12e evaluates the estimated three-dimensional shape of the user's face to determine whether the user's face is a spoof based on a non-biological object such as a photograph or video. For example, the spoofing determination unit 12e can determine whether the estimated three-dimensional shape is three-dimensional, i.e., whether it is three-dimensional or planar, to determine whether it is a spoof. In this case, the spoofing determination unit 12e can evaluate whether the three-dimensional shape is three-dimensional or planar by determining the magnitude relationship between an evaluation amount related to the estimated three-dimensional shape and a preset threshold. The evaluation amount is not particularly limited, and for example, a distance, angle, etc. related to a specific facial feature point, or an amount based on these, can be used. Note that the method for evaluating whether a three-dimensional shape is three-dimensional or planar is not particularly limited, and various methods can be used.

[0048] If the spoofing determination unit 12e evaluates that the estimated three-dimensional shape of the face is three-dimensional, it determines that the user's face is a biological face and not an spoof. On the other hand, if the spoofing determination unit 12e evaluates that the estimated three-dimensional shape is planar, it determines that the user's face is an spoof using a non-biological object such as a photograph or video.

[0049] The face matching unit 12f performs face matching based on one or more face images acquired by the face image acquisition unit 12c. In face matching, the face matching unit 12f compares a target face image, which is a face image of the user captured by the photographing unit 22 and acquired by the face image acquisition unit 12c, with a registered face image, which is a face image of the user registered in advance, and determines whether the two match. For example, the face matching unit 12f can compare the highest quality face image, such as a face image facing forward, among the multiple face images acquired by the face image acquisition unit 12c with the registered face image as the target face image. The face matching unit 12f can also compare each of the multiple face images with the registered face image. Note that the registered face image, which is the registered biometric information of the user, is pre-stored and registered in the storage unit 16, for example.

[0050] When matching both face images, the face matching unit 12f can calculate a matching score indicating the degree of similarity between the feature amounts of the target face image and the feature amounts of the registered face image. The higher the similarity between the feature amounts of both face images, the larger the matching score value. When matching each of multiple face images with a registered face image, the matching score can be the average value, maximum value, or the like of the matching scores calculated during each matching. When the matching score is equal to or greater than a predetermined threshold, the face matching unit 12f can determine that the matching between the target face image and the registered face image matches, i.e., the face matching matches. On the other hand, when the matching score is less than the predetermined threshold, the face matching unit 12f can determine that the matching between the target face image and the registered face image does not match, i.e., the face matching does not match.

[0051] Before the face image acquiring unit 12c acquires the specified number of face images, the face matching unit 12f can compare an already acquired face image as a target face image with the registered face images in the same manner as described above. In this case, when the face matching unit 12f determines that the matching does not match, the face image acquiring unit 12c can stop acquiring face images even before the specified number of face images are acquired.

[0052] Note that some or all of the functions of the CPU 12 described above do not necessarily need to be realized by the CPU 12 of the information processing device 10, which is a single device, but may be realized by another external device such as a server. For example, among the functions of the CPU 12 described above, functions such as the spoofing determination unit 12e and the face matching unit 12f may be realized by the CPU of a server connected to the information processing device 10 via a network so as to be able to communicate with the information processing device 10. In this case, the CPU 12 of the information processing device 10 transmits processing requests for the functions of the spoofing determination unit 12e, the face matching unit 12f, etc., to the server via the network, as well as data such as facial images and feature amounts extracted from the facial images required for such processing. The CPU 12 also receives processing results for the functions of the spoofing determination unit 12e, the face matching unit 12f, etc. from the server via the network.

[0053] The storage unit 16 is configured with a storage medium such as a nonvolatile memory, such as a flash memory or a hard disk drive. The storage unit 16 stores programs, such as specific application programs, executed by the CPU 12, and data referenced by the CPU 12 when executing the programs. For example, the storage unit 16 stores a facial image of the user acquired from an image captured by the imaging unit 22. The storage unit 16 also stores, for example, a registered facial image of the user used for face matching.

[0054] The input unit 18 accepts input of information, instructions, etc. from the user to the information processing device 10. The user can input various types of information and instructions to execute processing to the information processing device 10 via the input unit 18. The input unit 18 is configured, for example, by a touch panel incorporated in the display unit 20. Note that the input unit 18 may also be configured, for example, by a keyboard, a mouse, etc.

[0055] The display unit 20 displays various screens such as an execution screen of a specific application program under the control of the CPU 12. The execution screen of a specific application program includes, for example, a login screen, a face authentication screen, a logged-in screen, etc. The display unit 20 is configured, for example, by a liquid crystal display, an OLED (Organic Light Emitting Diode) display, etc., and is configured as a touch panel display together with the input unit 18.

[0056] The photographing unit 22 photographs an image including the face of a user who is to undergo face authentication. The photographing unit 22 is configured, for example, by a digital camera capable of photographing moving images, and acquires two-dimensional images that constitute the moving images at a predetermined frame rate.

[0057] The communication unit 26 transmits and receives data to and from external devices such as a server via a network under the control of the CPU 12. The communication standard, method, etc. of the communication unit 26 are not particularly limited, and may be a wireless method or a wired method.

[0058] Thus, the information processing device 10 according to this embodiment is configured.

[0059] In the information processing device 10 according to this embodiment, the action instructing unit 12b controls and changes the display on the display unit 20 instructing a specific action in accordance with the user's action status captured by the image capturing unit 22. This allows the user to determine whether their own action in response to the specific action instruction is appropriate and to appropriately modify their own action. Furthermore, the facial image acquiring unit 12c causes the display unit 20 to display the progress of facial image acquisition. The user can also use the progress of facial image acquisition as information to determine whether their own action is appropriate. Therefore, according to the information processing device 10 according to this embodiment, it is possible to make the user act appropriately in accordance with the specific action instruction and to appropriately acquire a facial image, which is biometric information necessary for determining whether or not the user is impersonating someone, based on the user's action.

[0060] Furthermore, in the information processing device 10 according to this embodiment, the information providing unit 12d provides guidance information when the facial image acquiring unit 12c fails to acquire the specified number of facial images. Therefore, according to the information processing device 10 according to this embodiment, when the user performs face authentication again, the user can take measures to enable the facial image acquiring unit 12c to acquire the specified number of facial images.

[0061] The operation of face authentication in the information processing device 10 according to this embodiment will be further described below with reference to Figs. 2 to 10. Fig. 2 is a flowchart showing the operation of face authentication in the information processing device 10. Fig. 3 is a flowchart showing the operation of spoofing determination in the information processing device 10. Fig. 4 is a flowchart showing the operation of face matching in the information processing device 10. Figs. 5 to 10 are each schematic diagrams showing an example of a series of screens on the display unit 20 during the operation of face authentication in the information processing device 10. The information processing method according to this embodiment is executed by performing the operation of face authentication in the information processing device 10.

[0062] First, when a specific application program is executed by the CPU 12, as shown in FIG. 2, the authentication processing unit 12a displays a login screen on the display unit 20 to request face authentication of the user, and accepts a login request from the user (step S102).

[0063] Fig. 5 shows an example of a login screen SL that requests facial authentication. As shown in Fig. 5, the login screen SL displays a face icon SL12 indicating that facial authentication is required for login. For example, by pressing the face icon SL12, the user can input a login request to the information processing device 10, requesting login to a specific application program.

[0064] The authentication processing unit 12a continuously determines whether or not a login request has been input by the user (step S104), and waits until a login request is input (step S104, NO).

[0065] When the authentication processing unit 12a determines that a login request has been input (step S104, YES), it causes the display unit 20 to display a face authentication screen for performing face authentication (step S106).

[0066] 6 shows an example of the face authentication screen SF at the start of face authentication, i.e., when the face image acquisition unit 12c starts acquiring a face image. As shown in FIG. 6, the face authentication screen SF includes a video display area SF12 that displays a video captured by the image capture unit 22 and an action instruction display area SF14 that displays instructions for a specific action from the action instructing unit 12b. The face authentication screen SF also includes a bar gauge SF16 that indicates the progress of face image acquisition, and box gauges SF18L and SF18R that also indicate the progress of face image acquisition. The face authentication screen SF also includes a "Use Password" button SF20 that switches the authentication method to the password method, and a "Cancel" button SF22 that cancels face authentication.

[0067] The moving image display area SF12 displays, in almost real time, a moving image including the user's face captured by the image capturing unit 22. By checking the moving image of the user's face displayed in the moving image display area SF12, the user can determine whether or not the user's own action is appropriate in response to a specific action instruction from the action instructing unit 12b.

[0068] The action instruction display area SF14 displays an instruction sentence SF142 instructing a specific action and a face icon SF144 instructing the specific action. The instruction sentence SF142 indicates the content of the specific action instructed by the action instruction unit 12b. The face icon SF144 performs an animation action in accordance with the specific action instructed by the action instruction unit 12b. The action instruction unit 12b instructs the user to perform a specific action using the instruction sentence SF142, such as "Wave your head." The action instruction unit 12b also instructs the user to perform a specific action using the face icon SF144, which performs an animation action, such as waving its head.

[0069] The length of the bar gauge SF16 changes according to the progress of the facial image acquisition by the facial image acquisition unit 12c. Specifically, the length of the bar gauge SF16 changes so as to extend from left to right according to the acquisition rate of facial images by the facial image acquisition unit 12c. The acquisition rate of facial images is displayed as a percentage within the bar gauge SF16.

[0070] The frame-shaped gauges SF18L and SF18R are arranged to form frame portions on the left and right halves of the moving image display area SF12, respectively. Similar to the bar-shaped gauge SF16, the lengths of the frame-shaped gauges SF18L and SF18R change according to the progress of facial image acquisition by the facial image acquisition unit 12c. Specifically, the lengths of the frame-shaped gauges SF18L and SF18R change so as to extend from bottom to top according to the acquisition rate of facial images by the facial image acquisition unit 12c.

[0071] In the frame that surrounds the moving image display area SF12, formed by the frame-shaped gauges SF18L and SF18R, a face icon SF144 is placed in the upper center, and a bar-shaped gauge SF16 is placed in the lower center. The frame-shaped gauges SF18L and SF18R each indicate a face image capture rate of 0% at the center of the bar-shaped gauge SF16 and a face image capture rate of 100% at the point where the face icon SF144 reaches.

[0072] The "Use Password" button SF20 is a button for switching the authentication method for logging in to a specific application program from the face authentication method to the password method. By pressing the "Use Password" button SF20, the user can switch the authentication method for logging in to the password method, which requires entering a password.

[0073] The "Cancel" button SF22 is a button for canceling face authentication for logging in to a specific application program. The user can cancel face authentication by pressing the "Cancel" button SF22.

[0074] 2, the operation instructing unit 12b instructs the user to perform a specific action by displaying an instruction message instructing the user to perform a specific action and a face icon that animates the specific action on the face authentication screen of the display unit 20 (step S108). The operation instructing unit 12b can instruct the user to perform a face shaking action, for example, by using an instruction message, a face icon, or the like.

[0075] When the action instructing unit 12b instructs the user to perform a specific action, the face image acquiring unit 12c attempts to acquire a specified number of face images of the user required for spoofing determination until the time limit has elapsed.

[0076] First, the face image acquisition unit 12c determines whether or not a particular action of the user in the moving image captured by the image capture unit 22 is appropriate (step S110).

[0077] If the facial image acquisition unit 12c determines that the user's specific action is inappropriate (step S110, NO), or if the time limit has not elapsed (step S112, NO), the action instruction unit 12b controls and changes the display instructing the specific action (step S114). In this case, the action instruction unit 12b can, for example, continue the head-shaking action of the face icon, make the head-shaking action of the face icon larger, or display an instruction message such as "Please shake your head" in an emphasized manner on the display unit 20. Thereafter, the facial image acquisition unit 12c continues to determine whether the user's specific action is appropriate (step S110). Note that if the time limit has elapsed (step S112, YES), the process proceeds to step S138, which will be described later.

[0078] On the other hand, if the face image acquisition unit 12c determines that the specific action of the user is appropriate (step S110, YES), the action instruction unit 12b controls and changes the display instructing the specific action (step S116). In this case, the action instruction unit 12b can, for example, stop the face icon from shaking its head, or change the display of the face icon to an expression such as a smile that suggests that the specific action of the user is appropriate.

[0079] When the facial image acquisition unit 12c determines that the specific action of the user is appropriate (step S110, YES), it detects a facial image of the user in the moving image captured by the image capture unit 22 (step S118).

[0080] Next, the facial image acquisition unit 12c evaluates the quality of the detected facial image (step S120). If the facial image acquisition unit 12c determines that the quality of the facial image is not equal to or higher than a predetermined quality (step S120, NO), the process proceeds to step S110 and attempts to acquire a facial image again.

[0081] On the other hand, when the face image acquisition unit 12c determines that the quality of the face image is equal to or higher than the predetermined quality (step S120, YES), it acquires a high-quality face image having a quality equal to or higher than the predetermined quality (step S122).

[0082] In response to acquiring the facial image, the facial image acquiring unit 12c controls and changes the display showing the progress of the facial image acquisition, such as a gauge showing the acquisition rate of the facial image, a progress bar, or a percentage value (step S124).

[0083] Fig. 7 shows an example of the face authentication screen SF while face images are being acquired by the face image acquisition unit 12c. Fig. 7 shows the face authentication screen SF when the face image acquisition rate is 50%. On the face authentication screen SF while face images are being acquired, as shown in Fig. 7, as the number of acquired face images increases, the bar gauge SF16 extends from left to right, and the frame gauges SF18L and SF18R extend from bottom to top. Furthermore, as the number of acquired face images increases, the numerical value of the face image acquisition rate in the bar gauge SF16 increases.

[0084] 2, the facial image acquisition unit 12c determines whether or not a predetermined number of facial images necessary for spoofing determination have been acquired (step S126). If it is determined that the predetermined number of facial images have not been acquired (step S126, NO), and if the time limit has not elapsed (step S128, NO), the facial image acquisition unit 12c proceeds to step S110 and attempts to acquire facial images again. If the time limit has elapsed (step S128, YES), the facial image acquisition unit 12c proceeds to step S138, which will be described later.

[0085] On the other hand, if the face image acquisition unit 12c determines that the specified number of face images has been acquired (step S126, YES), the impersonation determination unit 12e performs impersonation determination (step S130), and the face matching unit 12f performs face matching (step S132). Note that the impersonation determination by the impersonation determination unit 12e and the face matching by the face matching unit 12f may be performed either first or in parallel. Furthermore, the face matching unit 12f can also perform face matching using face images that have already been acquired before the specified number of face images are acquired by the face image acquisition unit 12c.

[0086] 8 shows an example of the face authentication screen SF when the face image acquisition unit 12c has completed acquisition of a specified number of face images. As shown in FIG. 8, on the face authentication screen SF when the face image acquisition is completed, the bar gauge SF16 and the frame gauges SF18L and SF18R are each extended to their maximum extent. The acquisition rate of face images within the bar gauge SF16 is 100%. On the face authentication screen SF when the face image acquisition is completed, a completion icon SF24 indicating the completion of face image acquisition is displayed in place of the face icon SF144, visually informing the user that the process is complete.

[0087] As shown in FIG. 3, the spoofing determination unit 12e first estimates the three-dimensional shape of the user's face based on the multiple face images acquired by the face image acquisition unit 12c (step S202).

[0088] Next, the spoofing determination unit 12e evaluates whether the estimated three-dimensional shape is stereoscopic or not (step S204).

[0089] If the estimated three-dimensional shape is evaluated as being stereoscopic (step S204, YES), the spoofing determination unit 12e determines that the user's face is a biological face and not an spoof (step S206).

[0090] On the other hand, if the estimated three-dimensional shape is evaluated as not being three-dimensional, i.e., being planar (step S204, NO), the spoofing determination unit 12e determines that the user's face has been spoofed using a photograph, video, etc. (step S208).

[0091] On the other hand, as shown in FIG. 4, the face matching unit 12f first selects a target face image to be used for matching with a registered face image from the plurality of face images acquired by the face image acquisition unit 12c (step S302).

[0092] Furthermore, the face matching unit 12f reads and acquires the registered face image with which the target face image is to be matched from the storage unit 16 or the like (step S304).

[0093] Next, the face matching unit 12f calculates a matching score indicating the degree of similarity between the feature amount of the target face image and the feature amount of the registered face image (step S306).

[0094] Next, the face matching unit 12f determines whether the calculated matching score is equal to or greater than a predetermined threshold value (step S308).

[0095] If the face matching unit 12f determines that the matching score is equal to or greater than the predetermined threshold (step S308, YES), it determines that the matching between the target face image and the registered face image matches, that is, that the face matching matches (step S310).

[0096] On the other hand, if the face matching unit 12f determines that the matching score is less than the predetermined threshold (step S308, NO), it determines that the matching between the target face image and the registered face image does not match, that is, the face matching does not match (step S312).

[0097] It should be noted that the face matching unit 12f can also execute the process shown in FIG. 4 using face images that have already been acquired, even before the face image acquisition unit 12c acquires the specified number of face images.

[0098] Next, as shown in FIG. 2, the authentication processing unit 12a determines whether or not the face authentication of the user has been successful based on the result of the spoofing determination by the spoofing determination unit 12e and the result of the face matching by the face matching unit 12f (step S134).

[0099] When it is determined that the user's face authentication has been successful (step S134, YES), the authentication processing unit 12a permits the user to log in to the specific application program and executes a login process to log the user in (step S136). The authentication processing unit 12a executes the login process and displays a logged-in screen for the specific application on the display unit 20. When the impersonation determination unit 12e determines that there is no impersonation (step S206) and the face matching unit 12f determines that the face matching results are a match (step S310), the authentication processing unit 12a determines that the face authentication has been successful.

[0100] 9 shows a logged-in screen SA of a specific application after successful face authentication. As shown in FIG. 9, the logged-in screen SA is the screen of the specific application in which the user is logged in.

[0101] On the other hand, if it is determined that the user's face authentication has failed (step S134, NO), the authentication processing unit 12a denies the user's login to the specific application program. The authentication processing unit 12a determines that the face authentication has failed when the impersonation determination unit 12e determines that the user is an impersonator (step S208) or when the face matching unit 12f determines that the face matching does not match (step S312). In this case, the authentication processing unit 12a executes failure processing, such as processing to notify the user that the face authentication has failed (step S140).

[0102] Unlike the case where the facial image acquisition unit 12c acquires a specified number of facial images and performs facial authentication as described above, there are cases where facial authentication fails as a result of the time limit elapsed without the specified number of facial images being acquired (step S112, YES or step S128, YES). In these cases, the information providing unit 12d provides guidance information regarding a specific user action or authentication environment depending on the cause of the failure of the facial image acquisition unit 12c to acquire facial images (step S138). The information providing unit 12d can provide the guidance information to the user by, for example, displaying it on the display unit 20 or outputting it as audio from the audio output unit 24. This can improve the probability that the facial image acquisition unit 12c will acquire the specified number of facial images within the time limit when facial authentication is performed again.

[0103] Next, the authentication processing unit 12a executes failure processing, such as processing to notify the user that face authentication has failed (step S140).

[0104] FIG. 10 shows the face authentication screen SF when face authentication fails because the specified number of face images have not been acquired and the time limit has elapsed. As shown in FIG. 10, on the face authentication screen SF when face authentication has failed, a failure icon SF26, such as an exclamation point icon, is displayed instead of the face icon SF144, indicating that face authentication has failed because the specified number of face images have not been acquired. Furthermore, a failure frame SF28, for example, a different color from the frame gauges SF18L and SF18R, is displayed around the video display area SF12, indicating that face authentication has failed. Furthermore, while the video display area SF12 displays a video captured by the image capture unit 22 in color when acquiring face images, it displays a frame of a specific point in time from the video captured by the image capture unit 22 in grayscale or monochrome when face authentication has failed. This indicates that face authentication has failed in the video display area SF12.

[0105] Furthermore, in the moving image display area SF12, a guidance statement SF30 is displayed as guidance information provided by the information providing unit 12d. The guidance statement SF30 is a statement whose content corresponds to the cause of failure to acquire the specified number of face images, such as "Your movements are too fast" or "It is too bright."

[0106] In this way, the information processing device 10 according to this embodiment performs face authentication on the user.

[0107] As described above, according to this embodiment, the display instructing a specific action on the display unit 20 is controlled and changed in accordance with the user's action status photographed by the photographing unit 22, and the progress of acquiring a facial image is also displayed on the display unit 20. Therefore, according to this embodiment, a facial image, which is biometric information necessary for impersonation determination, can be appropriately acquired based on the user's action.

[0108] [Second embodiment] An information processing device according to a second embodiment of the present invention will be described with reference to Fig. 11. Fig. 11 is a block diagram showing the configuration of the information processing device according to this embodiment. Note that components similar to those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted or simplified.

[0109] In the first embodiment, a case where the information processing device 10 is configured to perform face authentication when logging in to a specific application has been described, but the present invention is not limited to this. For example, face authentication may be performed when entering a specific location such as a room or area. In this embodiment, a case where the information processing device is configured as a control device that controls a door unit such as an automatic door or gate that restricts entry to a specific location and face authentication is performed when entering the specific location will be described.

[0110] As shown in FIG. 11, an information processing device 210 according to this embodiment has the same configuration as the information processing device 10 according to the first embodiment shown in FIG. 1, and further has a door unit 30 that regulates entry into a specific location.

[0111] The door unit 30 is configured, for example, by an automatic door, a security gate, or the like, and is installed at the entrance to a specific location such as a room or area where user entry is restricted. The door unit 30 performs opening and closing operations under the control of the CPU 12.

[0112] The information processing device 210 according to this embodiment performs face authentication similar to that of the information processing device 10 according to the first embodiment when a user enters a specific place whose entry is restricted by a door unit 30. That is, if the face authentication is successful, the information processing device 210 according to this embodiment opens the door unit 30 and allows the user to enter the specific place. On the other hand, if the face authentication is unsuccessful, the information processing device 210 keeps the door unit 30 closed and denies the user from entering the specific place.

[0113] As in this embodiment, the information processing device can also be configured to perform face authentication similar to that in the first embodiment when entering a specific location.

[0114] In addition to the above, for example, facial authentication may be performed when a user terminal logs in to a server. Therefore, the information processing device may be configured as a server that accepts logins from user terminals via a network, and facial authentication may be performed when the user terminal logs in. In this case, the information processing device configured as a server may have a configuration that functions as the CPU 12, RAM 14, and storage unit 16 similar to those in the first embodiment. The user terminal may have a configuration that functions as the input unit 18, display unit 20, image capture unit 22, and audio output unit 24 similar to those in the first embodiment.

[0115] In this way, the information processing device can be configured to perform face authentication similar to that of the first embodiment in various situations where authentication is required.

[0116] [Other embodiments] According to another embodiment, the information processing device described in the above embodiment can also be configured as shown in Fig. 12. Fig. 12 is a block diagram showing the configuration of an information processing device according to another embodiment.

[0117] 12, an information processing device 1000 according to another embodiment includes an operation instruction unit 1002 that instructs a user to perform an operation, and an information acquisition unit 1004 that acquires biometric information of the user from the user to whom the operation has been instructed. The information processing device 1000 also includes a display control unit 1006 that controls display for the user in accordance with the user's operating status.

[0118] According to another embodiment, the display directed to the user is controlled in accordance with the user's motion status, thereby making it possible to appropriately acquire necessary biometric information based on the user's motion.

[0119] [Modified embodiment] The present invention is not limited to the above-described embodiment, and various modifications are possible.

[0120] For example, in the above embodiment, a case where multiple facial images are acquired as biometric information for spoofing detection has been described as an example, but the present invention is not limited to this. In addition to facial images, other biometric information may also be acquired, such as gait images, fingerprint images, iris images, finger vein images, palm print images, and palm vein images. Instead of face authentication using facial images as biometric information, biometric authentication may be performed using biometric information acquired from a user who has received instructions to perform a specific action.

[0121] In the above embodiment, the case where the face image acquired as biometric information is used to perform impersonation determination and face matching has been described as an example, but the present invention is not limited to this. The acquired biometric information such as the face image can be used for various purposes.

[0122] Furthermore, the information processing devices 10 and 210 according to the above embodiments can be configured as a system made up of one or more devices.

[0123] Furthermore, the scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described computer program is recorded, but also the computer program itself.

[0124] Examples of the recording medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM (Compact Disc-Read Only Memory), a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment is not limited to programs that execute processing by themselves recorded on the recording medium, but also includes programs that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0125] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0126] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0127] (Appendix 1) an operation instruction unit that instructs a user to perform an operation; an information acquisition unit that acquires biometric information of the user from the user who has been instructed to perform the operation; a display control unit that controls a display directed to the user in accordance with an operating state of the user; An information processing device having the above.

[0128] (Appendix 2) the operation instruction unit instructs the operation by displaying the indication instructing the operation on a display unit; 2. The information processing device according to claim 1, wherein the display control unit controls the display instructing the operation in accordance with an operation status of the user.

[0129] (Appendix 3) 3. The information processing device according to claim 1, wherein the display control unit causes the display unit to display the display indicating the progress of acquisition of the biometric information in accordance with an operating status of the user.

[0130] (Appendix 4) 4. The information processing device according to claim 1, further comprising an information providing unit that provides guidance information when the information acquiring unit is unable to acquire a predetermined amount of the biometric information within a predetermined time.

[0131] (Appendix 5) 5. The information processing device according to claim 4, wherein the guidance information is information about the user's behavior or environment.

[0132] (Appendix 6) the action instruction unit instructs the user to perform the action related to the face; 6. The information processing device according to claim 1, wherein the information acquisition unit acquires a plurality of pieces of face information from the user as the biometric information.

[0133] (Appendix 7) 7. The information processing device according to claim 1, further comprising an impersonation determination unit that determines whether the user is an impersonator based on the biometric information.

[0134] (Appendix 8) 8. The information processing device according to claim 7, further comprising a matching unit that matches the biometric information acquired by the information acquisition unit with registered biometric information.

[0135] (Appendix 9) 9. The information processing device according to claim 8, wherein the information acquisition unit stops acquiring the biometric information when the matching by the matching unit does not match.

[0136] (Appendix 10) 10. The information processing device according to claim 8, further comprising an authentication processing unit that authenticates the user based on the result of the determination by the spoofing determination unit and the result of the matching by the matching unit.

[0137] (Appendix 11) Instruct the user to take action; acquiring biometric information of the user from the user who has instructed the operation; An information processing method for controlling a display directed to a user in accordance with an operating status of the user.

[0138] (Appendix 12) On the computer, Instruct the user to take action; acquiring biometric information of the user from the user who has instructed the operation; Controlling a display directed to the user in accordance with an operating state of the user A recording medium on which a program to perform the following is recorded.

[0139] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0140] 10, 210...Information processing device 12...CPU 12a...Authentication processing section 12b...Operation instruction section 12c...Facial image acquisition unit 12d…Information Department 12e…Spoofing detection section 12f...Face matching unit 14...RAM 16...Storage section 18...Input section 20…Display section 22...Photography Department 24...Audio output section 26…Communications Department 28...Common bus 30...door section

Claims

1. The computer Acquire multiple images, determining whether a specified number or more of images satisfying a predetermined quality have been acquired from the plurality of images; When the specified number of images satisfying the predetermined quality are not acquired, at least one image of the plurality of images is displayed.

2. The information processing method according to claim 1 , wherein, when the specified number of images satisfying the predetermined quality are not acquired, the computer displays guidance for acquiring images satisfying the predetermined quality.

3. The information processing method according to claim 2 , wherein, when the specified number of images satisfying the predetermined quality are not acquired, the computer displays the at least one image displayed together with the guidance.

4. The information processing method according to claim 2 , wherein the guidance is guidance regarding an acquisition environment.

5. 5. The information processing method according to claim 1, wherein if the specified number of images satisfying the predetermined quality are not acquired, the computer displays the at least one displayed image in grayscale or monochrome.

6. The information processing method according to claim 1 , wherein, if the specified number of images satisfying the predetermined quality are not acquired, the computer displays a visual symbol related to the reason for failure to acquire images satisfying the predetermined quality.

7. 5. An information processing method according to claim 1, wherein, if images satisfying the predetermined quality are not acquired in the specified number or more, the computer displays failure information indicating that acquisition of the specified number of facial images has failed above a portion of the at least one displayed image.

8. 5. The information processing method of claim 1, further comprising: if the specified number of images satisfying the predetermined quality are not obtained, the computer displays a cancel button below the at least one displayed image.

9. an acquisition means for acquiring a plurality of images; a determination means for determining whether a specified number of images satisfying a predetermined quality have been acquired from the plurality of images; a display means for displaying at least one image of the plurality of images when the number of images satisfying the predetermined quality is not equal to or greater than the specified number; An information processing device comprising:

10. On the computer, Acquire multiple images, determining whether a specified number or more of images satisfying a predetermined quality have been acquired from the plurality of images; If the specified number of images satisfying the predetermined quality are not acquired, at least one image of the plurality of images is displayed. A program that makes it happen.

Images