Information processing device, information processing method, computer program, and information processing system

Abstract

To provide an information processing system which accurately determines whether a user is a living body or not.SOLUTION: An information processing system 10 includes: a face direction acquisition unit 110 which acquires a face direction of a user; a visual line direction acquisition unit 120 which acquires a visual line direction of the user; a determination unit 130 which determines whether the user is a living body or not on the basis of the face direction and the visual line direction; and display means which displays a message for urging the user to shake his / her head.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, a computer program, and an information processing system that perform a determination regarding a user. [Background technology]

[0002] As this type of system, there is known a system that detects spoofing when performing biometric authentication. For example, Patent Document 1 discloses that authentication is failed if the direction of the detected face is equal to or greater than a predetermined angle. Patent Document 2 discloses that the positional relationship between the estimated gaze and the display is analyzed to determine whether the image is a spoofed image. Patent Document 3 discloses that if the detected gaze direction is an authentication-permitted gaze direction, it is determined that authentication has been performed correctly. Patent Document 4 discloses that the possibility that a face represented by a facial image sequence is a spoofed face is determined based on information on the change in gaze over time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-148968 A [Patent Document 2] JP 2008-015800 A [Patent Document 3] JP 2017-142859 A [Patent Document 4] JP 2020-194608 A Summary of the Invention [Problem to be solved by the invention]

[0004] This disclosure is intended to improve upon the techniques disclosed in the prior art documents. [Means for solving the problem]

[0005] One aspect of the information processing device disclosed herein is an information processing device comprising: a facial direction acquisition means for acquiring the facial direction of a user; a gaze direction acquisition means for acquiring the gaze direction of the user; a determination means for determining whether the user is a living body based on the facial direction and the gaze direction; and a display means for displaying a message encouraging the user to shake their head.

[0006] One aspect of the information processing method disclosed herein is one in which at least one computer acquires a user's facial direction, acquires the user's gaze direction, determines whether the user is a living body based on the facial direction and the gaze direction, and displays a message encouraging the user to shake their head.

[0007] One aspect of the computer program of the present disclosure causes at least one computer to execute an information processing method that acquires a facial direction of a user, acquires a gaze direction of the user, determines whether the user is a living body based on the facial direction and the gaze direction, and displays a message encouraging the user to shake their head.

[0008] One aspect of the information processing system disclosed herein comprises a facial orientation acquisition means for acquiring a facial orientation of a user, a gaze direction acquisition means for acquiring a gaze direction of the user, a determination means for determining whether or not the user is a living body based on the facial orientation and the gaze direction, and a display means for displaying a message encouraging the user to shake their head. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a functional configuration of an information processing system according to a first embodiment. [Diagram 3] 4 is a flowchart showing the flow of operations of the information processing system according to the first embodiment. [Figure 4] FIG. 11 is a block diagram showing a functional configuration of an information processing system according to a second embodiment. [Diagram 5] 10 is a flowchart showing the flow of operations of an information processing system according to a second embodiment. [Figure 6] 13 is a graph showing an example distribution of scores calculated in an information processing system according to a third embodiment. [Figure 7] 13 is a graph showing an example of a score correction method in the information processing system according to the third embodiment. [Figure 8] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a fourth embodiment. [Figure 9] 13 is a flowchart showing the flow of operations of an information processing system according to a fourth embodiment. [Figure 10] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a fifth embodiment. [Figure 11] 13 is a flowchart showing the flow of operations of an information processing system according to a fifth embodiment. [Figure 12] FIG. 13 is a diagram showing a display example by an information processing system according to the fifth embodiment. [Figure 13] 13 is a graph showing timing for acquiring a face direction and a gaze direction by an information processing system according to a sixth embodiment. [Figure 14] FIG. 13 is a conceptual diagram showing a score calculation method by an information processing system according to a sixth embodiment. [Figure 15] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to a seventh embodiment. [Figure 16] 20 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. [Figure 17] FIG. 13 is a diagram (part 1) showing a display example by the information processing system according to the seventh embodiment. [Figure 18] FIG. 23 is a diagram (part 2) showing a display example by the information processing system according to the seventh embodiment. [Figure 19] FIG. 13 is a block diagram showing a functional configuration of an information processing system according to an eighth embodiment. [Figure 20] 20 is a flowchart showing the flow of operations of an information processing system according to the eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of an information processing system, an information processing method, and a recording medium will be described with reference to the drawings.

[0011] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS. 1 to 3. FIG.

[0012] (Hardware configuration) First, the hardware configuration of an information processing system 10 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] 1, an information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 10 may further include an input device 15, an output device 16, and a camera 20. The processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, the output device 16, and the camera 20 are connected via a data bus 17.

[0014] The processor 11 reads a computer program. For example, the processor 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may read a computer program stored in a computer-readable recording medium using a recording medium reading device (not shown). The processor 11 may obtain (i.e., read) a computer program from a device (not shown) disposed outside the information processing system 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the computer program that the processor 11 reads. In particular, in this embodiment, when the processor 11 executes the computer program that the processor 11 reads, a functional block for determining whether or not the user is a living body is realized in the processor 11.

[0015] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), or an application specific integrated circuit (ASIC). The processor 11 may be configured as one of these, or may be configured to use a plurality of these in parallel.

[0016] The RAM 12 temporarily stores a computer program executed by the processor 11. The RAM 12 temporarily stores data that is temporarily used by the processor 11 when the processor 11 is executing the computer program. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

[0017] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store other fixed data. The ROM 13 may be, for example, a programmable ROM (P-ROM).

[0018] The storage device 14 stores data that is to be stored long-term by the information processing system 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0019] The input device 15 is a device that receives an input instruction from a user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel.

[0020] The output device 16 is a device that outputs information related to the information processing system 10 to the outside. For example, the output device 16 may be a display device (for example, a display) that can display information related to the information processing system 10.

[0021] The camera 20 is a camera installed at a location capable of capturing an image of the user (for example, an image including the user's face). The camera 20 may be a camera that captures still images, or may be a camera that captures moving images. The camera 20 may be configured as a visible light camera, or may be configured as a near-infrared camera.

[0022] (Functional configuration) Next, the functional configuration of the information processing system 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0023] As shown in Fig. 2, the information processing system 10 according to the first embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition section 110, a gaze direction acquisition section 120, a determination section 130, and an output section 140. The user information database 110 may be configured to include, for example, the above-mentioned storage device 14 (see Fig. 1). Each of the face direction acquisition section 110, the gaze direction acquisition section 120, the determination section 130, and the output section 140 may be realized by, for example, the above-mentioned processor 11 (see Fig. 1). The output section 140 may be configured to be capable of outputting its output via the above-mentioned output device 16 (see Fig. 1).

[0024] The face direction acquisition unit 110 is configured to be able to acquire the face direction of the user (i.e., the direction in which the user's face is facing). The face direction of the user may be acquired, for example, as information indicating how many degrees it deviates from a reference direction. Specifically, for example, assuming that the state in which the user faces the camera is the front, it may be acquired as information indicating how many degrees it deviates from the front. The face direction acquisition unit 110 may be configured to acquire (estimate) the face direction of the user from, for example, an image of the user's face. Note that a specific method for acquiring the face direction from an image can be appropriately adopted from existing technologies, and therefore a detailed description thereof will be omitted here.

[0025] The gaze direction acquisition unit 120 is configured to be able to acquire the gaze direction of the user (i.e., the direction in which the user's gaze is directed). The gaze direction of the user may be acquired, for example, as information indicating how many degrees it deviates from a reference direction, similar to the face direction. Specifically, for example, assuming that the state in which the user faces the camera is the front, it may be acquired as information indicating how many degrees it deviates from the front. The gaze direction acquisition unit 120 may be configured to acquire (estimate) the gaze direction of the user from, for example, a face image of the user (more specifically, an image including the area around the eyes). Note that a specific method for acquiring the gaze direction from an image can be appropriately adopted from existing technologies, and therefore a detailed description thereof will be omitted here.

[0026] The determination unit 130 is configured to be able to determine whether or not the user is a living body (in other words, whether or not the user is spoofing) based on the face direction of the user acquired by the face direction acquisition unit 110 and the gaze direction of the user acquired by the gaze direction acquisition unit 120. The determination unit 130 includes a threshold determination unit 131 and a difference calculation unit 132.

[0027] The threshold determination unit 131 is configured to be able to determine whether the face direction of the user acquired by the face direction acquisition unit 110 is equal to or greater than a predetermined threshold. The "predetermined threshold" here is a threshold for determining whether the face direction is large enough (i.e., facing sideways) to determine whether the user is a living body, and an optimal value may be obtained by a prior experiment, simulation, or the like. The predetermined threshold may be set as a threshold that can determine not only the size of the face direction of the user but also whether the direction of the face direction is a predetermined direction. For example, when the user is specified to face the right direction, the threshold may be set as a threshold that can determine that the user faces the right direction rather than another direction. The determination unit 130 is configured to determine whether the user is a living body when the threshold determination unit 131 determines that the face direction of the user is equal to or greater than the predetermined threshold. In other words, the determination unit 130 is configured to not determine whether the user is a living body when the face direction of the user is determined to be less than the predetermined threshold.

[0028] The difference calculation unit 132 is configured to be able to calculate the difference between the face direction of the user acquired by the face direction acquisition unit 110 and the gaze direction of the user acquired by the gaze direction acquisition unit 120. The determination unit 130 is configured to determine whether or not the user is a living body based on the difference between the face direction and the gaze direction. A specific information processing method using the difference between the face direction and the gaze direction will be described in detail in another embodiment described later.

[0029] The output unit 140 is configured to be able to output the determination result of the determination unit 130 (i.e., the determination result as to whether or not the user is a living body). The output mode of the output unit 140 is not particularly limited, but the output unit 140 may output an image using a display, for example. Alternatively, the output unit 140 may output an audio using a speaker.

[0030] (Operation flow) Next, the flow of operations of the information processing system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operations of the information processing system according to the first embodiment.

[0031] 3, when the information processing system 10 according to the first embodiment operates, the face direction acquisition unit 110 first acquires the face direction of the user (step S101). The gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102). Note that the processes of steps S101 and S102 may be executed one after the other, or may be executed simultaneously in parallel.

[0032] Next, the threshold determination unit 131 determines whether the user's facial direction acquired by the facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), the subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), the difference calculation unit 132 calculates the difference between the user's facial direction and line of sight (step S104).

[0033] Next, the determination unit 130 determines whether the user is a living body or not based on the difference between the user's face direction and line of sight direction (step S105). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0034] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the first embodiment will be described.

[0035] As described in Fig. 1 to Fig. 3, in the information processing system 10 according to the first embodiment, when the face direction is equal to or greater than a predetermined threshold, it is determined whether or not the user is a living body. In this way, it is possible to prevent an erroneous determination result from being output due to the user's face direction being small. Therefore, it is possible to more accurately determine whether or not the user is a living body.

[0036] According to the research of the present inventor, it is known that when the user's face direction is small, there is a high possibility that the user will be determined to be a non-living person even though he or she is a living person. Therefore, by not performing the determination when the user's face direction is small, it is possible to prevent the user from being mistakenly determined to be a non-living person.

[0037] <Second embodiment> An information processing system 10 according to the second embodiment will be described with reference to Figures 4 and 5. The second embodiment differs from the first embodiment described above only in some configurations and operations, and other parts may be the same as the first embodiment. Therefore, hereinafter, parts that differ from the first embodiment already described will be described in detail, and explanations of overlapping parts will be omitted as appropriate.

[0038] (Functional configuration) First, the functional configuration of the information processing system 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the information processing system according to the second embodiment. In Fig. 4, the same reference numerals are used to denote the same elements as those shown in Fig. 2.

[0039] As shown in Fig. 4, the information processing system 10 according to the second embodiment is configured to include, as processing blocks for realizing the functions thereof, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. The determination unit 130 according to the second embodiment particularly includes a threshold determination unit 131, a difference calculation unit 132, a score calculation unit 133, a score correction unit 134, and a determination processing unit 135. That is, the determination unit 130 according to the second embodiment is configured to further include a score calculation unit 133, a score correction unit 134, and a determination processing unit 135 in addition to the configuration of the first embodiment (see Fig. 2).

[0040] The score calculation unit 133 is configured to be able to calculate a score indicating the likelihood that the user is a living body from the difference between the user's face orientation and gaze direction. For example, the higher the score, the more likely the user is to be a living body. A specific method for calculating the score will be described in detail in another embodiment described later.

[0041] The score correction unit 134 is configured to be able to correct the score calculated by the score calculation unit 133 in accordance with the size of the user's face direction. The score is corrected so that the determination unit 130 is less likely to produce an erroneous determination result. A specific method of correcting the score will be described in detail in another embodiment described later.

[0042] The determination processing unit 135 is configured to be able to determine whether or not the user is a living organism, based on the score corrected by the score correction unit 134. The determination processing unit 135 may store, for example, a determination threshold value (i.e., a threshold value for determining whether or not the user is a living organism), and may determine whether or not the user is a living organism by comparing the corrected score with the determination threshold value.

[0043] (Operation flow) Next, the flow of operations of the information processing system 10 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of operations of the information processing system according to the second embodiment. In Fig. 5, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0044] 5, when the information processing system 10 according to the second embodiment operates, the face direction acquisition unit 110 first acquires the face direction of the user (step S101). Also, the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0045] Next, the threshold determination unit 131 determines whether the user's facial direction acquired by the facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), the subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), the difference calculation unit 132 calculates the difference between the user's facial direction and line of sight (step S104).

[0046] Next, the score calculation unit 133 calculates a score from the difference between the user's face direction and gaze direction (step S201). Then, the score correction unit 134 corrects the score according to the size of the user's face direction (step S202). After that, the determination processing unit 135 determines whether the user is a living body or not based on the corrected score (step S203). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0047] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the second embodiment will be described.

[0048] 4, in the information processing system 10 according to the second embodiment, the score calculated from the difference between the face direction and the gaze direction is corrected according to the size of the face direction, and whether or not the user is a living body is determined based on the corrected score. In this way, it is possible to more accurately determine whether or not the user is a living body compared to a case where the score is not corrected.

[0049] <Third embodiment> An information processing system 10 according to the third embodiment will be described with reference to Fig. 6 and Fig. 7. The third embodiment describes a specific correction example of the second embodiment described above, and other configurations and operations may be the same as those of the first and second embodiments. Therefore, in the following, only the parts that differ from the embodiments already described will be described in detail, and the explanation of other overlapping parts will be omitted as appropriate.

[0050] (Unadjusted score) First, the scores calculated in the information processing system 10 according to the third embodiment will be specifically described with reference to Fig. 6. Fig. 6 is a graph showing an example distribution of scores calculated in the information processing system according to the third embodiment.

[0051] As shown in FIG. 6, in the information processing system 10 according to the third embodiment, the score of a living body is calculated to be a positive value, and the score of a 3D mask (i.e., spoofing) is calculated to be a negative value. However, the larger the face direction of the user, the higher the score tends to be. Therefore, as can be seen from the figure, when the face direction is relatively small, the score may become a negative value even if the living body is not a living body. In addition, due to an estimation error of the face direction and the gaze direction, the score may become a positive value even if the living body is not a living body. If such a score is used as it is for judgment (for example, if "0" is used as the judgment threshold), an erroneous judgment result may be obtained. In order to prevent such erroneous judgment, in the information processing system 10 according to the third embodiment, the score is corrected according to the size of the face direction.

[0052] (Adjusted score) Next, the score correction in the information processing system 10 according to the third embodiment will be specifically described with reference to Fig. 7. Fig. 7 is a graph showing an example of a score correction method in the information processing system according to the third embodiment.

[0053] 7, in the information processing system 10 according to the third embodiment, the larger the face direction is, the larger the correction amount subtracted from the score is. Therefore, when the face direction is small, the score is corrected to be slightly smaller, and when the face direction is large, the score is corrected to be significantly smaller. Such a correction can be performed, for example, using the following correction formula (1).

[0054] Corrected score = -A × face direction angle + B × uncorrected score + C (1) In the above formula, A, B, and C are predetermined coefficients that are calculated in advance.

[0055] Looking at the scores after correction, all the scores for non-living bodies are negative values. This makes it possible to prevent non-living bodies from being mistakenly determined to be living bodies. On the other hand, the scores for living bodies are negative in some areas where the face direction is small, but all are positive in areas to be determined that are equal to or greater than a predetermined threshold (i.e., areas where the threshold determination unit 131 is to perform the determination). This makes it possible to prevent living bodies from being mistakenly determined to be non-living bodies.

[0056] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the third embodiment will be described.

[0057] 6 and 7, in the information processing system 10 according to the third embodiment, the larger the face direction of the user, the larger the correction amount to be subtracted from the score (i.e., the smaller the score). In this way, the calculated score is corrected to an appropriate value, making it possible to more accurately determine whether the user is a living body.

[0058] <Fourth embodiment> An information processing system 10 according to the fourth embodiment will be described with reference to Fig. 8 and Fig. 9. The fourth embodiment differs from the first to third embodiments in part of its configuration and operation, and other parts may be the same as the first to third embodiments. Therefore, in the following, explanations of parts that overlap with the embodiments already described will be omitted as appropriate.

[0059] (Functional configuration) First, the functional configuration of the information processing system 10 according to the fourth embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the functional configuration of the information processing system according to the fourth embodiment. In Fig. 8, the same reference numerals are used to denote the same elements as those shown in Figs. 2 and 4.

[0060] 8, the information processing system 10 according to the fourth embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. In particular, the determination unit 130 according to the second embodiment includes a score correction unit 134 that includes a correction formula storage unit 1341 and a correction formula selection unit 1342.

[0061] The correction formula storage unit 1341 stores a plurality of correction formulas for correcting the scores. The correction formula storage unit 1341 may store a plurality of correction formulas such as the above-mentioned formula (1) with different coefficients.

[0062] The correction formula selection unit 1342 is configured to be able to select one correction formula to be used for score correction from among a plurality of correction formulas stored in the correction formula storage unit 1341. The correction formula selection unit 1342 selects a correction formula according to information related to a user. Note that the "information related to a user" is information related to a user, for example, information indicating the user's characteristics, attributes, and status. The information related to a user may be, for example, information including numerical values ​​such as the position of the eyes (for example, how far apart the eyes are), or information indicating the tendency of facial features (for example, information regarding race). The information related to a user may also be information including the user's gender, face size, and the like. The information related to a user may be acquired from an image of the user, or may be acquired from other means (for example, input by the user, pre-registered information, etc.).

[0063] (Operation flow) Next, the flow of operations of the information processing system 10 according to the fourth embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of operations of the information processing system according to the fourth embodiment. In Fig. 9, the same processes as those shown in Figs. 3 and 5 are denoted by the same reference numerals.

[0064] 9, when the information processing system 10 according to the fourth embodiment operates, the face direction acquisition unit 110 first acquires the face direction of the user (step S101). Also, the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0065] Next, the threshold determination unit 131 determines whether the user's facial direction acquired by the facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), the subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), the difference calculation unit 132 calculates the difference between the user's facial direction and line of sight (step S104).

[0066] Next, the score calculation unit 133 calculates a score from the difference between the user's face orientation and gaze direction (step S201). In the fourth embodiment, the correction formula selection unit 1342 selects one correction formula to be used for correcting the score from among a plurality of correction formulas stored in the correction formula storage unit 1341 (step S401). Then, the score correction unit 134 corrects the score using the correction formula selected by the correction formula selection unit 1342 (step S402).

[0067] Next, the determination processing unit 135 determines whether or not the user is a living body based on the corrected score (step S203). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0068] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the fourth embodiment will be described.

[0069] As described in Fig. 8 and Fig. 9, in the information processing system 10 according to the fourth embodiment, the score correction unit stores a plurality of correction formulas, and selects a correction formula according to the user (i.e., uses different correction formulas) to correct the score. In this way, it is possible to correct the score to a more appropriate value compared to always correcting the score in the same way. As a result, it is possible to more accurately determine whether the user is a living body.

[0070] <Fifth embodiment> An information processing system 10 according to the fifth embodiment will be described with reference to Fig. 10 to Fig. 12. The fifth embodiment differs from the first to fourth embodiments described above only in some configurations and operations, and other parts may be the same as the first to fourth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0071] (Functional configuration) First, the functional configuration of the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the functional configuration of the information processing system according to the fifth embodiment. In Fig. 10, the same elements as those shown in Fig. 4 are denoted by the same reference numerals.

[0072] 10, the information processing system 10 according to the fifth embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, an output unit 140, and a notification unit 150. That is, the information processing system 10 according to the fifth embodiment is configured to further include a notification unit 150 in addition to the configuration of the second embodiment (see FIG. 4).

[0073] The notification unit 150 is configured to be able to give a notification according to the judgment result of the judgment processing unit 135. More specifically, the notification unit 150 is configured to be able to give a notification to the effect that the judgment result of the judgment processing unit 135 has changed before and after the score correction. Note that the judgment processing unit 135 according to the fifth embodiment is configured to make not only a judgment based on the corrected score, but also a judgment based on the score before the correction (hereinafter, appropriately referred to as a "provisional judgment").

[0074] (Operation flow) Next, the flow of operations of the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of operations of the information processing system according to the fifth embodiment. In Fig. 11, the same processes as those shown in Fig. 5 are denoted by the same reference numerals.

[0075] 11, when the information processing system 10 according to the fifth embodiment operates, the face direction acquisition unit 110 first acquires the face direction of the user (step S101). Also, the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0076] Next, the threshold determination unit 131 determines whether the user's facial direction acquired by the facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), the subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), the difference calculation unit 132 calculates the difference between the user's facial direction and line of sight (step S104).

[0077] Next, the score calculation unit 133 calculates a score from the difference between the user's face direction and gaze direction (step S201). Here, particularly in the fifth embodiment, the determination processing unit 135 makes a provisional determination as to whether or not the user is a living body based on the score before correction (step S501). Thereafter, the score correction unit 134 corrects the score according to the size of the user's face direction (step S202). Then, the determination processing unit 135 makes a determination as to whether or not the user is a living body based on the corrected score (step S203).

[0078] Next, the notification unit 150 determines whether the judgment result of the judgment processing unit 135 has changed before and after the score correction (step S502). If it is determined that the judgment result has changed before and after the score correction (step S502: YES), the notification unit 150 notifies that the judgment result has changed due to the correction (step S503). Then, the output unit 140 outputs the judgment result of the judgment unit 130 (step S106).

[0079] The notification by the notification unit 150 may be performed together with the output by the output unit 140. For example, the notification by the notification unit 150 and the output by the output unit 140 may be performed using, for example, a common display. The notification by the notification unit 150 may be performed to the user, or may be performed to a system administrator or the like.

[0080] (Example) Next, a display example in the information processing system 10 according to the fifth embodiment (specifically, a display example reflecting a notification by the notification unit 150) will be described with reference to Fig. 12. Fig. 12 is a diagram showing a display example in the information processing system according to the fifth embodiment.

[0081] As shown in FIG. 12, in the information processing system 10 according to the fifth embodiment, the judgment result by the judgment unit 130 (i.e., the output by the output unit 140) may be displayed together with the face image of the user. Here, the judgment result of the user is output as "OK" (i.e., the user is a living body). In particular, in the fifth embodiment, in addition to the judgment result, a notification by the notification unit 150 may be displayed. Here, a message saying "Results have changed due to score correction" is displayed below the judgment result. Instead of such a message, for example, a message suggesting that the reliability is low or a message urging the user to redo the judgment may be displayed.

[0082] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the fifth embodiment will be described.

[0083] 10 to 12, in the information processing system 10 according to the fifth embodiment, when the result of the determination as to whether the user is a living body changes before and after the score correction, a process of notifying the user of the change is executed. In this way, the notification is given in a situation where the original score is near the determination threshold, so that it becomes possible to know, for example, that the current determination was difficult (in other words, that the reliability of the determination may be low).

[0084] Sixth Embodiment An information processing system 10 according to the sixth embodiment will be described with reference to Fig. 13 and Fig. 14. The sixth embodiment differs from the first to fifth embodiments in some configurations and operations, and other parts may be the same as the first to fifth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0085] (Acquisition of face direction and gaze direction) First, a method for acquiring the face direction and the forward direction in the information processing system 10 according to the sixth embodiment will be specifically described with reference to Fig. 13. Fig. 13 is a graph showing the timing for acquiring the face direction and the gaze direction by the information processing system according to the sixth embodiment.

[0086] 13, in the information processing system 10 according to the sixth embodiment, the facial direction and gaze direction of the user are acquired while the user is shaking his / her head (i.e., while the user is repeatedly turning his / her face to the right and left). More specifically, a plurality of images are captured while the user is shaking his / her head, and the facial direction and gaze direction of the user are acquired (estimated) from those images.

[0087] The user's facial direction and gaze direction are acquired at the timings when the user's facial direction peaks (P1, P2, P3, and P4 in the figure). In this case, the user's facial direction and gaze direction are acquired a total of four times, two at the peaks to the right and two at the peaks to the left. Note that the number of acquisitions here is an example, and the user's facial direction and gaze direction may be acquired, for example, four or more times.

[0088] (Example of score calculation) Next, a method for calculating a score in the information processing system 10 according to the sixth embodiment will be specifically described with reference to Fig. 14. Fig. 14 is a conceptual diagram showing a method for calculating a score by the information processing system according to the sixth embodiment.

[0089] As shown in Fig. 14, in the information processing system 10 according to the sixth embodiment, a plurality of differences are calculated from the facial direction and gaze direction acquired at a plurality of timings. Therefore, as shown in Fig. 13, when the facial direction and gaze direction of the user are acquired four times, P1, P2, P3, and P4, the differences are calculated as a value including four values ​​corresponding to P1, P2, P3, and P4 (for example, a four-dimensional vector). The differences calculated in this way are input to the score calculation unit 133, and a score is calculated as the output. In this case, the score calculation unit 133 may be configured as, for example, a trained SVM (Support Vector Machine).

[0090] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the sixth embodiment will be described.

[0091] 13 and 14, in the information processing system 10 according to the sixth embodiment, the facial direction and gaze direction are acquired at multiple timings from a plurality of images captured while the user is shaking his / her head. In this way, it becomes possible to determine whether or not the user is a living body with higher accuracy compared to the case where the facial direction and gaze direction are acquired only once.

[0092] Seventh embodiment An information processing system 10 according to the seventh embodiment will be described with reference to Fig. 15 to Fig. 18. The seventh embodiment differs from the sixth embodiment described above only in some operations, and other parts may be the same as the first to sixth embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0093] (Functional configuration) First, the functional configuration of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the functional configuration of the information processing system according to the seventh embodiment. In Fig. 15, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0094] As shown in Fig. 15, the information processing system 10 according to the seventh embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition section 110, a gaze direction acquisition section 120, a determination section 130, an output section 140, and a head swing instruction section 160. That is, the information processing system 10 according to the seventh embodiment is configured to further include a head swing instruction section 160 in addition to the configuration of the first embodiment (see Fig. 2). Also, the determination section 130 according to the seventh embodiment includes a head swing determination section 136 in addition to a threshold determination section 131 and a difference calculation section 132.

[0095] The head-shake determination unit 136 is configured to be able to determine whether the head shake (sixth embodiment: see FIG. 13) when acquiring the user's face direction and line of sight direction is performed appropriately. The head-shake determination unit 136 may determine whether the head shake is appropriate based on, for example, whether the user shakes his / her head sufficiently widely (for example, whether the peak of the face direction exceeds a predetermined angle) or whether the user shakes his / her head in an appropriate direction (for example, whether the user shakes his / her head in a different direction (for example, downward) at the timing when the user should shake his / her head to the right).

[0096] The head swing instruction unit 160 is configured to be able to output an instruction to the user to perform an appropriate head swing when the head swing determination unit 136 determines that the user's head swing is inappropriate. The head swing instruction unit 160 may give an instruction together with the output by the output unit 140. For example, the instruction by the head swing instruction unit and the output by the output unit 140 may be given using, for example, a common display. In addition, when the user swings his / her head multiple times, an instruction may be output only for some of the movements. For example, when only the face direction acquired by P3 among P1, P2, P3, and P4 in FIG. 14 is inappropriate (for example, while the angles of P1, P2, and P4 are sufficient, the angle of P3 is small), the head swing instruction unit 160 may output only an instruction for the head swing movement corresponding to P3. Specifically, an instruction may be output to swing the head more greatly in the direction corresponding to the movement of P3.

[0097] (Operation flow) Next, the flow of operations of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. In Fig. 16, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0098] 16, when the information processing system 10 according to the seventh embodiment operates, the face direction acquisition unit 110 first acquires the face direction of the user (step S101). In addition, the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0099] Next, the head-shake determination unit 136 determines whether the user's head shake has been performed appropriately (step S701). If it is determined that the user's head shake has not been performed appropriately (step S701: NO), the head-shake instruction unit 160 outputs an instruction to the user to perform an appropriate head shake. Then, the processes of steps S101 and S102 (i.e., obtaining the face direction and line of sight direction) are executed again.

[0100] On the other hand, if it is determined that the user's head has been turned appropriately (step S701: YES), the threshold determination unit 131 determines whether the user's face direction acquired by the face direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's face direction is not equal to or greater than the predetermined threshold (step S103: NO), the subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's face direction is equal to or greater than the predetermined threshold (step S103: YES), the difference calculation unit 132 calculates the difference between the user's face direction and line of sight (step S104).

[0101] Next, the determination unit 130 determines whether the user is a living body or not based on the difference between the user's face direction and line of sight direction (step S105). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0102] (Example) Next, a display example in the information processing system 10 according to the seventh embodiment (specifically, a display example reflecting an instruction by the head swing instruction unit 160) will be described with reference to Fig. 17 and Fig. 18. Fig. 17 is a diagram (part 1) showing a display example in the information processing system according to the seventh embodiment. Fig. 18 is a diagram (part 2) showing a display example in the information processing system according to the seventh embodiment.

[0103] As shown in Fig. 17, in the information processing system 10 according to the seventh embodiment, a message encouraging the user to shake his / her head more widely may be displayed together with the image of the user. For example, as in the example of Fig. 17, a message saying "Please shake your head a little more widely" may be displayed. In this way, it is expected that the user will shake his / her head more widely, and as a result, it becomes possible to acquire the facial direction and gaze direction of the user with the facial direction being sufficiently large.

[0104] As shown in Fig. 18, in the information processing system 10 according to the seventh embodiment, a message indicating the direction in which the user should shake his / her head may be displayed together with an image of the user. For example, as in the example of Fig. 18, a message saying "Please shake your head a little more to the left" may be displayed. In this way, for example, when the head shake to the right is sufficient but the head shake to the left is insufficient, a more accurate instruction can be given to the user.

[0105] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the seventh embodiment will be described.

[0106] 15 to 18, in the information processing system 10 according to the seventh embodiment, when the user's head shaking is inappropriate, an instruction to the user to shake his / her head appropriately is output. In this way, the user's head shaking can be made appropriate, and the user's face direction and line of sight direction can be appropriately acquired, so that it is possible to more accurately determine whether the user is a living body.

[0107] Eighth embodiment An information processing system 10 according to the eighth embodiment will be described with reference to Fig. 19 and Fig. 20. Note that the information processing system 10 according to the eighth embodiment differs from the first to seventh embodiments described above only in some operations, and other parts may be the same as the first to seventh embodiments. Therefore, hereinafter, parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0108] (Functional configuration) First, the functional configuration of the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 19. Fig. 19 is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. In Fig. 19, the same elements as those shown in Figs. 2 and 4 are denoted by the same reference numerals.

[0109] 19, the information processing system 10 according to the eighth embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. The determination unit 130 according to the eighth embodiment particularly includes a difference calculation unit 132, a score calculation unit 133, a score correction unit 134, and a determination processing unit 135.

[0110] The information processing system 10 according to the eighth embodiment is different from the first to seventh embodiments in that it does not include the threshold determination unit 131. Therefore, regardless of whether the user's face direction is equal to or greater than a predetermined threshold, the determination unit 130 performs a determination. The specific operation of the determination unit 130 will be described in detail below.

[0111] (Operation flow) Next, the flow of operations of the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 20. Fig. 20 is a flowchart showing the flow of operations of the information processing system according to the eighth embodiment. In Fig. 20, the same processes as those shown in Figs. 3 and 5 are denoted by the same reference numerals.

[0112] 20, when the information processing system 10 according to the eighth embodiment operates, the face direction acquisition unit 110 first acquires the face direction of the user (step S801). Also, the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S802).

[0113] Next, the difference calculation unit 132 calculates the difference between the user's face direction and gaze direction (step S803). Then, the score calculation unit 133 calculates a score from the difference between the user's face direction and gaze direction (step S804). After that, the score correction unit 134 corrects the score according to the size of the user's face direction (step S805).

[0114] Next, the determination processing unit 135 determines whether or not the user is a living body based on the corrected score (step S806). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S807).

[0115] (Technical effect) Next, technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0116] 19 and 20, in the information processing system 10 according to the eighth embodiment, the score calculated from the difference between the face direction and the gaze direction is corrected according to the size of the face direction, and whether or not the user is a living body is determined based on the corrected score. In this way, the score becomes an appropriate value through the correction, so that it is possible to more accurately determine whether or not the user is a living body.

[0117] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-mentioned embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. That is, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, not only the recording medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment.

[0118] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the recording media, but also those that execute processing by operating on an OS in cooperation with other software and functions of an extension board.

[0119] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing methods, and recording media incorporating such modifications are also included in the technical idea of ​​this disclosure.

[0120] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0121] (Appendix 1) The information processing system described in Appendix 1 is an information processing system including: a facial direction acquisition means for acquiring a facial direction of a user; a gaze direction acquisition means for acquiring a gaze direction of the user; a determination means for determining whether or not the user is a living body based on a difference between the facial direction and the gaze direction when the facial direction is equal to or greater than a predetermined threshold; and an output means for outputting a result of the determination.

[0122] (Appendix 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, wherein the determination means includes a calculation means for calculating a score indicating the liveness of the user from the difference between the facial direction and the gaze direction, a correction means for correcting the score according to the magnitude of the facial direction, and a determination processing means for determining whether or not the user is a live person based on the corrected score.

[0123] (Appendix 3) The information processing system according to Supplementary Note 3 is the information processing system according to Supplementary Note 2, wherein the correction means performs correction such that the larger the face direction is, the larger the correction amount to be subtracted from the score is.

[0124] (Appendix 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 2 or 3, wherein the correction means stores a plurality of correction formulas for correcting the score, and selects a correction formula according to information related to the user to correct the score.

[0125] (Appendix 5) The information processing system described in Appendix 5 is an information processing system described in any one of Appendices 2 to 4, in which the judgment processing means, in addition to judging whether the user is a living body based on the corrected score, judges whether the user is a living body based on the score before correction, and further comprises a notification means for notifying that the judgment result has changed if the judgment result of the judgment processing means changes before and after correction.

[0126] (Appendix 6) The information processing system described in Appendix 6 is the information processing system described in any one of Appendixes 1 to 5, wherein the facial direction acquisition means and the gaze direction acquisition means acquire the facial direction and the gaze direction at multiple times from multiple images captured while the user is shaking his / her head, and the determination means determines whether or not the user is a living body based on the difference between the facial direction and the gaze direction at the multiple times.

[0127] (Appendix 7) The information processing system described in Appendix 7 is the information processing system described in Appendix 6, further comprising an instruction means for instructing the user on an appropriate head movement when it is detected that the user's head movement is not appropriate.

[0128] (Appendix 8) The information processing system described in Appendix 8 is an information processing system including: a facial direction acquisition means for acquiring a facial direction of a user; a gaze direction acquisition means for acquiring a gaze direction of the user; a calculation means for calculating a score indicating the liveness of the user from a difference between the facial direction and the gaze direction; a correction means for correcting the score according to a magnitude of the facial direction; a determination processing means for determining whether the user is a live person or not based on the corrected score; and an output means for outputting a result of the determination.

[0129] (Appendix 9) The information processing method described in Appendix 9 is an information processing method that acquires a facial direction of a user, acquires a gaze direction of the user, and if the facial direction is equal to or greater than a predetermined threshold, determines whether or not the user is a living body based on the difference between the facial direction and the gaze direction, and outputs a result of the determination.

[0130] (Appendix 10) The recording medium described in Appendix 10 is a recording medium having recorded thereon a computer program for causing a computer to execute an information processing method for acquiring a facial direction of a user, acquiring a gaze direction of the user, and, if the facial direction is equal to or greater than a predetermined threshold, determining whether or not the user is a living body based on the difference between the facial direction and the gaze direction, and outputting a result of the determination.

[0131] (Appendix 11) The computer program described in Appendix 11 is a computer program that executes an information processing method of acquiring a facial direction of a user, acquiring a gaze direction of the user, and, if the facial direction is equal to or greater than a predetermined threshold, determining whether or not the user is a living body based on a difference between the facial direction and the gaze direction, and outputting a result of the determination. [Explanation of symbols]

[0132] 10. Information Processing Systems 11 Processors 16 Output Devices 20 Camera 110 Face direction acquisition unit 120 Gaze direction acquisition unit 130 Judgment section 131 Threshold judgment unit 132 Difference calculation part 133 Score Calculation Section 134 Score Correction Section 1341 Correction formula storage unit 1342 Correction formula selection section 135 Judgment processing unit 136 Swing Judgment Unit 140 Output section 150 Notification Department 160 Swing indicator

Claims

1. A face direction acquisition means for acquiring a face direction of a user; A gaze direction acquisition means for acquiring a gaze direction of the user; a determination means for determining whether the user is a living body based on the face direction and the line of sight direction; A display means for displaying a message prompting the user to shake his / her head; An information processing device comprising:

2. The display means displays the message when the user's head swing is inappropriate. The information processing device according to claim 1 .

3. The display means displays the message when the face direction of the user is smaller than a predetermined value.

3. The information processing device according to claim 1 or 2.

4. The message is a message encouraging the user to shake his / her head in a direction in which the user's face faces more toward the front. The information processing device according to claim 1 .

5. The determination means determines whether the user is a living body based on a difference between the face direction and the line of sight direction. The information processing device according to claim 1 .

6. At least one computer Obtain the user's face direction, Acquire a gaze direction of the user; determining whether the user is a living body based on the face orientation and the gaze direction; displaying a message prompting the user to shake their head; Information processing methods.

7. At least one computer Obtain the user's face direction, Acquire a gaze direction of the user; determining whether the user is a living body based on the face orientation and the gaze direction; displaying a message prompting the user to shake their head; A computer program for executing an information processing method.

8. A face direction acquisition means for acquiring a face direction of a user; A gaze direction acquisition means for acquiring a gaze direction of the user; a determination means for determining whether the user is a living body based on the face direction and the line of sight direction; A display means for displaying a message prompting the user to shake his / her head; An information processing system comprising:

Images