Information processing device, information processing method, computer program, and information processing system

Abstract

To provide an information processing system which accurately determines whether a user is a living body or not.SOLUTION: An information processing system 10 includes: a face direction acquisition unit 110 which acquires a face direction of a user; a visual line direction acquisition unit 120 which acquires a visual line direction of the user; a determination unit 130 which determines whether the user is a living body or not on the basis of the face direction and the visual line direction; and display means which displays a message for urging the user to shake his / her head.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, a computer program, and an information processing system that make a determination regarding a user. [Background technology]

[0002] Known systems of this type detect impersonation when performing biometric authentication. For example, Patent Document 1 discloses that authentication fails if the direction of a detected face is equal to or greater than a predetermined angle. Patent Document 2 discloses that the positional relationship between an estimated gaze and a display is analyzed to determine whether the image is an impersonation image. Patent Document 3 discloses that if the detected gaze direction is an authentication-permitted gaze direction, it is determined that authentication has been performed correctly. Patent Document 4 discloses that the possibility that a face represented by a facial image sequence is an impersonation is determined based on information on changes in gaze over time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-148968 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-015800 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-142859 [Patent Document 4] Japanese Patent Application Publication No. 2020-194608 Summary of the Invention [Problem to be solved by the invention]

[0004] This disclosure aims to improve upon the techniques disclosed in the prior art documents. [Means for solving the problem]

[0005] One aspect of the information processing device disclosed herein is an information processing device that includes a facial direction acquisition means that acquires the facial direction of a user, a gaze direction acquisition means that acquires the gaze direction of the user, a determination means that determines whether the user is a living body based on the facial direction and the gaze direction, and a display means that displays a message encouraging the user to shake their head.

[0006] In one aspect of the information processing method disclosed herein, at least one computer acquires the facial orientation of a user, acquires the user's gaze direction, determines whether the user is a living body based on the facial orientation and the gaze direction, and displays a message prompting the user to shake their head.

[0007] One aspect of the computer program of the present disclosure causes at least one computer to execute an information processing method that acquires a facial orientation of a user, acquires a gaze direction of the user, determines whether the user is a living body based on the facial orientation and the gaze direction, and displays a message prompting the user to shake their head.

[0008] One aspect of the information processing system disclosed herein includes a facial orientation acquisition means for acquiring the facial orientation of a user, a gaze direction acquisition means for acquiring the gaze direction of the user, a determination means for determining whether the user is a living body based on the facial orientation and the gaze direction, and a display means for displaying a message prompting the user to shake their head. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing system according to a first embodiment. [Figure 2] 1 is a block diagram showing a functional configuration of an information processing system according to a first embodiment. [Figure 3] 4 is a flowchart showing the flow of operations of the information processing system according to the first embodiment. [Figure 4] FIG. 10 is a block diagram showing the functional configuration of an information processing system according to a second embodiment. [Figure 5] 10 is a flowchart showing the flow of operations of the information processing system according to the second embodiment. [Figure 6] 11 is a graph showing an example distribution of scores calculated in an information processing system according to a third embodiment. [Figure 7] 11 is a graph showing an example of a score correction method in the information processing system according to the third embodiment. [Figure 8] FIG. 10 is a block diagram showing the functional configuration of an information processing system according to a fourth embodiment. [Figure 9] 10 is a flowchart showing the flow of operations of the information processing system according to the fourth embodiment. [Figure 10] FIG. 11 is a block diagram showing the functional configuration of an information processing system according to a fifth embodiment. [Figure 11] 13 is a flowchart showing the flow of operations of the information processing system according to the fifth embodiment. [Figure 12] FIG. 13 is a diagram showing a display example by the information processing system according to the fifth embodiment. [Figure 13] 13 is a graph showing timings for acquiring a face direction and a line of sight direction by an information processing system according to a sixth embodiment. [Figure 14] FIG. 13 is a conceptual diagram showing a score calculation method by an information processing system according to a sixth embodiment. [Figure 15] FIG. 13 is a block diagram showing the functional configuration of an information processing system according to a seventh embodiment. [Figure 16] 13 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. [Figure 17] FIG. 13 is a diagram (part 1) showing a display example by the information processing system according to the seventh embodiment. [Figure 18] FIG. 20 is a diagram (part 2) showing a display example by the information processing system according to the seventh embodiment. [Figure 19] FIG. 13 is a block diagram showing the functional configuration of an information processing system according to an eighth embodiment. [Figure 20] 13 is a flowchart showing the flow of operations of the information processing system according to the eighth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an information processing system, an information processing method, and a recording medium will be described with reference to the drawings.

[0011] First Embodiment An information processing system according to a first embodiment will be described with reference to FIGS.

[0012] (Hardware configuration) First, the hardware configuration of an information processing system 10 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the information processing system according to the first embodiment.

[0013] 1, an information processing system 10 according to the first embodiment includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing system 10 may further include an input device 15, an output device 16, and a camera 20. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, output device 16, and camera 20 are connected via a data bus 17.

[0014] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing system 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, a functional block for determining whether the user is a living body is realized within the processor 11.

[0015] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a demand-side platform (DSP), or an application-specific integrated circuit (ASIC). The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0016] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that is temporarily used by the processor 11 while the processor 11 is executing the computer programs. The RAM 12 may be, for example, a D-RAM (Dynamic RAM).

[0017] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable ROM (P-ROM).

[0018] The storage device 14 stores data that is to be saved long-term by the information processing system 10. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0019] The input device 15 is a device that receives input instructions from a user of the information processing system 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel.

[0020] The output device 16 is a device that outputs information related to the information processing system 10 to the outside. For example, the output device 16 may be a display device (for example, a display) that can display information related to the information processing system 10.

[0021] Camera 20 is a camera installed in a location where it can capture an image of the user (for example, an image including the user's face). Camera 20 may be a camera that captures still images or a camera that captures moving images. Camera 20 may be configured as a visible light camera or a near-infrared camera.

[0022] (Functional configuration) Next, the functional configuration of the information processing system 10 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the information processing system according to the first embodiment.

[0023] As shown in FIG. 2, the information processing system 10 according to the first embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. The user information database 110 may be configured to include, for example, the above-mentioned storage device 14 (see FIG. 1). Each of the face direction acquisition unit 110, the gaze direction acquisition unit 120, the determination unit 130, and the output unit 140 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1). The output unit 140 may be configured to be able to output its output via the above-mentioned output device 16 (see FIG. 1).

[0024] The face direction acquisition unit 110 is configured to be able to acquire the face direction of the user (i.e., the direction in which the user's face is facing). The face direction of the user may be acquired, for example, as information indicating how many degrees it deviates from a reference direction. Specifically, for example, assuming that the state in which the user is facing the camera is the front, the face direction acquisition unit 110 may be acquired as information indicating how many degrees it deviates from the front. The face direction acquisition unit 110 may be configured, for example, to acquire (estimate) the face direction of the user from an image of the user's face. Note that a specific method for acquiring the face direction from an image can be appropriately adopted from existing technology, and therefore a detailed description thereof will be omitted here.

[0025] The gaze direction acquisition unit 120 is configured to be able to acquire the gaze direction of the user (i.e., the direction in which the user's gaze is directed). The gaze direction of the user may be acquired, for example, as information indicating how many degrees it deviates from a reference direction, similar to the face direction. Specifically, for example, assuming that the state in which the user is facing the camera is the front, the gaze direction acquisition unit 120 may be acquired as information indicating how many degrees it deviates from the front. The gaze direction acquisition unit 120 may be configured to acquire (estimate) the gaze direction of the user from, for example, an image of the user's face (more specifically, an image including the area around the eyes). Note that a specific method for acquiring the gaze direction from an image can be appropriately adopted from existing technology, and therefore a detailed description thereof will be omitted here.

[0026] The determination unit 130 is configured to be able to determine whether the user is a living body (in other words, whether the user is spoofing) based on the facial direction of the user acquired by the facial direction acquisition unit 110 and the gaze direction of the user acquired by the gaze direction acquisition unit 120. The determination unit 130 includes a threshold determination unit 131 and a difference calculation unit 132.

[0027] The threshold determination unit 131 is configured to determine whether the user's facial direction acquired by the facial direction acquisition unit 110 is equal to or greater than a predetermined threshold. The "predetermined threshold" here refers to a threshold for determining whether the user's facial direction is large enough (i.e., facing sideways) to determine whether the user is a living body, and an optimal value may be determined through prior experiments, simulations, or the like. The predetermined threshold may be set as a threshold that can determine not only the size of the user's facial direction but also whether the direction of the user's facial direction is a predetermined direction. For example, if the user is instructed to face right, the threshold may be set as a threshold that can determine whether the user is facing right rather than in another direction. The determination unit 130 is configured to determine whether the user is a living body when the threshold determination unit 131 determines that the user's facial direction is equal to or greater than the predetermined threshold. In other words, the determination unit 130 is configured not to determine whether the user is a living body when the user's facial direction is determined to be less than the predetermined threshold.

[0028] The difference calculation unit 132 is configured to be able to calculate the difference between the user's facial direction acquired by the facial direction acquisition unit 110 and the user's gaze direction acquired by the gaze direction acquisition unit 120. The determination unit 130 is configured to determine whether or not the user is a living body based on the difference between the facial direction and the gaze direction. A specific information processing method using the difference between the facial direction and the gaze direction will be described in detail in another embodiment described later.

[0029] The output unit 140 is configured to be able to output the determination result of the determination unit 130 (i.e., the determination result as to whether or not the user is a living body). The output mode of the output unit 140 is not particularly limited, but the output unit 140 may be configured to output an image using a display, for example. Alternatively, the output unit 140 may be configured to output an audio signal using a speaker.

[0030] (Operation flow) Next, the flow of operations of the information processing system 10 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operations of the information processing system according to the first embodiment.

[0031] 3, when the information processing system 10 according to the first embodiment operates, first, the face direction acquisition unit 110 acquires the face direction of the user (step S101). Then, the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102). Note that the processes of steps S101 and S102 may be executed one after the other, or may be executed simultaneously in parallel.

[0032] Next, threshold determination unit 131 determines whether the user's facial direction acquired by facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), difference calculation unit 132 calculates the difference between the user's facial direction and gaze direction (step S104).

[0033] Next, the determination unit 130 determines whether the user is a living body based on the difference between the user's facial orientation and gaze direction (step S105). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0034] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the first embodiment will be described.

[0035] As described with reference to Figures 1 to 3, the information processing system 10 according to the first embodiment determines whether the user is a living body when the face orientation is equal to or greater than a predetermined threshold. This prevents an erroneous determination result from being output due to the user's face orientation being small. Therefore, it is possible to more accurately determine whether the user is a living body.

[0036] According to the research of the present inventor, it has been found that when the user's face is turned slightly, there is a high possibility that the user will be determined to be non-living even though they are actually living. Therefore, by not performing the determination when the user's face is turned slightly, it is possible to prevent the user from being mistakenly determined to be non-living.

[0037] Second Embodiment An information processing system 10 according to the second embodiment will be described with reference to Figures 4 and 5. The second embodiment differs from the first embodiment described above only in part of its configuration and operation, and other parts may be the same as those of the first embodiment. Therefore, the following will describe in detail the parts that differ from the first embodiment already described, and will omit a description of overlapping parts as appropriate.

[0038] (Functional configuration) First, the functional configuration of the information processing system 10 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the information processing system according to the second embodiment. Note that in Fig. 4, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0039] 4, the information processing system 10 according to the second embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. The determination unit 130 according to the second embodiment particularly includes a threshold determination unit 131, a difference calculation unit 132, a score calculation unit 133, a score correction unit 134, and a determination processing unit 135. That is, the determination unit 130 according to the second embodiment is configured to further include the score calculation unit 133, the score correction unit 134, and the determination processing unit 135 in addition to the configuration of the first embodiment (see FIG. 2).

[0040] The score calculation unit 133 is configured to be able to calculate a score indicating the likelihood of the user being a living body from the difference between the user's facial orientation and gaze direction. For example, the higher the score, the more likely the user is to be a living body. A specific method for calculating the score will be described in detail in another embodiment described later.

[0041] The score correction unit 134 is configured to be able to correct the score calculated by the score calculation unit 133 in accordance with the size of the user's face direction. The score correction is performed so as to make it less likely that an erroneous determination result will be produced by the determination unit 130. A specific method for correcting the score will be described in detail in another embodiment described later.

[0042] The determination processing unit 135 is configured to be able to determine whether or not the user is a living organism based on the score corrected by the score correction unit 134. The determination processing unit 135 may store, for example, a determination threshold value (i.e., a threshold value for determining whether or not the user is a living organism), and may determine whether or not the user is a living organism by comparing the corrected score with the determination threshold value.

[0043] (Operation flow) Next, the flow of operations of the information processing system 10 according to the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of operations of the information processing system according to the second embodiment. Note that in Fig. 5, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0044] 5, when the information processing system 10 according to the second embodiment operates, the facial direction acquisition unit 110 first acquires the facial direction of the user (step S101), and the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0045] Next, threshold determination unit 131 determines whether the user's facial direction acquired by facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), difference calculation unit 132 calculates the difference between the user's facial direction and gaze direction (step S104).

[0046] Next, the score calculation unit 133 calculates a score from the difference between the user's facial orientation and gaze direction (step S201). Then, the score correction unit 134 corrects the score according to the size of the user's facial orientation (step S202). Thereafter, the determination processing unit 135 determines whether the user is a living body based on the corrected score (step S203). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0047] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the second embodiment will be described.

[0048] 4, in the information processing system 10 according to the second embodiment, the score calculated from the difference between the face direction and the gaze direction is corrected according to the magnitude of the face direction, and whether or not the user is a living body is determined based on the corrected score. In this way, it is possible to determine whether or not the user is a living body with greater accuracy than when the score is not corrected.

[0049] <Third embodiment> An information processing system 10 according to a third embodiment will be described with reference to Figures 6 and 7. The third embodiment describes a specific correction example of the second embodiment described above, and other configurations and operations may be the same as those of the first and second embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0050] (Uncorrected score) First, the scores calculated in the information processing system 10 according to the third embodiment will be specifically described with reference to Fig. 6. Fig. 6 is a graph showing an example distribution of scores calculated in the information processing system according to the third embodiment.

[0051] As shown in FIG. 6, in the information processing system 10 according to the third embodiment, the score for a biometric is calculated to be a positive value, and the score for a 3D mask (i.e., impersonation) is calculated to be a negative value. However, the larger the user's facial orientation, the higher the score tends to be. Therefore, as can be seen from the diagram, if the facial orientation is relatively small, the score may be negative even for a biometric. Furthermore, due to estimation errors in the facial orientation and gaze direction, the score may be positive even for a non-biometric subject. If such a score is used as is (for example, if "0" is used as the determination threshold), an erroneous determination result may be obtained. To prevent such erroneous determination, the information processing system 10 according to the third embodiment corrects the score according to the size of the facial orientation.

[0052] (Adjusted score) Next, the score correction in the information processing system 10 according to the third embodiment will be specifically described with reference to Fig. 7. Fig. 7 is a graph showing an example of a score correction method in the information processing system according to the third embodiment.

[0053] 7, in the information processing system 10 according to the third embodiment, the larger the facial orientation, the larger the correction amount subtracted from the score. Therefore, when the facial orientation is small, the score is corrected to be slightly smaller, and when the facial orientation is large, the score is corrected to be slightly smaller. Such a correction can be performed, for example, using the following correction formula (1):

[0054] Corrected score = -A × face direction angle + B × uncorrected score + C (1) It should be noted that A, B, and C in the above formula are predetermined coefficients that are calculated in advance.

[0055] Looking at the scores after correction, all scores for non-living people are negative values. This makes it possible to prevent non-living people from being mistakenly determined to be living people. On the other hand, the scores for living people are negative in some areas where the face orientation is small, but all scores are positive for areas to be determined that are equal to or greater than the predetermined threshold (i.e., areas where determination is to be made by the threshold determination unit 131). This makes it possible to prevent living people from being mistakenly determined to be non-living people.

[0056] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the third embodiment will be described.

[0057] 6 and 7, in the information processing system 10 according to the third embodiment, the larger the user's facial orientation, the larger the correction amount to be subtracted from the score (i.e., the smaller the score). In this way, the calculated score is corrected to an appropriate value, making it possible to more accurately determine whether the user is a living body.

[0058] <Fourth embodiment> An information processing system 10 according to the fourth embodiment will be described with reference to Figures 8 and 9. The fourth embodiment differs from the first to third embodiments in some configurations and operations, and other parts may be the same as the first to third embodiments. Therefore, in the following, explanations of parts that overlap with the embodiments already described will be omitted as appropriate.

[0059] (Functional configuration) First, the functional configuration of the information processing system 10 according to the fourth embodiment will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the functional configuration of the information processing system according to the fourth embodiment. Note that in Fig. 8, the same elements as those shown in Figs. 2 and 4 are denoted by the same reference numerals.

[0060] 8, the information processing system 10 according to the fourth embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. In particular, the determination unit 130 according to the second embodiment includes a score correction unit 134 that includes a correction formula storage unit 1341 and a correction formula selection unit 1342.

[0061] The correction formula storage unit 1341 stores a plurality of correction formulas for correcting the scores. The correction formula storage unit 1341 may store a plurality of correction formulas, such as the above-mentioned formula (1), with different coefficients.

[0062] The correction formula selection unit 1342 is configured to be able to select one correction formula to be used for score correction from among multiple correction formulas stored in the correction formula storage unit 1341. The correction formula selection unit 1342 selects a correction formula according to information related to a user. Note that "information related to a user" refers to information related to a user, such as information indicating the user's characteristics, attributes, and status. The information related to a user may be, for example, information including numerical values ​​such as the position of the eyes (e.g., how far apart the eyes are), or information indicating facial features (e.g., information about race). The information related to a user may also be information including the user's gender, face size, etc. The information related to a user may be acquired from an image of the user, or may be acquired by other means (e.g., user input, pre-registered information, etc.).

[0063] (Operation flow) Next, the flow of operations of the information processing system 10 according to the fourth embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of operations of the information processing system according to the fourth embodiment. Note that in Fig. 9, the same processes as those shown in Figs. 3 and 5 are denoted by the same reference numerals.

[0064] 9, when the information processing system 10 according to the fourth embodiment operates, first, the face direction acquisition unit 110 acquires the face direction of the user (step S101), and the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0065] Next, threshold determination unit 131 determines whether the user's facial direction acquired by facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), difference calculation unit 132 calculates the difference between the user's facial direction and gaze direction (step S104).

[0066] Next, the score calculation unit 133 calculates a score from the difference between the user's facial orientation and gaze direction (step S201). Particularly in the fourth embodiment, the correction formula selection unit 1342 selects one correction formula to be used for correcting the score from among a plurality of correction formulas stored in the correction formula storage unit 1341 (step S401). Then, the score correction unit 134 corrects the score using the correction formula selected by the correction formula selection unit 1342 (step S402).

[0067] Next, the determination processing unit 135 determines whether or not the user is a living body based on the corrected score (step S203). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0068] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the fourth embodiment will be described.

[0069] 8 and 9, in the information processing system 10 according to the fourth embodiment, the score correction unit stores a plurality of correction formulas, and selects a correction formula according to the user (i.e., uses different correction formulas) to correct the score. In this way, it is possible to correct the score to a more appropriate value compared to always correcting the score in the same way. As a result, it is possible to more accurately determine whether the user is a living body.

[0070] Fifth Embodiment An information processing system 10 according to the fifth embodiment will be described with reference to Figures 10 to 12. The fifth embodiment differs from the first to fourth embodiments described above only in some configurations and operations, and other parts may be the same as the first to fourth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0071] (Functional configuration) First, the functional configuration of the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the functional configuration of the information processing system according to the fifth embodiment. Note that in Fig. 10, the same elements as those shown in Fig. 4 are denoted by the same reference numerals.

[0072] 10, the information processing system 10 according to the fifth embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, an output unit 140, and a notification unit 150. That is, the information processing system 10 according to the fifth embodiment is configured to further include a notification unit 150 in addition to the configuration of the second embodiment (see FIG. 4).

[0073] The notification unit 150 is configured to be able to make a notification in accordance with the determination result of the determination processing unit 135. More specifically, the notification unit 150 is configured to be able to notify the user when the determination result of the determination processing unit 135 changes before and after the score correction. Note that the determination processing unit 135 according to the fifth embodiment is configured to make not only a determination based on the corrected score but also a determination based on the score before correction (hereinafter referred to as "provisional determination" as appropriate).

[0074] (Operation flow) Next, the flow of operations of the information processing system 10 according to the fifth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of operations of the information processing system according to the fifth embodiment. Note that in Fig. 11, the same processes as those shown in Fig. 5 are denoted by the same reference numerals.

[0075] 11, when the information processing system 10 according to the fifth embodiment operates, first, the face direction acquisition unit 110 acquires the face direction of the user (step S101), and the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0076] Next, threshold determination unit 131 determines whether the user's facial direction acquired by facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), difference calculation unit 132 calculates the difference between the user's facial direction and gaze direction (step S104).

[0077] Next, the score calculation unit 133 calculates a score from the difference between the user's facial orientation and gaze direction (step S201). Here, particularly in the fifth embodiment, the determination processing unit 135 makes a provisional determination as to whether or not the user is a living body based on the score before correction (step S501). Thereafter, the score correction unit 134 corrects the score according to the size of the user's facial orientation (step S202). Then, the determination processing unit 135 determines whether or not the user is a living body based on the corrected score (step S203).

[0078] Next, the notification unit 150 determines whether the determination result of the determination processing unit 135 has changed before and after the score correction (step S502). If it is determined that the determination result has changed before and after the score correction (step S502: YES), the notification unit 150 notifies that the determination result has changed due to the correction (step S503). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0079] The notification by the notification unit 150 may be performed together with the output by the output unit 140. For example, the notification by the notification unit 150 and the output by the output unit 140 may be performed using, for example, a common display. The notification by the notification unit 150 may be performed to the user, or may be performed to a system administrator or the like.

[0080] (Display example) Next, a display example in the information processing system 10 according to the fifth embodiment (specifically, a display example reflecting a notification by the notification unit 150) will be described with reference to Fig. 12. Fig. 12 is a diagram showing a display example in the information processing system according to the fifth embodiment.

[0081] As shown in FIG. 12, in the information processing system 10 according to the fifth embodiment, the determination result by the determination unit 130 (i.e., output by the output unit 140) may be displayed together with the user's facial image. Here, the user's determination result is output as "OK" (i.e., the user is alive). Particularly in the fifth embodiment, in addition to the determination result, a notification by the notification unit 150 may be displayed. Here, a message stating "Results have changed due to score correction" is displayed below the determination result. Instead of such a message, for example, a message suggesting low reliability or a message urging the user to redo the determination may be displayed.

[0082] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the fifth embodiment will be described.

[0083] 10 to 12, in the information processing system 10 according to the fifth embodiment, if the determination result as to whether the user is a living body changes before and after correcting the score, a process of notifying the user of this fact is executed. In this way, since the notification is made in a situation where the original score is near the determination threshold, it becomes possible to know, for example, that the current determination was difficult (in other words, that the reliability of the determination may be low).

[0084] Sixth Embodiment An information processing system 10 according to the sixth embodiment will be described with reference to Figures 13 and 14. The sixth embodiment differs from the first to fifth embodiments described above only in some configurations and operations, and other parts may be the same as the first to fifth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0085] (Acquisition of face direction and gaze direction) First, a method for acquiring the face direction and forward direction in the information processing system 10 according to the sixth embodiment will be specifically described with reference to Fig. 13. Fig. 13 is a graph showing the timing for acquiring the face direction and gaze direction by the information processing system according to the sixth embodiment.

[0086] 13, in the information processing system 10 according to the sixth embodiment, the facial direction and gaze direction of the user are acquired while the user is shaking his / her head (i.e., while the user is repeatedly turning his / her head to the right and left), More specifically, a plurality of images are captured while the user is shaking his / her head, and the facial direction and gaze direction of the user are acquired (estimated) from those images.

[0087] The user's facial direction and gaze direction are acquired at the timings when the user's facial direction reaches its peak (P1, P2, P3, and P4 in the figure). In this case, the user's facial direction and gaze direction are acquired four times in total: two peaks to the right and two peaks to the left. Note that the number of acquisitions here is an example, and the user's facial direction and gaze direction may be acquired, for example, four or more times.

[0088] (Example of score calculation) Next, a method for calculating a score in the information processing system 10 according to the sixth embodiment will be specifically described with reference to Fig. 14. Fig. 14 is a conceptual diagram showing a method for calculating a score by the information processing system according to the sixth embodiment.

[0089] As shown in Fig. 14, in the information processing system 10 according to the sixth embodiment, multiple differences are calculated from the facial direction and gaze direction acquired at multiple timings. Therefore, as shown in Fig. 13, when the facial direction and gaze direction of the user are acquired four times, at P1, P2, P3, and P4, the differences therebetween are calculated as a value including four values ​​corresponding to P1, P2, P3, and P4 (for example, a four-dimensional vector). The differences calculated in this manner are input to the score calculation unit 133, which calculates a score as its output. In this case, the score calculation unit 133 may be configured as, for example, a trained SVM (Support Vector Machine).

[0090] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the sixth embodiment will be described.

[0091] 13 and 14, in the information processing system 10 according to the sixth embodiment, the facial direction and gaze direction are acquired at multiple timings from a plurality of images captured while the user is shaking their head. This makes it possible to determine whether the user is a living body with greater accuracy than when the facial direction and gaze direction are acquired only once.

[0092] Seventh Embodiment An information processing system 10 according to the seventh embodiment will be described with reference to Figures 15 to 18. The seventh embodiment differs from the sixth embodiment described above only in some of its operations, and other parts may be the same as the first to sixth embodiments. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit a description of other overlapping parts as appropriate.

[0093] (Functional configuration) First, the functional configuration of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 15. Fig. 15 is a block diagram showing the functional configuration of the information processing system according to the seventh embodiment. In Fig. 15, the same elements as those shown in Fig. 2 are denoted by the same reference numerals.

[0094] 15, the information processing system 10 according to the seventh embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, an output unit 140, and a head swing instruction unit 160. That is, the information processing system 10 according to the seventh embodiment is configured to further include a head swing instruction unit 160 in addition to the configuration of the first embodiment (see FIG. 2). Furthermore, the determination unit 130 according to the seventh embodiment includes a head swing determination unit 136 in addition to a threshold determination unit 131 and a difference calculation unit 132.

[0095] The head shake determination unit 136 is configured to be able to determine whether the head shake (sixth embodiment: see FIG. 13) when acquiring the user's facial direction and gaze direction is performed appropriately. The head shake determination unit 136 may determine whether the head shake is appropriate, for example, based on whether the user is shaking their head sufficiently greatly (for example, whether the peak of the facial direction exceeds a predetermined angle) or whether the user is shaking their head in an appropriate direction (for example, whether the user is shaking their head in a different direction (for example, downward) at the timing when they should be shaking their head to the right).

[0096] The head swing instruction unit 160 is configured to be able to output an instruction to the user to perform an appropriate head swing when the head swing determination unit 136 determines that the user's head swing is inappropriate. The instruction by the head swing instruction unit 160 may be issued in conjunction with the output by the output unit 140. For example, the instruction by the head swing instruction unit and the output by the output unit 140 may be issued using, for example, a common display. Furthermore, when the user shakes their head multiple times, an instruction may be output only for some of the movements. For example, when only the face direction acquired at P3 among P1, P2, P3, and P4 in FIG. 14 is inappropriate (for example, when the angles of P1, P2, and P4 are sufficient, but only P3 is small), the head swing instruction unit 160 may output only an instruction for the head swing movement corresponding to P3. Specifically, an instruction may be output to shake the head more widely in the direction corresponding to the movement of P3.

[0097] (Operation flow) Next, the flow of operations of the information processing system 10 according to the seventh embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of operations of the information processing system according to the seventh embodiment. Note that in Fig. 16, the same processes as those shown in Fig. 3 are denoted by the same reference numerals.

[0098] 16, when the information processing system 10 according to the seventh embodiment operates, first, the face direction acquisition unit 110 acquires the face direction of the user (step S101), and the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S102).

[0099] Next, the head shake determination unit 136 determines whether the user's head shake has been performed appropriately (step S701). If it is determined that the user's head shake has not been performed appropriately (step S701: NO), the head shake instruction unit 160 outputs an instruction to the user to perform an appropriate head shake. Then, the processes of steps S101 and S102 (i.e., obtaining the face direction and line of sight direction) are executed again.

[0100] On the other hand, if it is determined that the user's head has been shook appropriately (step S701: YES), the threshold determination unit 131 determines whether the user's facial direction acquired by the facial direction acquisition unit 110 is equal to or greater than a predetermined threshold (step S103). If it is determined that the user's facial direction is not equal to or greater than the predetermined threshold (step S103: NO), the subsequent processes are omitted and the series of processes ends. On the other hand, if it is determined that the user's facial direction is equal to or greater than the predetermined threshold (step S103: YES), the difference calculation unit 132 calculates the difference between the user's facial direction and gaze direction (step S104).

[0101] Next, the determination unit 130 determines whether the user is a living body based on the difference between the user's facial orientation and gaze direction (step S105). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S106).

[0102] (Display example) Next, a display example in the information processing system 10 according to the seventh embodiment (specifically, a display example reflecting an instruction by the head swing instruction unit 160) will be described with reference to Fig. 17 and Fig. 18. Fig. 17 is a diagram (part 1) showing a display example in the information processing system according to the seventh embodiment. Fig. 18 is a diagram (part 2) showing a display example in the information processing system according to the seventh embodiment.

[0103] As shown in Fig. 17, in the information processing system 10 according to the seventh embodiment, a message urging the user to shake their head more vigorously may be displayed together with an image of the user. For example, as shown in the example of Fig. 17, a message saying "Please shake your head a little more vigorously" may be displayed. In this way, it is expected that the user will shake their head more vigorously, and as a result, it becomes possible to acquire the facial direction and gaze direction of the user when their facial direction is sufficiently large.

[0104] As shown in Fig. 18, in the information processing system 10 according to the seventh embodiment, a message indicating the direction in which the user should shake their head may be displayed together with an image of the user. For example, as shown in the example of Fig. 18, a message saying "Please shake your head a little more to the left" may be displayed. In this way, it is possible to give the user more accurate instructions when, for example, the head has been shaken sufficiently to the right but not to the left.

[0105] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the seventh embodiment will be described.

[0106] 15 to 18, in the information processing system 10 according to the seventh embodiment, if the user's head shaking is inappropriate, an instruction to shake the head appropriately is output to the user. In this way, the user's head shaking can be made appropriate, and the user's facial orientation and line of sight direction can be appropriately acquired, making it possible to more accurately determine whether the user is a living body.

[0107] Eighth Embodiment An information processing system 10 according to the eighth embodiment will be described with reference to Figures 19 and 20. Note that the information processing system 10 according to the eighth embodiment differs only in some of its operations from those of the first to seventh embodiments described above, and other parts may be the same as those of the first to seventh embodiments. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and explanations of other overlapping parts will be omitted as appropriate.

[0108] (Functional configuration) First, the functional configuration of the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 19. Fig. 19 is a block diagram showing the functional configuration of the information processing system according to the eighth embodiment. In Fig. 19, the same elements as those shown in Figs. 2 and 4 are denoted by the same reference numerals.

[0109] 19, the information processing system 10 according to the eighth embodiment is configured to include, as processing blocks for realizing its functions, a face direction acquisition unit 110, a gaze direction acquisition unit 120, a determination unit 130, and an output unit 140. The determination unit 130 according to the eighth embodiment particularly includes a difference calculation unit 132, a score calculation unit 133, a score correction unit 134, and a determination processing unit 135.

[0110] Unlike the first to seventh embodiments, the information processing system 10 according to the eighth embodiment does not include a threshold determination unit 131. Therefore, the determination unit 130 makes a determination regardless of whether the user's facial orientation is equal to or greater than a predetermined threshold. The specific operation of the determination unit 130 will be described in detail below.

[0111] (Operation flow) Next, the flow of operations of the information processing system 10 according to the eighth embodiment will be described with reference to Fig. 20. Fig. 20 is a flowchart showing the flow of operations of the information processing system according to the eighth embodiment. Note that in Fig. 20, the same processes as those shown in Figs. 3 and 5 are denoted by the same reference numerals.

[0112] 20, when the information processing system 10 according to the eighth embodiment operates, the facial direction acquisition unit 110 first acquires the facial direction of the user (step S801), and the gaze direction acquisition unit 120 acquires the gaze direction of the user (step S802).

[0113] Next, the difference calculation unit 132 calculates the difference between the user's facial direction and gaze direction (step S803). Then, the score calculation unit 133 calculates a score from the difference between the user's facial direction and gaze direction (step S804). Thereafter, the score correction unit 134 corrects the score according to the size of the user's facial direction (step S805).

[0114] Next, the determination processing unit 135 determines whether the user is a living body based on the corrected score (step S806). Then, the output unit 140 outputs the determination result of the determination unit 130 (step S807).

[0115] (Technical Effects) Next, the technical effects obtained by the information processing system 10 according to the eighth embodiment will be described.

[0116] 19 and 20, in the information processing system 10 according to the eighth embodiment, the score calculated from the difference between the facial direction and the gaze direction is corrected according to the magnitude of the facial direction, and whether or not the user is a living body is determined based on the corrected score. In this way, the score becomes an appropriate value through the correction, making it possible to more accurately determine whether or not the user is a living body.

[0117] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0118] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. The scope of each embodiment is not limited to programs recorded on the recording media that execute processing by themselves, but also includes programs that run on an OS in cooperation with other software and functions of an expansion board to execute processing.

[0119] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing systems, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0120] <Additional Notes> The above-described embodiment may be further described as follows, but is not limited to the following.

[0121] (Appendix 1) The information processing system described in Appendix 1 is an information processing system including: a facial orientation acquisition means for acquiring the facial orientation of a user; a gaze direction acquisition means for acquiring the gaze direction of the user; a determination means for determining whether the user is a living body or not based on the difference between the facial orientation and the gaze direction when the facial orientation is equal to or greater than a predetermined threshold; and an output means for outputting the result of the determination.

[0122] (Appendix 2) The information processing system described in Appendix 2 is the information processing system described in Appendix 1, wherein the determination means includes a calculation means that calculates a score indicating the liveness of the user from the difference between the facial direction and the gaze direction, a correction means that corrects the score according to the magnitude of the facial direction, and a determination processing means that determines whether the user is a live person or not based on the corrected score.

[0123] (Appendix 3) The information processing system according to Supplementary Note 3 is the information processing system according to Supplementary Note 2, wherein the correction means corrects the score so that the larger the face direction, the larger the correction amount to be subtracted from the score.

[0124] (Appendix 4) The information processing system described in Appendix 4 is the information processing system described in Appendix 2 or 3, in which the correction means stores multiple correction formulas for correcting the score, selects a correction formula according to information related to the user, and corrects the score.

[0125] (Appendix 5) The information processing system described in Appendix 5 is an information processing system described in any one of Appendixes 2 to 4, in which the judgment processing means, in addition to determining whether the user is a living organism based on the corrected score, also determines whether the user is a living organism based on the score before correction, and if the judgment result of the judgment processing means changes before and after correction, is further provided with a notification means for notifying that the judgment result has changed.

[0126] (Appendix 6) The information processing system described in Appendix 6 is the information processing system described in any one of Appendixes 1 to 5, wherein the facial direction acquisition means and the gaze direction acquisition means acquire the facial direction and the gaze direction at multiple times from multiple images captured while the user is shaking their head from side to side, and the determination means determines whether the user is a living body based on the difference between the facial direction and the gaze direction at the multiple times.

[0127] (Appendix 7) The information processing system described in Appendix 7 is the information processing system described in Appendix 6, further comprising an instruction means for instructing the user on an appropriate head movement when it is detected that the user's head movement is inappropriate.

[0128] (Appendix 8) The information processing system described in Appendix 8 is an information processing system including: a facial direction acquisition means for acquiring a facial direction of a user; a gaze direction acquisition means for acquiring a gaze direction of the user; a calculation means for calculating a score indicating the liveness of the user from the difference between the facial direction and the gaze direction; a correction means for correcting the score according to the magnitude of the facial direction; a determination processing means for determining whether the user is a live person or not based on the corrected score; and an output means for outputting the result of the determination.

[0129] (Appendix 9) The information processing method described in Appendix 9 is an information processing method that acquires a facial orientation of a user, acquires a gaze direction of the user, and, if the facial orientation is equal to or greater than a predetermined threshold, determines whether the user is a living body based on the difference between the facial orientation and the gaze direction, and outputs the result of the determination.

[0130] (Appendix 10) The recording medium described in Appendix 10 is a recording medium having recorded thereon a computer program that causes a computer to execute an information processing method that acquires a facial orientation of a user, acquires a gaze direction of the user, and, if the facial orientation is equal to or greater than a predetermined threshold, determines whether the user is a living body based on the difference between the facial orientation and the gaze direction, and outputs the result of the determination.

[0131] (Appendix 11) The computer program described in Appendix 11 is a computer program that executes an information processing method that acquires a facial orientation of a user, acquires a gaze direction of the user, and, if the facial orientation is equal to or greater than a predetermined threshold, determines whether the user is a living body based on the difference between the facial orientation and the gaze direction, and outputs the result of the determination. [Explanation of symbols]

[0132] 10 Information Processing Systems 11 processors 16 Output Devices 20 Camera 110 Face direction acquisition unit 120 Gaze direction acquisition unit 130 Judgment section 131 Threshold judgment unit 132 Difference calculation part 133 Score Calculation Section 134 Score Correction Section 1341 Correction formula storage unit 1342 Correction formula selection section 135 Judgment processing unit 136 Swing determination unit 140 Output section 150 Notification Department 160 Swing indicator

Claims

1. A face direction acquisition means for acquiring a face direction of a user; a gaze direction acquisition means for acquiring a gaze direction of the user; a determination means for determining whether the user is a living body based on a difference between the facial orientation and the gaze direction; a display means for displaying a message prompting the user to shake their head; An information processing device comprising:

2. the display means displays the message when the user's head movement is inappropriate. The information processing device according to claim 1 .

3. the display means displays the message when the orientation of the user's face is smaller than a predetermined value.

3. The information processing device according to claim 1.

4. the message is a message that prompts the user to shake their head in a direction that increases the direction of their face relative to the front, The information processing device according to claim 1 .

5. At least one computer Obtain the user's facial orientation, acquiring a gaze direction of the user; determining whether the user is a living body based on a difference between the face orientation and the gaze direction; displaying a message prompting the user to shake their head; Information processing methods.

6. At least one computer Obtain the user's facial orientation, acquiring a gaze direction of the user; determining whether the user is a living body based on a difference between the face orientation and the gaze direction; displaying a message prompting the user to shake their head; A computer program that executes an information processing method.

7. A face direction acquisition means for acquiring a face direction of a user; a gaze direction acquisition means for acquiring a gaze direction of the user; a determination means for determining whether the user is a living body based on a difference between the facial orientation and the gaze direction; a display means for displaying a message prompting the user to shake their head; An information processing system comprising:

Images