Information processing device, information processing method, and recording medium

Abstract

To provide an information processing system, an information processing device, an information processing method and a storage medium for more accurately determining whether or not impersonation occurs.SOLUTION: An information processing system SYSc comprises: an imaging apparatus which can generate a plurality of images by continuously imaging an object in time series; and an information processing device which authenticates the object by using at least one of the plurality of images. The information processing device comprises: a feature extraction unit 211 which extracts feature quantities of the object which vary according to motions of the object from each of the plurality of images; and an authentication unit 212 which determines that authentication of the object fails when the feature quantities non-continuously vary in time series among the plurality of images.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to, for example, the technical fields of an information processing system, an information processing device, an information processing method, and a recording medium that authenticate an object. [Background technology]

[0002] An information processing device that authenticates a person using an image generated by an imaging device that captures the person's face is required to prevent impersonation. Examples of such information processing devices that prevent impersonation are described in Patent Documents 1 and 2.

[0003] Other prior art documents related to this disclosure include Patent Document 3. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-067371 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-128572 [Patent Document 3] Patent No. 6727684 Summary of the Invention [Problem to be solved by the invention]

[0005] An object of this disclosure is to provide an information processing system, an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents. [Means for solving the problem]

[0006] The disclosed information processing system includes an imaging device capable of generating multiple images in which an object appears by capturing images of the object continuously in time series, and an information processing device that authenticates the object using at least one of the multiple images, wherein the information processing device includes an extraction means that extracts, from each of the multiple images, features of the object that change in accordance with the movement of the object, and an authentication means that determines that authentication of the object has failed if the features change discontinuously in time series between the multiple images.

[0007] The disclosed information processing device includes an extraction means for extracting features of an object that change in accordance with the movement of the object from each of a plurality of images in which the object appears, the images being generated by an imaging device successively capturing images of the object in chronological order, and an authentication means for determining that authentication of the object has failed if the features change discontinuously in chronological order among the plurality of images.

[0008] The disclosed information processing method generates multiple images of an object by capturing images of the object continuously in a time series, extracts features of the object that change in accordance with the movement of the object from each of the multiple images, and determines that authentication of the object has failed if the features change discontinuously in a time series between the multiple images.

[0009] The recording medium disclosed herein is a recording medium having recorded thereon a computer program that causes a computer to execute an information processing method that generates multiple images of an object by capturing images of the object continuously in time series, extracts features of the object that change in accordance with the movement of the object from each of the multiple images, and determines that authentication of the object has failed if the features change discontinuously in time series between the multiple images. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of an information processing system according to the first embodiment. [Figure 2]FIG. 2 is a block diagram showing the configuration of the information processing apparatus according to the first embodiment. [Figure 3] FIG. 3 is a flowchart showing the flow of the authentication operation performed by the information processing system of the first embodiment. [Figure 4] FIG. 4 shows a camera that captures an image of a person to be authenticated who is actually present in front of the camera. [Figure 5] FIG. 5 is a graph showing the change over time in feature amounts when a camera captures an image of a person to be authenticated who is actually in front of the camera. [Figure 6] FIG. 6 shows a camera capturing an image of a display showing a registered person who is different from the person to be authenticated and who is actually present in front of the camera. [Figure 7] FIG. 7 is a graph showing the change over time in feature amounts when the display rate of a display showing registered people is lower than the image capturing rate of a camera. [Figure 8] FIG. 8 is a graph showing the change over time in feature amounts when the display rate of the display showing the registered person is higher than the image capturing rate of the camera. [Figure 9] FIG. 9 is a block diagram showing the configuration of an information processing apparatus according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing the configuration of an information processing apparatus according to the third embodiment. [Figure 11] FIG. 11 is a flowchart showing the flow of authentication operations performed by the information processing system of the third embodiment. [Figure 12] FIG. 12 shows an example of a question screen displayed by the information processing device of the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an information processing system, an information processing device, an information processing method, and a recording medium will be described with reference to the drawings. An information processing system SYS to which the information processing system, the information processing device, the information processing method, and the recording medium are applied will be described.

[0012] (1) Information processing system SYS of the first embodiment First, an information processing system SYS according to the first embodiment will be described. In the following description, the information processing system SYS according to the first embodiment will be referred to as an "information processing system SYSa."

[0013] (1-1) Overview of the information processing system SYSa The information processing system SYSa authenticates an object (hereinafter referred to as an "authentication object") that is present in front of a camera 1 (see FIG. 1 described later) by using at least one of a plurality of images generated by the camera 1 successively capturing images of the object in time series. For example, the information processing system SYSa may authenticate the authentication object by performing a matching process to determine whether the object (hereinafter referred to as an "imaged object") that is captured in the image matches an object (hereinafter referred to as a "registered object") that has been registered in advance in the information processing system SYSa.

[0014] The target may include, for example, a person. The target may include an animal other than a person (for example, at least one of mammals such as dogs and cats, birds such as sparrows, reptiles such as snakes, amphibians such as frogs, and fish such as goldfish). The target may include an inanimate object. The inanimate object may include a robot that resembles a person or an animal.

[0015] In the following description, an example will be described in which the target is a person. In this case, the information processing system SYSa authenticates the person appearing in the person image IMG using at least one of a plurality of person images IMG generated by the camera 1 (see FIG. 1 described later) successively capturing images of people in front of the camera 1 in time series. For example, the information processing system SYSa may authenticate the person to be authenticated by performing a matching process to determine whether the person appearing in each of the plurality of person images IMG (hereinafter referred to as the "photographed person") matches a person pre-registered in the information processing system SYSa (hereinafter referred to as the "registered person").

[0016] Here, the camera 1 usually captures an image of a person to be authenticated who actually exists in front of the camera 1. Therefore, the imaged person captured in the person image IMG usually matches the person to be authenticated who actually exists in front of the camera 1. In this case, the matching process for determining whether the imaged person matches the registered person is equivalent to the matching process for determining whether the person to be authenticated matches the registered person. Therefore, when the information processing system SYSa determines that the imaged person matches the registered person, it determines that authentication of the person to be authenticated (that is, authentication of the imaged person captured in the person image IMG; the same applies hereinafter) has been successful because the person to be authenticated matches the registered person. On the other hand, when the information processing system SYSa determines that the imaged person does not match the registered person, it determines that authentication of the person to be authenticated has failed because the person to be authenticated does not match the registered person.

[0017] On the other hand, a malicious person to be authenticated may cause the camera to capture an image in which a registered person different from the person to be authenticated is captured, in order to masquerade as a registered person different from the person to be authenticated. Specifically, a malicious person to be authenticated in front of the camera 1 may place a portable display (e.g., the display of a smartphone or tablet terminal) displaying an image of the registered person in front of the camera 1. As a result, the camera 1 will capture an image of the registered person displayed on the display, rather than the person to be authenticated who is actually in front of the camera 1. In this case, even though the registered person is not actually present in front of the camera 1, the registered person will appear as the captured person in the person image IMG generated by the camera 1. As a result, the information processing system SYSa may perform a matching process and determine that the captured person captured in the person image IMG matches the registered person. This may cause the information processing system SYSa to erroneously determine that the authentication of the person to be authenticated has been successful. Specifically, the information processing system SYSa may erroneously determine that the person to be authenticated who is actually present in front of the camera 1 matches the registered person, even though the registered person is not actually present in front of the camera. In other words, the information processing system SYSa may erroneously determine that authentication of the person to be authenticated has been successful, even though it should have determined that authentication of the person to be authenticated has failed because a person to be authenticated who is different from the registered person is present in front of camera 1.

[0018] For this reason, the information processing system SYSa that authenticates the person to be authenticated is required to prevent "spoofing," which corresponds to an act by a malicious person to be authenticated to pose as a registered person different from the person to be authenticated. Therefore, the information processing system SYSa of the first embodiment performs an authentication operation to authenticate the person to be authenticated while preventing spoofing. In particular, in the first embodiment, the information processing system SYSa performs an authentication operation to authenticate the person to be authenticated while preventing spoofing, which corresponds to an act by the person to be authenticated to pose as a registered person, by using a display that displays an image of the registered person.

[0019] Specifically, as will be described in detail later, if there is no continuity in the movement of the person to be authenticated among the plurality of person images IMG, the information processing system SYSa determines that the authentication of the person to be authenticated (i.e., authentication of the person to be authenticated) has failed. Therefore, in the first embodiment, if there is no continuity in the movement of the person to be authenticated among the plurality of person images IMG, the information processing system SYSa determines that the authentication of the person to be authenticated (i.e., authentication of the person to be authenticated) has failed, even if it is determined that the person to be authenticated matches a registered person.

[0020] The configuration and operation of the information processing system SYSa that performs such authentication operations will be described in detail below.

[0021] (1-2) Configuration of information processing system SYSa First, the configuration of the information processing system SYSa of the first embodiment will be described.

[0022] (1-2-1) Overall configuration of information processing system SYSa First, the overall configuration of the information processing system SYSa of the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the overall configuration of the information processing system SYSa of the first embodiment.

[0023] As shown in Fig. 1, the information processing system SYSa includes a camera 1 and an information processing device 2. The information processing system SYSa may include a single camera 1 or multiple cameras 1. The camera 1 and the information processing device 2 can communicate with each other via a communication network 3. The communication network 3 may include a wired communication network. The communication network 3 may also include a wireless communication network.

[0024] The camera 1 is an imaging device capable of capturing images of the imaging target range continuously in time series. The camera 1 performs imaging processing that constitutes part of the authentication operation performed by the information processing system SYSa. The imaging processing includes processing for capturing images of the imaging target range to generate an image that includes the imaging target range.

[0025] Typically, the imaging target range includes a person to be authenticated who is actually present in front of camera 1. In this case, camera 1 captures images of the person to be authenticated continuously in time series, thereby generating multiple person images IMG in which the person to be authenticated appears as an imaged person. Alternatively, as described above, if the person to be authenticated is impersonating a registered person using a display displaying an image of the registered person, the imaging target range includes a display (particularly, its display screen) displaying an image of the registered person. In this case, camera 1 captures images of the registered person displayed on the display continuously in time series, thereby generating multiple person images IMG in which a registered person different from the person to be authenticated appears as an imaged person.

[0026] The camera 1 captures images of the imaging target range continuously in time series at a predetermined imaging rate. The imaging rate may indicate the number of times the camera 1 captures images of the imaging target range per unit time. For example, the camera 1 may capture images of the scene within the imaging target range continuously in time series at an imaging rate of 30 fps (frames per second). An imaging rate of "x fps" (where x is a positive real number) means an imaging rate at which the imaging target range can be captured x times per second (i.e., x person images IMG can be generated per second).

[0027] The camera 1 outputs the generated plurality of person images IMG to the information processing device 2. Specifically, the camera 1 transmits the generated plurality of person images IMG to the information processing device 2 via the communication network 3.

[0028] The information processing device 2 receives a plurality of person images IMG transmitted from the camera 1 via the communication network 3. The information processing device 2 performs an authentication process, which is a part of the authentication operation performed by the information processing system SYSa, using the received plurality of person images IMG. The authentication process is a process for authenticating a person to be authenticated while preventing impersonation, using the plurality of person images IMG. In the first embodiment, in particular, the authentication process includes a continuity determination process for determining whether there is continuity in the movement of the person to be authenticated among the plurality of person images IMG. If it is determined as a result of the continuity determination process that there is no continuity in the movement of the person to be authenticated among the plurality of person images IMG, as described above, the information processing device 2 determines that authentication of the person to be authenticated has failed. Note that the configuration of the information processing device 2 and the authentication process performed by the information processing device 2 (i.e., a part of the authentication operation) will be described in detail later with reference to the drawings, and therefore will not be described here.

[0029] As described above, when the information processing system SYSa includes a plurality of cameras 1, the information processing device 2 may perform a separate authentication process for each camera 1. For example, the information processing device 2 may perform a first authentication process to authenticate a person to be authenticated who is in front of the first camera 1, using a plurality of person images IMG transmitted from a first camera 1 among the plurality of cameras 1. For example, the information processing device 2 may perform a second authentication process to authenticate a person to be authenticated who is in front of the second camera 1, using a plurality of person images IMG transmitted from a second camera 1 different from the first camera 1 among the plurality of cameras 1, in parallel with or before or after the first authentication process.

[0030] Such an information processing system SYSa may perform an authentication operation to determine whether or not to permit the person to be authenticated to enter a predetermined area. In this case, the camera 1 may be disposed, for example, at an entrance / exit of the predetermined area that the person to be authenticated can use to enter the predetermined area. The registered person may also be a person authorized to enter the predetermined area. For example, if the information processing system SYSa determines that the authentication of the person to be authenticated is successful, it may determine to permit the person to be authenticated to enter the predetermined area. In this case, the information processing system SYSa may set the state of a gate device disposed at the entrance / exit of the predetermined area to an open state that does not prevent the person to be authenticated from entering the predetermined area. On the other hand, for example, if the information processing system SYSa determines that the authentication of the person to be authenticated is unsuccessful, it may determine not to permit the person to be authenticated to enter the predetermined area. In this case, the information processing system SYSa may set the state of a gate device disposed at the entrance / exit of the predetermined area to a closed state that prohibits the person to be authenticated from entering the predetermined area. However, the situation in which the information processing system SYSa performs the authentication operation is not limited to the situation in which it is determined whether or not to permit the person to be authenticated to enter a predetermined area.

[0031] (1-2-2) Configuration of information processing device 2 Next, the configuration of the information processing device 2 of the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the information processing device 2 of the first embodiment.

[0032] 2, the information processing device 2 includes a calculation device 21, a storage device 22, and a communication device 23. The information processing device 2 may further include an input device 24 and an output device 25. However, the information processing device 2 does not necessarily have to include at least one of the input device 24 and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0033] The arithmetic device 21 includes, for example, at least one of a CPU, a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (not shown). The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing processing to be performed by the information processing device 2 (for example, the above-mentioned authentication processing, particularly the authentication processing including the continuity determination processing) are realized within the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing processing to be performed by the information processing device 2.

[0034] FIG. 2 shows an example of logical functional blocks implemented within the arithmetic device 21. As shown in FIG. 2, the arithmetic device 21 implements a feature extraction unit 211, which is a specific example of "extraction means," and an authentication unit 212, which is a specific example of "authentication means." The feature extraction unit 211 extracts feature amounts of the captured person from each of the multiple person images IMG. In particular, the feature extraction unit 211 extracts feature amounts that can be used to determine whether there is continuity in the movement of the captured person. The authentication unit 212 determines whether there is continuity in the movement of the captured person among the multiple person images IMG. If it is determined based on the feature amounts that there is no continuity in the movement of the captured person among the multiple person images IMG, the authentication unit 212 determines that authentication of the person to be authenticated (i.e., authentication of the captured person) has failed.

[0035] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0036] The communication device 23 is capable of communicating with the camera 1 via the communication network 3 (see FIG. 1). In the first embodiment, the communication device 23 receives (i.e., acquires) a person image IMG from the camera 1 via the communication network 3.

[0037] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0038] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper.

[0039] (1-3) Authentication operation performed by information processing system SYSa Next, the authentication operation performed by the entrance management system SYSa will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the authentication operation performed by the entrance management system SYSa.

[0040] As shown in FIG. 3, the camera 1 captures images of the imaging target range continuously in time series (step S11). That is, the camera 1 performs imaging processing (step S11). As a result, the camera 1 generates a plurality of person images IMG. As described above, when a person to be authenticated who actually exists in front of the camera 1 is present in the imaging target range, each of the plurality of person images IMG includes the person to be authenticated who actually exists in front of the camera 1 as the imaged person. On the other hand, when a display displaying an image of a registered person different from the person to be authenticated who actually exists in front of the camera 1 is present in the imaging target range, each of the plurality of person images IMG includes the registered person displayed on the display as the imaged person. The camera 1 transmits the generated plurality of person images IMG to the information processing device 2.

[0041] Following or in parallel with the imaging process performed by the camera 1, the information processing device 2 performs authentication process (steps S12 to S15). For convenience of explanation, Fig. 3 shows an example in which the information processing device 2 performs authentication process following the imaging process performed by the camera 1. However, the information processing device 2 may perform authentication process in parallel with the imaging process performed by the camera 1.

[0042] To perform the authentication process, the feature extraction unit 211 extracts feature amounts of the captured person appearing in the person image IMG from each of the multiple person images IMG generated in step S11 (step S12). In particular, as described above, the feature extraction unit 211 extracts feature amounts that can be used to determine whether or not there is continuity in the movement of the captured person. In the first embodiment, the feature amounts that can be used to determine whether or not there is continuity in the movement of the captured person are typically feature amounts of the captured person that change depending on the movement of the captured person.

[0043] When the person to be imaged moves, a certain part of the person's body may move. For this reason, in step S12, the feature extraction unit 211 may extract a feature amount related to the position of a certain part of the person's body (i.e., a certain feature point of the person's body). For example, at least one of a feature amount related to the position of at least a part of the person's head, a feature amount related to the position of at least a part of the person's face, a feature amount related to the position of at least a part of the person's hands, a feature amount related to the position of at least a part of the person's torso, a feature amount related to the position of at least a part of the person's legs, a feature amount related to the position of at least a part of the person's eyebrows, a feature amount related to the position of at least a part of the person's eyelashes, a feature amount related to the position of at least a part of the person's eyes, a feature amount related to the position of at least a part of the person's eyeballs, and a feature amount related to the position of at least a part of the person's mouth may be extracted.

[0044] The feature amount extracted in step S12 is used to determine whether or not there is continuity in the movement of the photographed person. Therefore, the process of step S12 may constitute a part of the continuity determination process. In other words, the feature extraction unit 211 may perform the process of step S12 as a part of the continuity determination process.

[0045] Thereafter, the authentication unit 212 determines whether or not there is continuity in the movement of the captured person among the multiple person images IMG based on the feature amount extracted in step S12 (step S13). That is, the feature extraction unit 211 performs the process of step S13 as part of the continuity determination process.

[0046] In the first embodiment, in order to determine whether there is continuity in the movement of the captured person, the authentication unit 212 determines, for example, whether the feature amount extracted in step S12 changes continuously over time among the multiple person images IMG. If it is determined that the feature amount changes continuously over time among the multiple person images IMG, the authentication unit 212 determines that there is continuity in the movement of the captured person. On the other hand, if it is determined that the feature amount does not change continuously over time among the multiple person images IMG, the authentication unit 212 determines that there is no continuity in the movement of the captured person. In other words, if it is determined that the feature amount changes discontinuously over time among the multiple person images IMG, the authentication unit 212 determines that there is no continuity in the movement of the captured person.

[0047] If it is determined in step S13 that there is continuity in the movement of the person to be authenticated (step S13: Yes), the authentication unit 212 determines that there is no possibility that the person to be authenticated is masquerading as a different person from the person to be authenticated. In other words, the authentication unit 212 determines that the person to be authenticated who actually exists in front of the camera 1 is captured as the person to be authenticated in the person image IMG. In this case, the authentication unit 212 determines that authentication of the person to be authenticated has been successful (step S14).

[0048] However, the authentication unit 212 may determine that the authentication of the person to be authenticated has been successful not only if there is continuity in the movements of the imaged person, but also if the condition that the imaged person matches a registered person is met. Therefore, in the first embodiment, the authentication unit 212 may perform a matching process to determine whether the imaged person matches a registered person, in addition to the continuity determination process. Information about the registered person (e.g., a database) may be stored in, for example, the storage device 22. In this case, the authentication unit 212 may perform the matching process using the database stored in the storage device 22.

[0049] On the other hand, if it is determined in step S13 that there is no continuity in the movement of the person to be imaged (step S13: No), the authentication unit 212 determines that there is a possibility that the person to be authenticated is impersonating a person different from the person to be authenticated (typically, a registered person). In other words, the authentication unit 212 determines that a person different from the person to be authenticated who actually exists in front of the camera 1 (typically, a registered person) appears as the imaged person in the person image IMG. For example, the authentication unit 212 determines that a person displayed on a display in front of the camera 1 (typically, a registered person) appears as the imaged person in the person image IMG. In this case, the authentication unit 212 determines that authentication of the person to be authenticated has failed (step S15).

[0050] In summary, in the first embodiment, when there is continuity in the movement of the captured person among the multiple person images IMG (i.e., the feature amounts of the captured person change continuously over time), the authentication unit 212 determines that there is no possibility that the person to be authenticated is masquerading as a different person from the person to be authenticated (as a result, authentication of the person to be authenticated is successful). In other words, the authentication unit 212 determines that there is no possibility that masquerading has occurred (as a result, authentication of the person to be authenticated is successful). On the other hand, when there is no continuity in the movement of the captured person among the multiple person images IMG (i.e., the feature amounts of the captured person change discontinuously over time), the authentication unit 212 determines that there is a possibility that the person to be authenticated is masquerading as a different person from the person to be authenticated (as a result, authentication of the person to be authenticated has failed). In other words, the authentication unit 212 determines that there is a possibility that masquerading has occurred (as a result, authentication of the person to be authenticated has failed). In this way, the authentication unit 212 can appropriately determine whether masquerading has occurred by performing the continuity determination process. The technical reasons why it is possible to appropriately determine whether or not spoofing has occurred by performing the continuity determination process will be described below with reference to FIGS.

[0051] FIG. 4 shows camera 1 capturing an image of a person to be authenticated who is actually present in front of camera 1. In this case, camera 1 captures images of the person to be authenticated continuously in time series at a timing corresponding to the imaging rate of camera 1. As a result, the feature amounts of the captured person are likely to change in a manner corresponding to the movement of the person to be authenticated. For example, FIG. 5 is a graph showing the temporal change in feature amounts under a situation in which camera 1 captures an image of a person to be authenticated who is actually present in front of camera 1. As shown in FIG. 5, the feature amounts of the captured person are likely to change faithfully to the movement of the person to be authenticated (in other words, to follow the movement of the person to be authenticated). In other words, the feature amounts of the captured person are likely to change in a smooth manner in accordance with the movement of the person to be authenticated. In particular, the feature amounts of the captured person are likely to change in accordance with the movement of the person to be authenticated at a period corresponding to the imaging rate. In this case, in the first embodiment, authentication unit 212 may determine that the feature amounts of the captured person change continuously in time series. Therefore, in the first embodiment, the state in which "the feature values ​​of the photographed person change continuously over time" (i.e., the state in which there is continuity in the movement of the photographed person) may also mean the state in which "the feature values ​​of the photographed person change in a manner that corresponds to the movement of the person to be authenticated."

[0052] On the other hand, FIG. 6 shows a camera 1 capturing an image of a display displaying an image of a registered person who is different from the person to be authenticated and who is actually present in front of the camera 1. In this case, the camera 1 captures images of the registered person displayed on the display in chronological order at a timing according to the imaging rate of the camera 1. Here, the display generally displays images at a predetermined display rate (i.e., display rate). The display rate may indicate the number of times the display displays (i.e., updates or rewrites) images per unit time. For example, the display may display images at a display rate of 60 fps. A display rate of "x fps" (x is a positive real number) means a display rate at which an image can be displayed x times per second. In this case, depending on the magnitude relationship between the imaging rate of the camera 1 and the display rate of the display, the feature values ​​of the captured person are likely to change in a manner different from the manner of change corresponding to the movement of the person to be authenticated. Typically, when the imaging rate of the camera 1 is different from the display rate of the display, the feature values ​​of the captured person are likely to change in a manner different from the manner of change corresponding to the movement of the person to be authenticated.

[0053] For example, FIG. 7 is a graph showing the change over time in feature amounts when a display is displaying multiple images of a registered person making a certain movement at a display rate of DR#1 [fps], and camera 1 captures the images at an imaging rate of IR#1 [fps] (where IR#1>DR#1). That is, FIG. 7 is a graph showing the change over time in feature amounts when the display rate of the display displaying the registered person is lower than the imaging rate of camera 1. As shown in FIG. 7, when the display rate is lower than the imaging rate of camera 1, the feature amounts of the imaged person are likely to change in a manner different from the manner of change corresponding to the movement of the person to be authenticated. Specifically, as shown in FIG. 7, at time t1, the display displays display image A, in which the registered person in a first state is captured. As a result, at time t1, camera 1 captures display image A displayed on the display to generate person image IMG#1 in which the registered person in the first state is captured. Thereafter, at time t2, when an imaging period IC#1 corresponding to the reciprocal of the imaging rate IR#1 has elapsed since time t1, camera 1 captures the display again. At time t2, the display continues to display display image A because the display cycle DC#1, which is the reciprocal of the display rate DR#1, has not yet elapsed since time t1. As a result, camera 1 captures display image A again to generate person image IMG#2, in which the registered person in the first state appears. Thereafter, at time t3, when the imaging cycle IC#1 has elapsed since time t2, camera 1 captures the display again. At time t3, the display cycle DC#1 has elapsed since time t1, so the display displays display image B, in which the registered person has changed from the first state to the second state in accordance with the registered person's movement. As a result, at time t3, camera 1 captures display image B again to generate person image IMG#3, in which the registered person in the second state appears. Similar operations are then repeated. As a result, at time t4, which is one imaging period IC#1 after time t3, camera 1 captures display image B displayed on the display to generate person image IMG#4, which includes the registered person in the second state.Furthermore, at time t5, when imaging cycle IC#1 has elapsed since time t4, and at time t6, when imaging cycle IC#1 has elapsed since time t5, the display displays display image C, in which the registered person changes from the second state to the third state in accordance with the registered person's movement. As a result, at times t5 and t6, camera 1 captures display image C displayed on the display to generate person images IMG#5 and #6, in which the registered person in the third state is captured. Furthermore, at time t7, when imaging cycle IC#1 has elapsed since time t6, and at time t8, when imaging cycle IC#1 has elapsed since time t7, the display displays display image D, in which the registered person changes from the third state to the fourth state in accordance with the registered person's movement. As a result, at times t7 and t8, camera 1 captures display image D displayed on the display to generate person images IMG#7 and #8, in which the registered person in the fourth state is captured. Furthermore, at time t9, when imaging cycle IC#1 has elapsed since time t8, and at time t10, when imaging cycle IC#1 has elapsed since time t9, the display displays display image E, in which the registered person changes from state 4 to state 5 in accordance with the registered person's movements. As a result, at times t9 and t10, camera 1 captures display image E displayed on the display to generate person images IMG#9 and IMG#10 in which the registered person in state 5 is captured. In this case, as shown in FIG. 7, the feature values ​​of the imaged person are likely to not change faithfully to the movement of the person to be authenticated (in other words, they change so as not to follow the movement of the person to be authenticated). In other words, the feature values ​​of the imaged person are likely to change in a discontinuous manner that does not match the movement of the person to be authenticated. This is because camera 1 captures the same display image displayed on the display at different times in an overlapping manner.

[0054] On the one hand, for example, FIG. 8 is a graph showing the time change of feature amounts in a situation where a display that displays images of a registered person captured while making a certain movement at a display rate of DR#2 [fps] is imaged by camera 1 at an imaging rate of IR#2 [fps] (where IR#2 < DR#2). That is, FIG. 8 is a graph showing the time change of feature amounts in a situation where the display rate of the display for displaying the registered person is higher than the imaging rate of camera 1. As shown in FIG. 8, when the display rate of the display is higher than the imaging rate of camera 1, the feature amounts of the imaged person are likely to change in a manner different from the change pattern corresponding to the movement of the authentication target person. Specifically, as shown in FIG. 8, at time t1, the display displays a display image A in which a registered person in the first state is captured. As a result, at time t1, camera 1 generates a person image IMG#1 in which a registered person in the first state is captured by imaging the display image A displayed on the display. Then, at time t2 when an imaging cycle IC#2 corresponding to the reciprocal of the imaging rate IR#2 has elapsed from time t1, camera 1 images the display again. At time t2, since a display cycle DC#2 corresponding to the reciprocal of the display rate DR#2 has already elapsed from time t1, the display displays a display image B in which the registered person has changed from the first state to the second state in accordance with the movement of the registered person. As a result, at time t2, camera 1 generates a person image IMG#2 in which a registered person in the second state is captured by imaging the display image B displayed on the display. Then, at time t3 when the imaging cycle IC#2 has elapsed from time t2, camera 1 images the display again. Here, at time t3, a period corresponding to three times the display cycle DC#2 has elapsed from time t1. As a result, the display has already finished displaying a display image C in which the registered person has changed from the second state to the third state in accordance with the movement of the registered person, and is displaying a display image D in which the registered person has changed from the third state to the fourth state in accordance with the movement of the registered person.As a result, at time t3, camera 1 captures display image D displayed on the display without capturing display image C in which the registered person in the third state is captured, thereby generating person image IMG#3 in which the registered person in the fourth state is captured. In other words, person images IMG#1 to #3 do not capture a registered person moving naturally, transitioning from the first state to the second and third states to the fourth state, but rather a registered person moving unnaturally (i.e., non-continuously), transitioning from the first state to the second state and then to the fourth state without passing through the third state. Similar operations are repeated thereafter. As a result, at time t4, when imaging cycle IC#2 has elapsed since time t3, the display displays display image E in which the registered person has changed from the fourth state to the fifth state in accordance with the registered person's movement. As a result, at time t4, camera 1 captures display image E displayed on the display, thereby generating person image IMG#4 in which the registered person in the fifth state is captured. Furthermore, at time t5, when imaging cycle IC#2 has elapsed since time t4, the display displays display image G, in which the registered person appears, changing from the fifth state to the sixth state to the seventh state in accordance with the registered person's movements. As a result, at time t5, camera 1 captures display image G displayed on the display to generate person image IMG#5, in which the registered person in the seventh state appears. Furthermore, at time t6, when imaging cycle IC#2 has elapsed since time t5, the display displays display image H, in which the registered person appears, changing from the seventh state to the eighth state in accordance with the registered person's movements. As a result, at time t6, camera 1 captures display image H displayed on the display to generate person image IMG#6, in which the registered person in the eighth state appears. Furthermore, at time t7, when imaging cycle IC#2 has elapsed since time t6, the display displays display image J, in which the registered person appears, changing from the eighth state to the ninth state to the tenth state in accordance with the registered person's movements. As a result, at time t7, camera 1 captures display image J displayed on the display, thereby generating person image IMG#7 in which the registered person in the tenth state is captured.In this case, as shown in Fig. 8, the feature amounts of the imaged person are likely to not change faithfully to the movement of the person to be authenticated (in other words, they are likely to change in a manner that does not follow the movement of the person to be authenticated). In other words, the feature amounts of the imaged person are likely to change in a discontinuous manner that does not match the movement of the person to be authenticated. This is because person images IMG#1 to #7 do not depict a registered person moving naturally, transitioning from state 1 to state 10 via states 2, 3, 4, 5, 6, 7, 8, and 9 in order, but rather depict a registered person moving unnaturally (i.e., discontinuously) to state 10 without transitioning from state 1 to state 3, 6, and 9.

[0055] Thus, even if the display shows multiple images in which a registered person making a certain movement appears, the feature values ​​of the captured person are likely to change discontinuously over time. In other words, even if the registered person displayed on the display moves in the same way as the person to be authenticated who is actually standing in front of the camera 1, the feature values ​​of the captured person captured in the person image IMG generated by capturing the registered person displayed on the display are likely to change in a different manner from the feature values ​​of the captured person captured in the person image IMG generated by capturing the person to be authenticated. In this case, in the first embodiment, the authentication unit 212 determines that the feature values ​​of the captured person are changing discontinuously over time. Therefore, in the first embodiment, the state in which "the feature values ​​of the photographed person change discontinuously over time" (i.e., the state in which there is no continuity in the movement of the photographed person) may also mean the state in which "the feature values ​​of the photographed person change in a manner different from the manner of change corresponding to the movement of the person to be authenticated."

[0056] In the example shown in FIG. 7 , the feature amounts of the imaged person change at certain times (e.g., times t3, t5, t7, and t9) but do not change at other times (e.g., times t2, t4, t6, t8, and t10). That is, in the example shown in FIG. 7 , the feature amounts of the imaged person change in a manner in which feature amounts that remain unchanged compared to feature amounts at previous and subsequent times appear at regular intervals. Therefore, the state in which "the feature amounts of the imaged person change in a manner different from the manner in which they change depending on the movement of the person to be authenticated" may include, for example, a state in which "feature amounts that remain unchanged compared to feature amounts at previous and subsequent times appear at regular intervals," as shown in FIG. 7 . In this case, the authentication unit 212 may determine whether there is continuity in the movement of the imaged person by determining whether feature amounts that remain unchanged compared to feature amounts at previous and subsequent times appear at regular intervals. For example, if the authentication unit 212 determines that feature amounts that remain unchanged compared to feature amounts at previous and subsequent times appear at regular intervals, it may determine that there is no continuity in the movement of the imaged person. For example, if the authentication unit 212 determines that a feature that has not changed compared to the feature at the previous or next time does not appear at a regular interval, it may determine that there is continuity in the movement of the person being photographed.

[0057] 8, it can be said that the feature amounts of the photographed person change in such a manner that the amount of deviation (i.e., difference) from the approximation line (i.e., regression curve) connecting the feature amounts increases at certain times. In other words, in the example shown in Fig. 8, the feature amounts of the photographed person change in such a manner that the difference between the feature amount and the approximation line increases by a certain amount or more at regular intervals. Therefore, the state in which "the feature amounts of the photographed person change in a manner different from the manner of change corresponding to the movement of the person to be authenticated" may include, for example, a state in which "the difference between the feature amounts and the approximation line becomes greater than or equal to a predetermined threshold in a regular cycle (i.e., the phenomenon in which the difference between the feature amounts and the approximation line becomes greater than or equal to a predetermined threshold is observed in a regular cycle)" as shown in FIG. 8. In this case, the authentication unit 212 may determine whether there is continuity in the movement of the photographed person based on the difference between the feature amounts and the approximation line connecting the feature amounts. For example, if the authentication unit 212 determines that the difference between the feature amounts and the approximation line becomes greater than or equal to a predetermined threshold in a regular cycle, it may determine that there is continuity in the movement of the photographed person. For example, if the authentication unit 212 determines that the difference between the feature amounts and the approximation line does not become greater than or equal to a predetermined threshold in a regular cycle, it may determine that there is continuity in the movement of the photographed person.

[0058] (1-4) Technical Effects of Information Processing System SYSa As described above, the information processing system SYSa of the first embodiment focuses on the significant difference between the continuity of the imaged person's movement when impersonation is not occurring and the continuity of the imaged person's movement when impersonation is occurring, and determines whether impersonation is occurring based on the continuity of the imaged person's movement. Therefore, the information processing system SYSa (particularly, the information processing device 2 that actually performs the authentication process) can determine whether impersonation is occurring with higher accuracy than the information processing device of the comparative example that determines whether impersonation is occurring without focusing on the continuity of the imaged person's movement. As a result, the information processing system SYSa can more appropriately prevent impersonation. In other words, the information processing system SYSa (particularly, the information processing device 2 that actually performs the authentication process) can accurately authenticate the person to be authenticated by more appropriately preventing impersonation.

[0059] Note that, since the authentication unit 212 determines whether there is continuity in the movement of the person to be imaged in order to determine whether impersonation is occurring, it is preferable that the person to be authenticated moves in front of the camera 1. For this reason, the calculation device 21 may use the output device 25 to instruct the person to be authenticated to move in front of the camera 1. For example, the calculation device 21 may use the output device 25 to instruct the person to be authenticated to make a specific movement in front of the camera 1. An example of the specific movement is a continuous movement of the person to be authenticated. An example of the continuous movement of the person to be authenticated is at least one of the movement of the person to be authenticated walking, the movement of the person to be authenticated running, the movement of the person to be authenticated jumping, and the movement of the person to be authenticated crouching. An example of the specific movement is a continuous movement of a certain part of the body of the person to be authenticated. Examples of continuous movements of a certain part of the body of the person to be authenticated include at least one of the following movements: raising a hand, lowering a hand, waving a hand, raising a leg, lowering a leg, shaking a leg, tilting a face, shaking one's head (i.e., shaking one's face), lifting an eyebrow, opening an eye, closing an eye, moving an eyeball in a predetermined direction, opening a mouth, and closing a mouth.

[0060] Alternatively, camera 1 may capture an image of the person to be authenticated walking toward camera 1. For example, if camera 1 is placed at the entrance / exit of a predetermined area, the person to be authenticated may walk toward camera 1 (i.e., toward the entrance / exit) to enter the predetermined area. In this case, if camera 1 captures an image of the person to be authenticated walking toward camera 1, feature extraction unit 211 can extract, from the multiple person images IMG generated by camera 1, feature amounts that change in accordance with the movement of the person to be authenticated walking toward camera 1. As a result, authentication unit 212 can determine whether or not impersonation is occurring based on the continuity of the movement of the person to be authenticated walking toward camera 1. In this case, the person to be authenticated does not need to actively or intentionally make a specific movement, which reduces the effort required of the person to be authenticated.

[0061] (2) Information processing system SYS of the second embodiment Next, an information processing system SYS of a second embodiment will be described. In the following description, the information processing system SYS of the second embodiment will be referred to as "information processing system SYSb." The information processing system SYSb of the second embodiment differs from the information processing system SYSa of the first embodiment described above in that it includes an information processing device 2b instead of the information processing device 2. Other features of the information processing system SYSb may be the same as other features of the information processing system SYSa. Therefore, hereinafter, the information processing device 2b of the second embodiment will be described with reference to FIG. 9. FIG. 9 is a block diagram showing the configuration of the information processing device 2b of the second embodiment.

[0062] 9, the information processing device 2b of the second embodiment differs from the information processing device 2 of the first embodiment in that a rate change unit 213b is implemented as a logical functional block in the calculation device 21. The rate change unit 213b is a specific example of "change means." Other features of the information processing device 2b may be the same as other features of the information processing device 2.

[0063] The rate changer 213b can change the imaging rate of camera 1. For example, the rate changer 213b may select one rate candidate from a plurality of rate candidates that can be set as the imaging rate of camera 1, and change the imaging rate of camera 1 to the selected rate candidate. In this case, the rate changer 213b may randomly select one rate candidate from the plurality of rate candidates. Alternatively, the rate changer 213b may select one rate candidate from the plurality of rate candidates in accordance with a predetermined selection criterion. For example, the rate changer 213b may change the imaging rate of camera 1 so that the imaging rate after the change is higher than the imaging rate before the change. For example, the rate changer 213b may change the imaging rate of camera 1 so that the imaging rate after the change is lower than the imaging rate before the change.

[0064] When the image capturing rate is changeable in this way, the information processing device 2b can more appropriately determine whether or not spoofing has occurred, compared to when the image capturing rate is not changeable (i.e., the image capturing rate is fixed), for the following reasons.

[0065] First, to determine whether impersonation using a display displaying an image of a registered person is occurring, the information processing device 2b utilizes the fact that when the camera 1 captures an image of a display displaying an image of a registered person performing a certain movement, the movement of the imaged person becomes discontinuous, as described above. On the other hand, if the display rate matches the image capture rate of the camera 1, the continuity of the movement of the imaged person may be maintained even when the camera 1 captures an image of the registered person performing a certain movement on the display. This is because the timing at which the camera 1 captures the image of the registered person displayed on the display and the timing at which the display redisplays the image of the registered person are synchronized. As a result, the information processing device 2b may erroneously determine that impersonation is not occurring when impersonation is actually occurring. In other words, the information processing device 2b may erroneously determine that authentication of the person to be authenticated is successful when the person to be authenticated is impersonating the registered person using a display displaying an image of the registered person.

[0066] However, when the imaging rate of camera 1 is changed, the display rate of the display is more likely to differ from the imaging rate of camera 1 compared to when the imaging rate of camera 1 is not changed. As a result, when camera 1 captures an image of a registered person displayed on the display, the display rate is more likely to differ from the imaging rate of camera 1. This is because the display rate is usually not changed, and information regarding the imaging rate of camera 1 is information that a malicious person to be authenticated who is impersonating a registered person cannot know. Therefore, when camera 1 captures an image of a display displaying an image of a registered person performing a certain movement, the continuity of the captured person's movement is more likely to be lost. As a result, information processing device 2b can appropriately determine that impersonation is occurring even when impersonation is occurring. In other words, information processing device 2b can more accurately determine whether impersonation is occurring.

[0067] For example, if the image capture rate of camera 1 is fixed at 60 fps while the display rate is set to 60 fps, information processing device 2b may erroneously determine that impersonation is not occurring when impersonation is occurring. On the other hand, if the image capture rate of camera 1 is changed from 60 fps to 90 fps, which is higher than 60 fps, under such circumstances, the feature values ​​of the captured person are likely to change discontinuously over time, as described with reference to FIG. 7 . Alternatively, if the image capture rate of camera 1 is changed from 60 fps to 30 fps, which is lower than 60 fps, under such circumstances, the feature values ​​of the captured person are likely to change discontinuously over time, as described with reference to FIG. 8 . As a result, information processing device 2b is less likely to erroneously determine that impersonation is not occurring when impersonation is occurring. In other words, information processing device 2b can appropriately determine that impersonation is occurring when impersonation is occurring.

[0068] The rate change unit 213b may change the image capture rate of the camera 1 at random timing. In this case, compared to when the rate change unit 213b changes the image capture rate of the camera 1 at predetermined timing, the possibility that a malicious person to be authenticated will notice a change in the image capture rate of the camera 1 is reduced. This makes it difficult for a malicious person to be authenticated to take measures to make the display rate of the display match the image capture rate of the camera 1. As a result, the information processing device 2b can appropriately determine that spoofing is occurring when spoofing is occurring.

[0069] The rate changer 213b may change the imaging rate of the camera 1 during a period in which the same person to be authenticated is present in front of the camera 1. For example, the rate changer 213b may change the imaging rate of the camera 1 from a first imaging rate to a second imaging rate during a period in which the same person to be authenticated is present in the imaging range of the camera 1. That is, the rate changer 213b may change the imaging rate of the camera 1 from the first imaging rate to the second imaging rate during a period in which the camera 1 is imaging the same person to be authenticated. In this case, the camera 1 may image the person to be authenticated at the first imaging rate, and then image the person to be authenticated at the second imaging rate. Alternatively, for example, the rate changer 213b may change the imaging rate of the camera 1 from the first imaging rate to the second imaging rate during a period in which a display used by the same malicious person to be authenticated is present in the imaging range of the camera 1. That is, the rate change unit 213b may change the imaging rate of the camera 1 from the first imaging rate to the second imaging rate while the camera 1 is capturing an image of the same display (that is, an image of the same registered person displayed on the same display). In this case, the camera 1 may capture an image of the registered person displayed on the display at the first imaging rate, and then capture an image of the same registered person displayed on the display at the second imaging rate.

[0070] In this case, the authentication unit 212 may determine that spoofing has occurred (and as a result, authentication of the person to be authenticated has failed) when a first condition is met, that is, there is no continuity in the movement of the person to be captured among the multiple person images IMG generated by capturing images of the image capture target range at a first image capture rate. Alternatively, the authentication unit 212 may determine that spoofing has occurred (and as a result, authentication of the person to be authenticated has failed) when, in addition to or instead of the first condition, a second condition is met, that there is no continuity in the movement of the person to be captured among the multiple person images IMG generated by capturing images of the image capture target range at a second image capture rate. In other words, the authentication unit 212 may determine that spoofing has occurred (and as a result, authentication of the person to be authenticated has failed) when at least one of the first and second conditions is met. As a result, the authentication unit 212 can appropriately determine whether spoofing has occurred even when the image capture rate of the camera 1 is changed.

[0071] As described above, the information processing system SYSb (particularly, the information processing device 2b) of the second embodiment can more appropriately (e.g., more accurately) determine whether or not impersonation has occurred, while enjoying the same effects as those that can be enjoyed by the information processing system SYSa (particularly, the information processing device 2) of the first embodiment described above.

[0072] (3) Information processing system SYS of the third embodiment Next, an information processing system SYS of a third embodiment will be described. In the following description, the information processing system SYS of the third embodiment will be referred to as "information processing system SYSc." The information processing system SYSc of the third embodiment differs from the information processing system SYSa of the first embodiment or the information processing system SYS2b of the second embodiment in that it includes an information processing device 2c instead of the information processing device 2. Other features of the information processing system SYSc may be the same as other features of the information processing system SYSa or the information processing system SYS2b. Therefore, hereinafter, the information processing device 2c of the third embodiment will be described with reference to FIG. 10. FIG. 10 is a block diagram showing the configuration of the information processing device 2c of the third embodiment.

[0073] As shown in FIG. 10 , the information processing device 2c of the third embodiment differs from the information processing device 2 of the first embodiment or the information processing device 2b of the second embodiment in that a matching unit 214c and a question processing unit 215c are implemented as logical functional blocks within the calculation device 21. Note that, in FIG. 10 , the information processing device 2c includes a rate changing unit 213b, just like the information processing device 2b, but the information processing device 2c does not necessarily include the rate changing unit 213b, just like the information processing device 2. Furthermore, the information processing device 2c of the third embodiment differs from the information processing device 2 of the first embodiment in that the storage device 22 stores a matching DB 221c and a question DB 222c. Other features of the information processing device 2c may be the same as those of the information processing device 2.

[0074] Next, the authentication operation performed by the information processing system SYSc of the third embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the authentication operation performed by the information processing system SYSc of the third embodiment.

[0075] As shown in Fig. 11, camera 1 captures images of the imaging target range continuously in time series (step S21c). That is, camera 1 performs imaging processing (step S21c). Note that the processing in step S21c in Fig. 11 may be the same as the processing in step S11 in Fig. 3. Therefore, a detailed description of the processing in step S21c in Fig. 11 will be omitted.

[0076] Following or in parallel with the imaging process performed by the camera 1 in step S21c, the information processing device 2c performs a matching process (steps S22c to S23c). For convenience of explanation, Fig. 11 shows an example in which the information processing device 2c performs the matching process following the imaging process performed by the camera 1. However, the information processing device 2c may perform the matching process in parallel with the imaging process performed by the camera 1.

[0077] To perform the matching process, the feature extraction unit 211 extracts feature amounts of the captured person appearing in at least one of the multiple person images IMG generated in step S11 (step S22c). The feature amounts extracted in step S22c are used to determine whether the captured person matches a registered person. Therefore, the process of step S22c may constitute part of the matching process. In other words, the feature extraction unit 211 may perform the process of step S12 as part of the matching process.

[0078] Thereafter, the matching unit 214c determines whether or not the photographed person matches a registered person based on the feature amount extracted in step S22c (step S23c). That is, the matching unit 214c performs the process of step S23c as part of the matching process.

[0079] The matching unit 214c may determine whether the captured person matches the registered person by comparing the feature amounts of the captured person extracted in step S22c with the feature amounts of the registered person. The matching unit 214c may determine that the captured person matches the registered person if the difference between the feature amounts of the captured person and the feature amounts of the registered person is smaller than a threshold. On the other hand, the matching unit 214c may determine that the captured person does not match the registered person if the difference between the feature amounts of the captured person and the feature amounts of the registered person is larger than the threshold.

[0080] The matching unit 214c may determine whether the captured person matches the registered person by comparing the facial feature amounts of the captured person with the facial feature amounts of the registered person. In this case, the feature extraction unit 211 may extract the facial feature amounts of the captured person in step S22c. The matching process of determining whether the captured person matches the registered person by comparing the facial feature amounts of the captured person with the facial feature amounts of the registered person may be substantially the same as the matching process performed in face authentication.

[0081] The matching unit 214c may determine whether the imaged person matches the registered person by comparing the iris feature amount of the imaged person with the iris feature amount of the registered person. In this case, the feature extraction unit 211 may extract the iris feature amount of the imaged person in step S22c. The matching process of determining whether the imaged person matches the registered person by comparing the iris feature amount of the imaged person with the iris feature amount of the registered person may be substantially the same as the matching process performed in iris authentication.

[0082] The matching unit 214c may determine whether the imaged person matches the registered person by comparing the fingerprint feature amounts of the imaged person with the fingerprint feature amounts of the registered person. In this case, the feature extraction unit 211 may extract the fingerprint feature amounts of the imaged person in step S22c. The matching process of determining whether the imaged person matches the registered person by comparing the fingerprint feature amounts of the imaged person with the fingerprint feature amounts of the registered person may be substantially the same as the matching process performed in fingerprint authentication.

[0083] The matching unit 214c may determine whether the imaged person matches the enrolled person by comparing feature amounts related to any biometric information of the imaged person (for example, at least one of a palm print and a vein pattern) with feature amounts related to any biometric information of the enrolled person. In this case, the feature extraction unit 211 may extract feature amounts related to the biometric information of the imaged person in step S22c. The matching process of determining whether the imaged person matches the enrolled person by comparing feature amounts related to the biometric information of the imaged person with feature amounts related to the biometric information of the enrolled person may be substantially the same as the matching process performed in biometric authentication.

[0084] An example of biometric information is information about a person's voice. However, it is not easy to extract features related to the voice of the person being imaged from the person image IMG. Therefore, when information about a person's voice is used as biometric information, the information processing system SYSc may be equipped with a detection device (e.g., a microphone) capable of detecting the voice of the person to be authenticated. In this case, the feature extraction unit 211 may extract features related to the voice of the person being imaged. Thereafter, the matching unit 214c may determine whether the person being imaged matches the registered person by comparing the features related to the voice of the person being imaged with the features related to the voice of the registered person. The matching process for determining whether the person being imaged matches the registered person by comparing the features related to the voice of the person being imaged with the features related to the voice of the registered person may be substantially the same as the matching process performed in voice authentication.

[0085] Information about the registered person may be stored in a verification DB 221c stored in the storage device 22. The verification DB 221c may store, for example, a person record including information about the feature amounts of the registered person. Generally, a plurality of registered people are registered in the information processing system SYSc. Therefore, the verification DB 221c may store a plurality of person records corresponding to the plurality of registered people, respectively. In this case, the process of determining whether the photographed person matches the registered person is equivalent to the process of determining whether a registered person matching the photographed person exists among the plurality of registered people.

[0086] If it is determined in step S23c that the photographed person does not match the registered person (step S23c: No), the person to be authenticated who is actually present in front of the camera 1 or the person displayed on the display photographed by the camera 1 is not registered as a registered person in the information processing system SYSc. In this case, the authentication unit 212 determines that authentication of the person to be authenticated has failed (step S15).

[0087] On the other hand, if the result of the determination in step S23c is that the imaged person matches the registered person (step S23c: Yes), the person to be authenticated who actually exists in front of the camera 1 or the person displayed on the display captured by the camera 1 is registered in the information processing system SYSc as a registered person. However, in the third embodiment, the authentication unit 212 does not determine that authentication of the person to be authenticated has been successful just because the matching condition that the imaged person matches the registered person is met. Specifically, the authentication unit 212 determines that authentication of the person to be authenticated has been successful when, in addition to the matching condition, both a question correctness condition that the person to be authenticated correctly answers a question posed to the person to be authenticated by the information processing system SYSc and a continuity condition that there is continuity in the movements of the imaged person are met. Therefore, in order to determine whether the question correctness condition is met, the information processing device 2c performs a question process, and in order to determine whether the continuity condition is met, the information processing device 2c performs the continuity determination process described above. That is, in the third embodiment, the information processing device 2c performs authentication processing including a matching process, a questioning process, and a continuity determination process.

[0088] Specifically, first, as part of the question processing, the question processor 215c uses the question DB 222c to generate a question to be posed to the person to be authenticated (step S31c). The question DB 222c stores question records including registered persons, questions whose correct answers are known to the registered persons (however, questions that are difficult for persons other than the registered persons to answer), and correct answers to the questions. Generally, a plurality of registered persons are registered in the information processing system SYSc. Therefore, the question DB 222c may store a plurality of question records corresponding to the plurality of registered persons, respectively. The question processor 215c extracts, from the question DB 222c, a question record corresponding to the registered person determined to match the imaged person in step S23c. The question processor 215c may set the question included in the extracted question record as the question to be posed to the person to be authenticated.

[0089] Thereafter, as part of the question processing, the question processing unit 215c outputs the generated question using the output device 25 so that the person to be authenticated who is in front of the camera 1 can recognize the generated question (step S31c). For example, as shown in Fig. 12, if the output device 25 is equipped with a display device, the question processing unit 215c may display the generated question using the output device 25 so that the person to be authenticated can visually recognize the generated question. As a result, the person to be authenticated answers the question posed by the information processing device 2c.

[0090] In the third embodiment, the authentication target person's answering method is preferably an answering method that allows the information processing device 2c to identify the content of the answer from the person image IMG. Specifically, the authentication target person's answering method is preferably an answering method that allows the information processing device 2c to identify the content of the answer from the movement of the person captured in the person image IMG. As an example, when the information processing device 2c asks the authentication target person a multiple-choice question in which the answer is either "yes" or "no," the authentication target person may nod their head to answer "yes" to the question, or may shake their head from side to side to answer "no" to the question. In this case, the information processing device 2c can identify whether the authentication target person's answer is "yes" or "no" from the movement of the person captured in the person image IMG. Therefore, the question generator 215c may generate a question that the authentication target person can answer using an answering method that allows the information processing device 2c to identify the content of the answer from the person image IMG. The question records stored in the question DB 222c may store questions that the person to be authenticated can answer using an answering method that allows the information processing device 2c to identify the content of the answer from the person image IMG. When outputting a question, the question processing unit 215c may also output (for example, display) the answering method using the output device 25 so that the person to be authenticated can recognize the answering method, as shown in Fig. 12 .

[0091] After the question generated by question processing unit 215c is output, camera 1 continuously captures images of the imaging target range in time series (step S32c). That is, camera 1 performs imaging processing (step S32c). Note that the processing in step S32c in Fig. 11 may be the same as the processing in step S11 in Fig. 3. Therefore, a detailed description of the processing in step S32c in Fig. 11 will be omitted.

[0092] In the third embodiment, in step S32c, the camera 1 successively captures images of a person answering a question in time series. As described above, if a person to be authenticated who is actually present in front of the camera 1 is present within the imaging range, there is a possibility that the person to be authenticated will answer the question. For this reason, the camera 1 may successively capture images of the person to be authenticated who is answering a question in time series, thereby generating a plurality of person images IMG in which the person to be authenticated appears as an imaged person. On the other hand, if a display displaying an image of a person (typically a registered person) different from the person to be authenticated who is actually present in front of the camera 1 is present within the imaging range, there is a possibility that the person displayed on the display will make a movement to answer the question. For this reason, the camera 1 may successively capture images of a person displayed on the display in time series, thereby generating a plurality of person images IMG in which the person displayed on the display appears as an imaged person. The camera 1 transmits the generated plurality of person images IMG to the information processing device 2.

[0093] Following or in parallel with the imaging process performed by the camera 1 in step S32c, the information processing device 2 performs a question process and a continuity determination process (steps S33c to S35c). For convenience of explanation, Fig. 3 shows an example in which the information processing device 2c performs a question process and a continuity determination process following the imaging process performed by the camera 1. However, the information processing device 2c may perform a question process and a continuity determination process in parallel with the imaging process performed by the camera 1.

[0094] To perform the question processing and the continuity determination processing, the feature extraction unit 211 extracts feature amounts of the captured person appearing in the person image IMG from each of the multiple person images IMG generated in step S32c (step S33c). In particular, the feature extraction unit 211 extracts feature amounts that can be used to identify an answer to a question from the captured person's movements and to determine whether there is continuity in the captured person's movements. Such feature amounts may be the same as the feature amounts extracted in the processing in step S12 of FIG. 3 described above. In other words, the feature extraction unit 211 may extract feature amounts of the captured person that change depending on the captured person's movements. In this case, the processing in step S33c of FIG. 11 may be the same as the processing in step S12 of FIG. 3. Therefore, a detailed description of the processing in step S33c of FIG. 11 will be omitted.

[0095] The feature extracted in step S33c is used to identify an answer to a question from the imaged person's movement and to determine whether there is continuity in the imaged person's movement. Therefore, the process of step S33c may constitute a part of each of the question processing and the continuity determination processing. In other words, the feature extraction unit 211 may perform the process of step S33c as a part of the question processing and the continuity determination processing.

[0096] Thereafter, the question processing unit 215c determines whether the imaged person's answer to the question is correct based on the feature amount extracted in step S33c (step S34c). That is, the feature extraction unit 211 performs the process of step S34c as part of the question processing. For example, the question processing unit 215c identifies the movement of the imaged person based on the feature amount extracted in step S33c. Thereafter, the question processing unit 215c identifies the imaged person's answer (that is, the answer of the person to be authenticated who is actually present in front of the camera 1 or the person displayed on the display) based on the imaged person's movement. Thereafter, the question processing unit 215c determines whether the imaged person's answer to the question is correct by determining whether the identified imaged person's answer matches the correct answer included in the question record stored in the question DB 222c.

[0097] Before or after or in parallel with the processing of step S34c, the authentication unit 212 determines whether there is continuity in the movement of the captured person among the multiple person images IMG based on the feature amount extracted in step S33c (step S35c). That is, the feature extraction unit 211 performs the processing of step S35c as part of the continuity determination processing. Note that the processing of step S35c in FIG. 11 may be the same as the processing of step S13 in FIG. 3. Therefore, a detailed description of the processing of step S35c in FIG. 11 will be omitted. Also, for convenience of explanation, FIG. 11 shows an example in which the information processing device 2c performs the question processing of step S34c followed by the continuity determination processing of step S35c. However, the information processing device 2c may perform the question processing of step S34c and the continuity determination processing of step S35c in parallel. The information processing device 2c may perform the question processing of step S34c followed by the continuity determination processing of step S35c.

[0098] If the result of the determination in step S34c is that the imaged person's answer to the question is not correct (step S34c: No), this means that the imaged person (i.e., the person to be authenticated who is actually in front of camera 1 or the person displayed on the display) has incorrectly answered a question that a registered person would naturally have answered correctly. In this case, the person who answered the question (i.e., the person to be authenticated who is actually in front of camera 1 or the person displayed on the display) may not be a registered person. Therefore, in this case, the authentication unit 212 determines that the person to be authenticated may be impersonating a person different from the person to be authenticated (typically, a registered person). As a result, the authentication unit 212 determines that authentication of the person to be authenticated has failed (step S15).

[0099] Furthermore, if the result of the determination in step S35c is that there is no continuity in the movement of the person to be photographed (step S35c: No), the authentication unit 212 determines that authentication of the person to be authenticated has failed (step S15), as described in the first embodiment.

[0100] On the other hand, if the results of the determinations in steps S34c and S35c indicate that the imaged person's answer to the question is correct and that there is continuity in the imaged person's movements (step S34c: Yes and step S35c: Yes), the authentication unit 212 determines that there is no possibility that the person to be authenticated is masquerading as a different person from the person to be authenticated. As a result, the authentication unit 212 determines that the authentication of the person to be authenticated has been successful (step S14).

[0101] As described above, the information processing system SYSc (particularly, the information processing device 2c) of the third embodiment can achieve the same effects as those achieved by the information processing system SYSa (particularly, the information processing device 2) of the first embodiment or the information processing system SYSb (particularly, the information processing device 2b) of the second embodiment. In particular, the information processing device 2c determines that authentication of the person to be authenticated has been successful only when both the continuity condition that there is continuity in the movements of the person to be photographed, which was used in the first embodiment, are met, as well as the matching condition that the person to be photographed matches a registered person and the question correctness condition that the person to be authenticated correctly answers the question. Therefore, the information processing device 2c can more strictly authenticate the person to be authenticated under stricter conditions than those in the first embodiment.

[0102] However, when the continuity condition and the question correctness condition are satisfied, the information processing device 2c may determine that the authentication of the person to be authenticated is successful even when the matching condition is not satisfied. In this case, the information processing device 2c does not need to perform the matching process. The information processing device 2c does not need to include the matching unit 214c for performing the matching process.

[0103] Furthermore, when the continuity condition and the matching condition are satisfied, the information processing device 2c may determine that the authentication of the person to be authenticated has been successful even if the question correctness condition is not satisfied. In this case, the information processing device 2c may not need to perform question processing. The information processing device 2c may not need to include the question processing unit 215c for performing question processing. You don't have to.

[0104] Instead of the question correct answer condition, the information processing device 2c may use any correct answer condition that the person to be authenticated inputs information known only to the registered person, and the information input by the person to be authenticated is correct. For example, the information processing device 2c may use a correct answer condition that the person to be authenticated inputs a correct password or PIN (Personal Identification Number) code.

[0105] (4) Variations The camera 1 may include a global shutter imaging element. A global shutter imaging element is an imaging element in which the timing of exposure and the timing of charge readout are the same for all pixels of the imaging element. Alternatively, the camera 1 may include a rolling shutter imaging element. A rolling shutter imaging element is an imaging element in which exposure and charge readout are performed row by row, starting from the top row of pixels and progressing to the bottom row of pixels. As a result, a person image IMG generated by a camera 1 equipped with a rolling shutter imaging element will suffer from distortion (typically a horizontal shift) due to differences in exposure and readout times. This distortion may be particularly pronounced in a person image IMG that captures a moving person. In this case, the feature values ​​of the captured person, which should normally change continuously, will change discontinuously due to the distortion, and the information processing device 2b may erroneously determine that impersonation is occurring even when no impersonation is occurring. Conversely, the feature values ​​of the photographed person, which should change discontinuously, may change continuously due to distortion, which may lead the information processing device 2b to erroneously determine that impersonation is not occurring even when impersonation is occurring. Therefore, if the camera 1 is equipped with a rolling shutter image sensor, the information processing device 2b may perform continuity determination processing to eliminate the influence of this distortion. For example, the information processing device 2b may perform continuity determination processing to eliminate the influence of distortion based on the positional relationship between pixels corresponding to a certain part of the photographed person's body from which the feature values ​​were extracted and the row in which exposure and charge readout are performed.

[0106] (4) Supplementary notes The following additional notes are provided regarding the above-described embodiment. [Appendix 1] an imaging device capable of generating a plurality of images in which an object is captured by continuously capturing images of the object in time series; an information processing device that authenticates the object using at least one of the plurality of images; Equipped with The information processing device includes: extraction means for extracting, from each of the plurality of images, a feature amount of the object that changes in accordance with the movement of the object; an authentication means for determining that authentication of the target has failed when the feature amount changes discontinuously over time among the plurality of target images; An information processing system comprising: [Appendix 2] The information processing device further includes a change unit capable of changing an image capture rate indicating the number of times the image capture device captures an image of the target per unit time. 10. The information processing system of claim 1. [Appendix 3] the changing means changes the imaging rate from a first imaging rate to a second imaging rate different from the first imaging rate during a period in which the imaging device images the same object; the imaging device continuously images the object in time series at the first imaging rate, and then continuously images the same object in time series at the second imaging rate; The authentication means determines that authentication of the object has failed when at least one of a first condition that the feature amount varies discontinuously in time series among the plurality of images generated by successively capturing images of the object at the first imaging rate and a second condition that the feature amount varies discontinuously in time series among the plurality of images generated by successively capturing images of the object at the second imaging rate is satisfied. 10. The information processing system of claim 2. [Appendix 4] The imaging device captures images of the object moving toward the imaging device in a time-series manner. 4. An information processing system according to any one of claims 1 to 3. [Appendix 5] an extraction means for extracting a feature amount of an object that changes in accordance with the movement of the object from each of a plurality of images in which the object appears, the images being generated by an imaging device successively capturing images of the object in time series; an authentication means for determining that authentication of the target has failed when the feature amount changes discontinuously over time among the plurality of images; An information processing device comprising: [Appendix 6] generating a plurality of images in which the object is captured by capturing images of the object in a time series sequence; extracting, from each of the plurality of images, a feature of the object that changes in accordance with the movement of the object; If the feature amount changes discontinuously over time among the plurality of target images, it is determined that authentication of the target has failed. Information processing methods. [Appendix 7] On the computer, generating a plurality of images in which the object is captured by capturing images of the object in a time series sequence; extracting, from each of the plurality of images, a feature of the object that changes in accordance with the movement of the object; If the feature amount changes discontinuously over time among the plurality of images, it is determined that authentication of the target has failed. A recording medium on which a computer program for executing an information processing method is recorded. [Appendix 8] On the computer, generating a plurality of images in which the object is captured by capturing images of the object in a time series sequence; extracting, from each of the plurality of images, a feature of the object that changes in accordance with the movement of the object; If the feature amount changes discontinuously over time among the plurality of images, it is determined that authentication of the target has failed. A computer program that executes an information processing method.

[0107] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​this disclosure that can be read from the claims and the entire specification, and information processing systems, information processing devices, information processing methods, computer programs, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0108] To the extent permitted by law, this application claims priority to Japanese Patent Application No. 2020-218564, filed December 28, 2020, the disclosure of which is incorporated herein in its entirety. Furthermore, to the extent permitted by law, all publications and papers mentioned in this specification are incorporated herein by reference. [Explanation of symbols]

[0109] SYS Information Processing System 1 camera 2. Information processing equipment 21 Arithmetic unit 211 Feature Extraction Unit 212 Authentication Department 213b Rate change section 214c Collation section 215c Question Processing Section IMG Portrait

Claims

1. an extraction means for extracting a feature amount of an object that changes in accordance with the movement of the object from each of a plurality of images in which the object appears, the images being generated by an imaging device continuously capturing images of the object; a question processing means for asking a question to the subject and determining whether the answer given by the subject in response to the question is correct; an authentication means for performing authentication processing based on matching of the subject using at least one of the plurality of images, the result of the determination of correctness or incorrectness, and whether or not changes in the feature amount when the subject answers are continuous; An information processing device comprising:

2. The authentication means determines that the authentication process is successful if the matching is successful, the determination result is positive, and there is continuity in the change in the feature amount. The information processing device according to claim 1 .

3. The question processing means identifies the answer based on the change in the feature amount. The information processing device according to claim 1 .

4. 3. The information processing apparatus according to claim 1, further comprising a change unit capable of changing an image capture rate indicating the number of times the image capture device captures an image of the target per unit time.

5. the change means changes the imaging rate from a first imaging rate to a second imaging rate different from the first imaging rate during a period in which the imaging device images the same object; the imaging device continuously images the object in time series at the first imaging rate, and then continuously images the same object in time series at the second imaging rate; The authentication means determines that authentication of the object has failed when at least one of a first condition that the feature amount varies discontinuously in time series among the plurality of images generated by successively capturing images of the object at the first imaging rate and a second condition that the feature amount varies discontinuously in time series among the plurality of images generated by successively capturing images of the object at the second imaging rate is satisfied. The information processing device according to claim 4 .

6. By continuously capturing images of an object, a plurality of images in which the object is captured are generated; extracting, from each of the plurality of images, a feature of the object that changes in accordance with the movement of the object; Asking a question to the subject and determining whether the subject's answer to the question is correct or not; The authentication process is performed based on the matching of the subject using at least one of the plurality of images, the result of the determination of correctness, and whether the change in the feature amount when the subject answers is continuous. Information processing methods.

7. On the computer, By continuously capturing images of an object, a plurality of images in which the object is captured are generated; extracting, from each of the plurality of images, a feature of the object that changes in accordance with the movement of the object; Asking a question to the subject and determining whether the subject's answer to the question is correct or not; The authentication process is performed based on the matching of the subject using at least one of the plurality of images, the result of the determination of correctness, and whether the change in the feature amount when the subject answers is continuous. A computer program that executes an information processing method.

Images