Information processing device, information processing method, and recording medium

Abstract

To provide an information processing apparatus, an information processing method and a recording medium capable of determining whether or not an object person reflected in a figure image is disguising himself / herself as another person.SOLUTION: In an information processing system including a camera for taking an image of an object person (figure) to be output to an information processing apparatus and the information processing apparatus, a computing device of the information processing apparatus includes: an image acquiring section that acquires a figure image (IMG) including an eye region (EA) of the object person; and a disguise determining section that, if there exists an aperture (AP) surrounding at least a part of an eye of the object person in the figure image, determines that the object person is disguising himself / herself as another person.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to, for example, the technical field of an information processing device, an information processing method, and a recording medium capable of determining whether or not a subject person appearing in a person image is masquerading as another person. [Background technology]

[0002] An example of an information processing device capable of determining whether a subject appearing in a person image is masquerading as another person is described in Patent Document 1. Patent Document 1 describes an information processing device capable of determining whether a face appearing in an image captured by a camera is a real person's face or a fake face.

[0003] Other prior art documents related to this disclosure include Patent Documents 2 to 5. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-184331 A [Patent Document 2] Japanese Patent Application Publication No. 06-3094578 [Patent Document 3] JP 2004-178402 A [Patent Document 4] JP 2006-330936 A [Patent Document 5] JP 2010-286959 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents. [Means for solving the problem]

[0006] One aspect of the information processing device includes an acquisition means for acquiring a portrait image in which a subject is captured, and a determination means for determining that the subject is impersonating another person if an opening surrounding at least a portion of the subject's eye included in the eye region is present in the portrait image.

[0007] One aspect of the information processing method involves obtaining a person image in which a subject is captured, and if there is an opening in the person image that surrounds at least a portion of the subject's eye contained in the eye region, determining that the subject is impersonating another person.

[0008] One aspect of the recording medium is a recording medium having recorded thereon a computer program that causes a computer to execute an information processing method that acquires a person image in which a subject is captured, and determines that the subject is impersonating another person if an opening is present in the person image that surrounds at least a portion of the subject's eye that is included in the eye region. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of an information processing system according to this embodiment. [Diagram 2] FIG. 2 shows an example of a mask that a subject can wear to impersonate another person. [Diagram 3] FIG. 3 is a block diagram showing the configuration of the information processing device of this embodiment. [Figure 4] FIG. 4 is a flowchart showing the flow of the spoofing determination operation performed by the information processing device of this embodiment. [Diagram 5] FIG. 5 is a plan view showing an example of a person image in which a subject wearing a mask with openings formed therein is captured, together with an enlarged view of a portion of the person image that includes the subject's eyes. [Figure 6]FIG. 6 is a plan view showing an example of a person image in which a subject is captured who is not wearing a mask with an opening formed therein, together with an enlarged view of a portion of the person image that includes the subject's eyes. [Figure 7] FIG. 7 is a plan view showing an example of a person image in which a subject is wearing a mask with an opening and has his / her eyes closed, together with an enlarged view of a portion of the person image that includes the subject's eyes. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an information processing device, an information processing method, and a recording medium will be described. Hereinafter, an information processing device, an information processing method, and a recording medium will be described using an information processing system SYS to which the information processing device, the information processing method, and the recording medium are applied.

[0011] (1) Configuration of information processing system SYS First, the configuration of the information processing system SYS of this embodiment will be described.

[0012] (1-1) Overall configuration of information processing system SYS First, the overall configuration of the information processing system SYS of this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the overall configuration of the information processing system SYS of this embodiment.

[0013] As shown in Fig. 1, the information processing system SYS includes a camera 1 and an information processing device 2. The information processing system SYS may include a single camera 1, or may include a plurality of cameras 1. The camera 1 and the information processing device 2 are capable of communicating with each other via a communication network 3. The communication network 3 may include a wired communication network. The communication network 3 may include a wireless communication network.

[0014] The camera 1 is an imaging device capable of imaging a subject (i.e., a person) located within the imaging target range of the camera 1. The camera 1 images the subject, thereby generating a person image IMG in which the subject imaged by the camera 1 is captured. In particular, the camera 1 images the face of the subject (particularly, at least a part of the face including the eyes), thereby generating a person image IMG in which the person's face (particularly, at least a part of the face including the eyes) is captured. That is, the camera 1 generates a person image IMG including the eye area EA of the subject. The eye area EA of the subject may mean, for example, an area of ​​a predetermined shape (e.g., a rectangular shape) including the subject's eyes. The eye area EA of the subject may mean, for example, an area of ​​a predetermined shape located within a certain distance from the subject's eyes. As an example, as shown in FIG. 5 and FIG. 6 to be generated later, the eye area EA of the subject may mean an area included in the eye image IMG_EYE. Note that the person image IMG may or may not include areas other than the eye area of ​​the subject as long as it includes the eye area of ​​the subject. Therefore, in this embodiment, the "portrait image IMG in which the subject is captured" may mean a portrait image IMG in which at least the eye region of the subject is captured, or a portrait image IMG in which a part of the subject including the eye region (e.g., the face) is captured, or a portrait image IMG in which the entire body of the subject is captured. In any case, an image including at least the eye region of the subject may be used as the portrait image IMG. The camera 1 outputs the generated portrait image IMG to the information processing device 2. Specifically, the camera 1 transmits the generated portrait image IMG to the information processing device 2 via the communication network 3.

[0015] The information processing device 2 is capable of performing an impersonation determination operation. Specifically, the information processing device 2 is capable of receiving a person image IMG transmitted from the camera 1 via the communication network 3. Furthermore, based on the received person image IMG, the information processing device 2 is capable of determining whether or not a subject captured in the person image IMG (i.e., a subject in front of the camera 1) is attempting to impersonate another person different from the subject.

[0016] In this embodiment, the information processing device 2 can determine whether or not the subject is trying to impersonate another person by wearing (for example, putting on) a mask M imitating another person. An example of the mask M that is the target of the impersonation determination operation is shown in FIG. 2. As shown in FIG. 2, in this embodiment, the information processing device 2 can determine whether or not the subject is trying to impersonate another person by wearing a mask M in which an opening AP is formed in a portion corresponding to the eyes of the other person. Note that FIG. 2 shows an example in which the opening AP is an opening having a horizontally elongated circular shape. However, the shape of the opening AP is not limited to the shape shown in FIG. 2. For example, the shape of the opening AP may be any circular shape, any elliptical shape, any polygonal shape, or a shape corresponding to any closed curve.

[0017] In this embodiment, the "mask" may refer to an object (component) accessory that can be worn by the subject and can cover (in other words, hide) at least a part of the subject's face when worn by the subject. For example, a mask that completely covers the subject's head may be used as the mask M. For example, a mask (so-called a face) that partially covers the subject's head (for example, covers the front of the subject's face) may be used as the mask M.

[0018] Here, as an example of a method for determining whether a subject in front of the camera 1 is disguised as another person, there is a method for determining whether the subject is disguised as another person by determining whether the captured person in the person image IMG blinks according to a predetermined instruction. The information processing device of the comparative example using this method can appropriately determine whether the subject is disguised as another person mainly when the subject causes the camera 1 to capture a photo in which another person is captured or a display displaying another person in order to disguise himself / herself as another person. This is because it is difficult for another person captured in a photo or displayed on a display to blink according to a predetermined instruction. For this reason, the information processing device of the comparative example typically determines that the captured person captured in the person image IMG is a person different from the subject in front of the camera 1 (as a result, the subject is disguised as another person) when the captured person captured in the person image IMG does not blink according to a predetermined instruction. However, when the subject wears a mask M with an opening AP formed in a portion corresponding to the eyes of the other person, the eyes of the subject wearing the mask M will be exposed from the opening AP. Therefore, if a subject wearing the mask M blinks, the information processing device of the comparative example will determine that the subject in the person image IMG is blinking in accordance with a predetermined instruction. As a result, the information processing device of the comparative example has a technical problem in that it erroneously determines that the subject is not impersonating another person, even though the subject is actually impersonating another person.

[0019] Therefore, in this embodiment, an information processing device 2 capable of solving the above-mentioned technical problems will be described. That is, in this embodiment, an information processing device 2 capable of appropriately determining that a subject is impersonating another person in a situation where the subject is impersonating another person by wearing a mask M having an opening AP formed in a portion corresponding to the eyes of the other person will be described.

[0020] (1-2) Configuration of information processing device 2 Next, the configuration of the information processing device 2 of this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2 of this embodiment.

[0021] 3, the information processing device 2 includes a calculation device 21, a storage device 22, and a communication device 23. Furthermore, the information processing device 2 may include an input device 24 and an output device 25. However, the information processing device 2 does not have to include at least one of the input device 24 and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0022] The arithmetic device 21 includes, for example, at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a field programmable gate array (FPGA). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable and non-transient recording medium using a recording medium reading device (not shown) included in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) a computer program from a device (not shown) disposed outside the information processing device 2 via the communication device 23 (or other communication device). The arithmetic device 21 executes the read computer program. As a result, a logical function block for executing an operation (for example, the above-mentioned spoofing determination operation) to be performed by the information processing device 2 is realized in the arithmetic device 21. That is, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processes) that the information processing device 2 should perform.

[0023] 3 shows an example of logical functional blocks realized in the arithmetic device 21 to execute the spoofing judgment operation. As shown in FIG. 3, an image acquisition unit 211, which is a specific example of "acquisition means", and an spoofing judgment unit 212, which is a specific example of "judgment means", are realized in the arithmetic device 21. The image acquisition unit 211 can acquire a person image IMG from the camera 1. The spoofing judgment unit 212 can judge, based on the person image IMG acquired by the image acquisition unit 211, whether or not a subject captured in the person image IMG (i.e., a subject in front of the camera 1) is trying to pose as a different person from the subject.

[0024] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0025] The communication device 23 is capable of communicating with the camera 1 via the communication network 3. In this embodiment, the communication device 23 receives (that is, acquires) a person image IMG from the camera 1 via the communication network 3.

[0026] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0027] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called a display) capable of displaying an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (so-called a speaker) capable of outputting sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (so-called a printer) capable of printing desired information on paper.

[0028] (2) Spoofing determination operation performed by the information processing device 2 Next, the spoofing determination operation performed by the information processing device 2 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of the spoofing determination operation performed by the information processing device 2.

[0029] As shown in FIG. 4, the image acquisition unit 211 acquires a person image IMG from the camera 1 using the communication device 23 (step S11).

[0030] Thereafter, based on the person image IMG acquired in step S11, the impersonation determination unit 212 determines whether the subject captured in the person image IMG (i.e., the subject in front of the camera 1) is attempting to impersonate a person different from the subject (steps S12 to S14).

[0031] Here, Fig. 5 shows an example of a person image IMG in which a subject wearing a mask M with an opening AP is captured, together with an enlarged view of a portion of the person image IMG including the subject's eye area EA (hereinafter referred to as "eye image IMG_EYE"). As shown in Fig. 5, when a subject wears a mask M with an opening AP, the person image IMG captures the subject's eyes, at least partially exposed through the opening AP, along with a mask M that resembles a person other than the subject. That is, in the person image IMG, at least a portion of the subject's eyes are exposed through the opening AP.

[0032] In other words, when the subject wears a mask M with an opening AP formed therein, the subject's eyes are reflected in the person image IMG with at least a portion of the subject's eyes surrounded by the opening AP. That is, in the person image IMG, at least a portion of the subject's eyes is surrounded by the opening AP. Note that the state in which "at least a portion of the subject's eyes is surrounded by the opening AP" may mean, for example, a state in which "at least a portion of the subject's eyes is surrounded by the outer edge AP_edge of the opening AP (i.e., the outer edge of the mask M that forms the opening AP)."

[0033] 5 shows an example in which the entire eye of the subject is exposed from the aperture AP (i.e., the entire eye of the subject is surrounded by the aperture AP). The state in which "the entire eye of the subject is exposed from the aperture AP" may mean, for example, a state in which "the entire lower edge ul_edge of the upper eyelid and the entire upper edge ul_edge of the lower eyelid of the subject are exposed from the aperture AP". The state in which "the entire eye of the subject is surrounded by the aperture AP" may mean, for example, a state in which "the entire lower edge ul_edge of the upper eyelid and the entire upper edge ul_edge of the lower eyelid of the subject are surrounded by the aperture AP".

[0034] However, a part of the subject's eye may not be exposed from the opening AP. That is, a part of the subject's eye may not be surrounded by the opening AP. For example, when the horizontal size of the subject's eye (e.g., the length from the outer corner to the inner corner of the eye) is larger than the horizontal size of the opening AP, a part of the subject's eye is covered by the mask M without being exposed from the opening AP. Similarly, when the vertical size of the subject's eye (e.g., the distance between the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid) is larger than the vertical size of the opening AP, a part of the subject's eye is covered by the mask M without being exposed from the opening AP. Therefore, in these cases, while a part of the subject's eye is exposed from the opening AP, another part of the subject's eye may not be exposed from the opening AP. That is, while a part of the subject's eye is exposed from the opening AP, another part of the subject's eye may be covered by the mask M. In other words, while a part of the subject's eye is surrounded by the opening AP, another part of the subject's eye may not be surrounded by the opening AP. In any case, when a subject is wearing a mask M having an aperture AP formed therein, at least a portion (i.e., all or part) of the subject's eye is exposed through the aperture AP (i.e., surrounded by the aperture AP).

[0035] Considering such a state in which the subject's eye is surrounded by the aperture AP, the "eye" in this embodiment may mean a part of the subject including not only the eyeball but also the appendages attached to the eyeball (particularly, the appendages visible from the outside, for example, the eyelids including the upper eyelid and the lower eyelid described above). Therefore, the state in this embodiment in which "at least a part of the subject's eye is surrounded by the aperture AP" may include a state in which at least a part of the subject's eyeball is surrounded by the aperture AP, or a state in which at least a part of the appendages attached to the subject's eyeball are surrounded by the aperture AP. However, the "eye" in this embodiment may mean a part of the subject that includes the eyeball but does not include the appendages attached to the eyeball.

[0036] On the other hand, Fig. 6 shows an example of a person image IMG in which a subject not wearing a mask M with an opening AP is captured, together with an enlarged view of an eye image IMG_EYE including the subject's eye area EA in the person image IMG. As shown in Fig. 6, when the subject is not wearing a mask M with an opening AP, at least a part of the subject's eye is not captured in the person image IMG so as to be exposed from the opening AP. In other words, at least a part of the subject's eye is not surrounded by the opening AP in the person image IMG (for example, is not surrounded by the outer edge AP_edge of the opening AP).

[0037] In this embodiment, the masquerade determination unit 212 determines whether or not the subject is trying to masquerade as another person based on the difference between a person image IMG in which the subject wearing such a mask M appears and a person image IMG in which the subject not wearing the mask M appears. Specifically, the masquerade determination unit 212 determines whether or not at least a part of the subject's eyes are surrounded by an aperture AP (for example, whether or not they are surrounded by an outer edge AP_edge of the aperture AP) in the person image IMG acquired in step S11 (step S12). That is, the masquerade determination unit 212 determines whether or not an aperture AP surrounding at least a part of the subject's eyes exists in the person image IMG acquired in step S11 (for example, whether or not an outer edge AP_edge of the aperture AP surrounding at least a part of the subject's eyes exists) (step S12). In other words, the impersonation determination unit 212 determines whether or not an object (in this embodiment, a mask M) having an opening AP surrounding at least a portion of the subject's eye is present in the person image IMG acquired in step S11 (step S12).

[0038] As described above, when at least a part of the subject's eye is surrounded by the opening AP, at least a part of the subject's eye is exposed from the opening AP. Therefore, the operation of determining whether or not at least a part of the subject's eye is surrounded by the opening AP in step S12 may be considered to be substantially equivalent to the operation of determining whether or not at least a part of the subject's eye is exposed from the opening AP. The operation of determining whether or not at least a part of the subject's eye is surrounded by the opening AP in step S12 may be considered to be substantially equivalent to the operation of determining whether or not there is an opening AP through which at least a part of the subject's eye is exposed. The operation of determining whether or not at least a part of the subject's eye is surrounded by the opening AP in step S12 may be considered to be substantially equivalent to the operation of determining whether or not there is an object (in this embodiment, a mask M) in which an opening AP through which at least a part of the subject's eye is exposed is formed.

[0039] The masquerade determination unit 212 may determine whether or not at least a part of the subject's eye is surrounded by the aperture AP by performing an image analysis process for analyzing at least a part of the person image IMG (for example, the above-mentioned eye image IMG_EYE). For example, the masquerade determination unit 212 may detect the subject's eye from the person image IMG and determine whether or not the aperture AP surrounding the subject's eye is present around the subject's eye. In this case, the masquerade determination unit 212 may determine whether or not the aperture AP is present in the person image IMG by performing template matching using a template corresponding to the aperture AP.

[0040] The masquerade determination unit 212 may perform edge extraction processing on at least a part of the person image IMG and determine whether or not an edge surrounding the subject's eye (that is, the contour of an arbitrary object surrounding the subject's eye, and here, the contour of the mask M forming the outer edge AP_edge of the aperture AP) exists, thereby determining whether or not at least a part of the subject's eye is surrounded by the aperture AP. Specifically, the masquerade determination unit 212 may perform edge extraction processing on at least a part of the person image IMG (for example, the above-mentioned eye image IMG_EYE) to generate an edge image representing an edge in at least a part of the person image IMG. For example, the masquerade determination unit 212 may generate an edge image by performing filtering processing on the person image IMG using a first-order differential filter. The masquerade determination unit 212 may perform filtering processing using at least one of a vertical first-order differential filter and a horizontal first-order differential filter. An edge corresponding to the subject's eye usually appears in the edge image. The edge corresponding to the subject's eyes may include at least one of an edge corresponding to the lower edge ul_edge of the upper eyelid and an edge corresponding to the upper edge ul_edge of the lower eyelid. Furthermore, when the subject wears a mask M, an edge corresponding to the opening AP appears in the edge image. The edge corresponding to the opening AP may include an edge corresponding to the outer edge AP_edge of the opening AP (that is, an edge of a part of the mask M that forms the outer edge AP_edge of the opening AP, typically an edge that forms a closed loop). For this reason, the masquerade determination unit 212 may determine whether or not an edge corresponding to the opening AP surrounding the edge corresponding to the subject's eyes exists in the edge image. When an edge corresponding to the opening AP surrounding the edge corresponding to the subject's eyes exists, the masquerade determination unit 212 may determine that at least a part of the subject's eyes is surrounded by the opening AP.

[0041] The spoofing determination unit 212 may determine whether or not at least a part of the subject's eye is surrounded by the aperture AP using a computation model that can be trained. For example, the spoofing determination unit 212 may determine whether or not at least a part of the subject's eye is surrounded by the aperture AP using a computation model that outputs a determination result as to whether or not at least a part of the subject's eye is surrounded by the aperture AP when at least a part of the person image IMG is input. For example, the spoofing determination unit 212 may determine whether or not at least a part of the subject's eye is surrounded by the aperture AP using a computation model that outputs a determination result as to whether or not at least a part of the subject's eye is surrounded by the aperture AP when an edge image generated by performing an edge extraction process on at least a part of the person image IMG is input.

[0042] An example of a trainable computation model is a computation model using a neural network. The spoofing determination unit 212 may determine whether or not at least a part of the target person's eye is surrounded by the aperture AP by using a computation model trained by machine learning based on at least one of supervised learning and unsupervised learning.

[0043] If it is determined in step S12 that at least a part of the target person's eye is surrounded by the aperture AP (step S12: Yes), it is assumed that the target person appearing in the person image IMG is likely wearing a mask M in which the aperture AP is formed. In this case, the masquerade determination unit 212 determines that the target person is masquerading as another person (step S13).

[0044] On the other hand, if it is determined in step S12 that at least a part of the target person's eye is not surrounded by the aperture AP (step S12: No), it is assumed that the target person appearing in the person image IMG is unlikely to be wearing a mask M with an aperture AP formed therein. In this case, the masquerade determination unit 212 determines that the target person is not masquerading as another person (step S14).

[0045] When the masquerade determination unit 212 determines that the subject is not impersonating another person, the information processing device 2 may authenticate the subject using the person image IMG. For example, when the face of the subject is reflected in the person image IMG, the information processing device 2 may authenticate the subject by performing face authentication using the face of the subject reflected in the person image IMG. Furthermore, since the person image IMG includes the eye area EA of the subject, the information processing device 2 may authenticate the subject by performing iris authentication using the iris of the subject reflected in the person image IMG. Furthermore, the information processing device 2 may authenticate the subject by performing any authentication different from the face authentication and iris authentication. Furthermore, the information processing device 2 may authenticate the subject by performing any authentication different from the authentication using the person image IMG.

[0046] (3) Technical Effects of Information Processing Device 2 As described above, the information processing device 2 determines that the subject is impersonating another person when at least a part of the subject's eyes is surrounded by the aperture AP in the person image IMG (i.e., there is an aperture AP surrounding at least a part of the subject's eyes). Therefore, the information processing device 2 can appropriately determine that the subject is impersonating another person in a situation where the subject is impersonating another person by wearing a mask M in which an aperture AP is formed in a portion corresponding to the eyes of the other person.

[0047] As described above, the information processing device 2 may use an edge image generated by performing edge extraction processing on at least a part of the person image IMG in order to determine whether or not at least a part of the target's eyes are surrounded by the aperture AP in the person image IMG. That is, the information processing device 2 may use an edge image in which the contours of objects (e.g., a mask M in which the target's eyes and the aperture AP are formed) present in the person image IMG are emphasized to determine whether or not at least a part of the target's eyes are surrounded by the aperture AP. In this case, the information processing device 2 can more accurately determine whether or not at least a part of the target's eyes are surrounded by the aperture AP compared to a case in which an edge image is not used. As a result, the information processing device 2 can more accurately determine whether or not the target is impersonating another person.

[0048] (4) Variations (4-1) First Modification When the subject wears a mask M, as shown in Fig. 5, at least a part of the lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the subject's lower eyelid are located away from the outer edge AP_edge of the opening AP in the person image IMG. This is because the size of the opening AP formed in the mask M is usually unlikely to match the size of the subject's eyes wearing the mask M. In this way, when at least a part of the lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the subject's lower eyelid are located away from the outer edge AP_edge of the opening AP in the person image IMG, the masquerade determination unit 212 is likely to determine that at least a part of the subject's eyes are surrounded by the opening AP in the person image IMG.

[0049] On the other hand, the possibility that the size of the opening AP formed in the mask M and the size of the eyes of the subject wearing the mask M match cannot be said to be zero. When the size of the opening AP formed in the mask M and the size of the eyes of the subject wearing the mask M match, the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid of the subject may overlap the outer edge AP_edge of the opening AP in the person image IMG. In this case, the impersonation determination unit 212 may determine that at least a part of the eyes of the subject is not surrounded by the opening AP in the person image IMG. This is because the outer edge AP_edge of the opening AP, which is overlapped by the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid, may be erroneously recognized as the outline of the eyes (i.e., the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid). As a result, the accuracy of determining whether the subject is impersonating another person may decrease.

[0050] Therefore, in order to accurately determine whether or not the subject is impersonating another person even when the subject is wearing a mask M in which an opening AP having a size that matches the size of the subject's eyes is formed, the image acquisition unit 211 may acquire a person image IMG in which the subject is photographed with his / her eyes closed in step S11 of FIG. 4. An example of a person image IMG in which a subject is photographed wearing a mask M in which an opening AP is formed and with his / her eyes closed is shown in FIG. 7. As shown in FIG. 7, when the subject has his / her eyes closed, even if the size of the opening AP formed in the mask M matches the size of the subject's eyes wearing the mask M, the lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the subject's lower eyelid are unlikely to overlap with the outer edge AP_edge of the opening AP in the person image IMG. Therefore, the impersonation determination unit 212 can determine that at least a part of the subject's eyes are surrounded by the opening AP in the person image IMG in a situation in which the subject is wearing a mask M in which an opening AP having a size that matches the size of the subject's eyes is formed. As a result, the impersonation determination unit 212 can accurately determine whether the subject is impersonating another person even when the subject is wearing a mask M having an opening AP formed therein whose size matches the size of the subject's eyes.

[0051] In addition, the image acquisition unit 211 may acquire a person image IMG in which the subject has his / her eyes closed in step S11 of FIG. 4, not only in a situation where the subject is wearing a mask M having an opening AP formed therein having a size matching the size of the subject's eyes.

[0052] In order to obtain a person image IMG in which a subject with his / her eyes closed appears, the information processing device 2 may instruct the subject to close his / her eyes. For example, if the information processing device 2 has a display near the camera 1, the information processing device 2 may display a GUI (Graphical User Interface) for instructing the subject to close his / her eyes.

[0053] Alternatively, the camera 1 may capture the subject continuously over time. That is, the camera 1 may generate a plurality of person images IMG as time-series data. In this case, the masquerade determination unit 212 may extract at least one person image IMG in which the subject with his / her eyes closed is captured from the plurality of person images IMG, and determine whether or not at least a part of the subject's eye is surrounded by the aperture AP in the extracted at least one person image IMG.

[0054] In addition, the information processing device 2 may perform an impersonation determination operation in parallel with the impersonation determination operation shown in FIG. 4, in which a subject reflected in a person image IMG is blinking or not, to determine whether the subject is impersonating another person, in the same manner as the information processing device of the comparative example described above. In this case, in order to determine whether the subject is blinking or not, the camera 1 usually takes pictures of the subject continuously in time. That is, the camera 1 generates a plurality of person images IMG as time-series data. In this case, the impersonation determination unit 212 may also extract at least one person image IMG in which a subject with closed eyes is reflected from the plurality of person images IMG, and determine whether at least a part of the subject's eyes is surrounded by an opening AP in the at least one extracted person image IMG. As a result, the information processing device 2 can perform the impersonation determination operation shown in FIG. 4 and the impersonation determination operation using blinking in parallel.

[0055] (4-2) Second Modification The image acquisition unit 211 may acquire both a person image IMG in which a subject with eyes closed appears and a person image IMG in which a subject with eyes open appears in step S11 of Fig. 4. In this case, the masquerade determination unit 212 may determine whether or not a subject is masquerading as another person by using the person image IMG in which a subject with eyes closed appears and the person image IMG in which a subject with eyes open appears.

[0056] As an example, the impersonation determination unit 212 may determine whether or not the subject is impersonating another person based on the difference between the person image IMG in which the subject with his / her eyes closed is reflected and the person image IMG in which the subject with his / her eyes open is reflected. Specifically, when the subject is wearing a mask M, a part of the subject that was reflected in the person image IMG when the subject's eyes are closed may not be reflected in the person image IMG when the subject's eyes are open. For example, when the subject's eyes are closed, the upper eyelids of the subject are reflected in the person image IMG, whereas when the subject's eyes are open, the upper eyelids of the subject may be hidden by the mask M. As a result, when the subject's eyes are open, the upper eyelids of the subject may not be reflected in the person image IMG. In this case, there is a difference between the person image IMG in which the subject with his / her eyes closed is reflected and the person image IMG in which the subject with his / her eyes open is reflected, with respect to whether the upper eyelids are reflected. Therefore, the masquerade determination unit 212 can determine whether or not the subject is masquerading as another person based on the difference between the person image IMG in which the subject with his / her eyes closed is captured and the person image IMG in which the subject with his / her eyes open is captured. Specifically, for example, the masquerade determination unit 212 may determine whether or not a part of the subject that was captured in the person image IMG when the subject had his / her eyes closed is no longer captured in the person image IMG when the subject had his / her eyes open. If a part of the subject that was captured in the person image IMG when the subject had his / her eyes closed is no longer captured in the person image IMG when the subject had his / her eyes open, the masquerade determination unit 212 may determine that the subject is masquerading as another person.

[0057] Even in this case, the reason why there is a difference between the person image IMG in which the subject has his / her eyes closed and the person image IMG in which the subject has his / her eyes open is still due to the aperture AP (specifically, the mask M in which the aperture AP is formed) surrounding the subject's eyes. In other words, the reason why a part of the subject that was reflected in the person image IMG when the subject has his / her eyes closed is no longer reflected in the person image IMG when the subject has his / her eyes open is still due to the aperture AP (specifically, the mask M in which the aperture AP is formed) surrounding the subject's eyes. For this reason, the operation of determining whether or not a part of the subject that was reflected in the person image IMG when the subject has his / her eyes closed is no longer reflected in the person image IMG when the subject has his / her eyes open may be considered to be substantially equivalent to the operation of determining whether or not at least a part of the subject's eye is surrounded by the aperture AP in the person image IMG.

[0058] (5) Additional Notes The following supplementary notes are further disclosed regarding the above-described embodiment. [Appendix 1] An acquisition means for acquiring a person image including an eye region of a subject; a determining means for determining that the subject is impersonating another person when an opening surrounding at least a part of the subject's eye included in the eye region is present in the person image; An information processing device comprising: [Appendix 2] The acquiring means acquires the person image including the eye region of the subject with the eyes closed. 2. An information processing device according to claim 1. [Appendix 3] The determining means (i) performs an edge extraction process on at least a portion of the person image, and (ii) determines that an opening surrounding at least a portion of the subject's eye included in the eye region is present when an edge surrounding at least a portion of the subject's eye included in the eye region is extracted. 3. The information processing device according to claim 1 or 2. [Appendix 4] The acquisition means acquires a first image which is the person image including the eye region of the subject with eyes closed, and a second image which is the person image including the eye region of the subject with eyes open, The determination means determines whether the target person is impersonating another person based on a difference between the first image and the second image. 4. The information processing device according to claim 1 . [Appendix 5] Obtaining a person image including an eye region of a subject; When an opening surrounding at least a part of the eye of the subject included in the eye region is present in the person image, the subject is determined to be impersonating another person. Information processing methods. [Appendix 6] On the computer, Acquire a person image including the eye region of the subject, When an object having an opening through which at least a part of the eye of the subject included in the eye region is exposed is present in the person image, the subject is determined to be masquerading as another person. A recording medium on which a computer program for executing an information processing method is recorded.

[0059] At least a part of the constituent elements of each of the above-mentioned embodiments can be appropriately combined with at least a part of the other constituent elements of each of the above-mentioned embodiments. Some of the constituent elements of each of the above-mentioned embodiments may not be used. In addition, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure above are incorporated by reference as part of the description of this disclosure.

[0060] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Information processing devices, information processing methods, computer programs, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0061] 1 Camera 2. Information processing equipment 21 Arithmetic unit 211 Image acquisition unit 212 Spoofing Detection Unit 3. Communication Network SYS Spoofing Detection System M Mask AP aperture IMG People Images

Claims

1. An acquisition means for acquiring a person image including an eye region of a subject; a determination means for determining that the subject is impersonating another person based on an outline of an accessory located outside at least a part of the subject's eye in a horizontal or vertical direction in the person image; An information processing device comprising:

2. The determining means determines that the subject is impersonating another person if the contour surrounds at least a portion of the subject's eye. The information processing device according to claim 1 .

3. The accessory is a mask, and the contour is a contour of an opening formed in the mask.

3. The information processing device according to claim 1 or 2.

4. The determination means determines that the subject is impersonating another person by using the person image including the eye region of the subject with the eyes closed. The information processing device according to claim 1 .

5. The determining means detects an outline of the accessory by performing an edge extraction process on at least a part of the person image. The information processing device according to claim 1 .

6. The acquisition means acquires a first image which is the person image including the eye region of the subject with eyes closed, and a second image which is the person image including the eye region of the subject with eyes open, The determination means determines whether the target person is impersonating another person based on a difference between the first image and the second image. The information processing device according to claim 1 .

7. Obtaining a person image including an eye region of a subject; It is determined that the subject is impersonating another person based on an outline of an accessory that is located outside at least a part of the subject's eye in a horizontal or vertical direction in the person image. Information processing methods.

8. On the computer, Acquire a person image that includes the target person, It is determined that the subject is impersonating another person based on an outline of an accessory that is located outside at least a part of the subject's eye in a horizontal or vertical direction in the person image. A computer program for executing an information processing method.

Images