Information processing device, information processing method, and recording medium

Abstract

To provide an information processing device capable of determining whether or not a target person in a portrait image is pretending to be another person.SOLUTION: An information processing device (2) is provided, comprising acquisition means (211) for acquiring a portrait image (IMG) including an eye area (EA) of a target person, and determination means (212) configured to determine that the target person is impersonating another person when apertures (AP) surrounding at least portions of eyes of the target person are present in the portrait image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to the technical field of, for example, an information processing apparatus, an information processing method, and a recording medium capable of determining whether a person depicted in a person image is impersonating another person.

Background Art

[0002] An example of an information processing apparatus capable of determining whether a person depicted in a person image is impersonating another person is described in Patent Document 1. Patent Document 1 describes an information processing apparatus capable of determining whether a face depicted in an image captured using a camera is a real face of a person or a fake face.

[0003] In addition, Patent Documents 2 to 5 are cited as prior art documents related to this disclosure.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure aims to provide an information processing apparatus, an information processing method, and a recording medium for improving the technologies described in the prior art documents.

Means for Solving the Problems

[0006] One aspect of the information processing apparatus includes an acquisition unit that acquires a person image in which a target person is reflected, and a determination unit that determines that the target person is impersonating another person when there is an opening in the person image that surrounds at least a part of the eyes of the target person included in the eye region.

[0007] One aspect of the information processing method is to acquire a person image in which a target person is reflected, and when there is an opening in the person image that surrounds at least a part of the eyes of the target person included in the eye region, determine that the target person is impersonating another person.

[0008] One aspect of the recording medium is a recording medium on which a computer program is recorded that causes a computer to execute an information processing method of acquiring a person image in which a target person is reflected, and when there is an opening in the person image that surrounds at least a part of the eyes of the target person included in the eye region, determining that the target person is impersonating another person.

Brief Description of Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of an information processing apparatus, an information processing method, and a recording medium will be described. Hereinafter, the embodiments of the information processing apparatus, the information processing method, and the recording medium will be described using an information processing system SYS to which the embodiments of the information processing apparatus, the information processing method, and the recording medium are applied.

[0011] (1) Configuration of Information Processing System SYS First, the configuration of the information processing system SYS of the present embodiment will be described.

[0012] (1-1) Overall Configuration of Information Processing System SYS First, with reference to FIG. 1, the overall configuration of the information processing system SYS of the present embodiment will be described. FIG. 1 is a block diagram showing the overall configuration of the information processing system SYS of the present embodiment.

[0013] As shown in FIG. 1, the information processing system SYS includes a camera 1 and an information processing apparatus 2. The information processing system SYS may include a single camera 1 or a plurality of cameras 1. The camera 1 and the information processing apparatus 2 can communicate with each other via a communication network 3. The communication network 3 may include a wired communication network. The communication network 3 may include a wireless communication network.

[0014] Camera 1 is an imaging device capable of imaging a subject (i.e., a person) located within the imaging target range of Camera 1. By imaging the subject, Camera 1 generates a person image IMG in which the subject imaged by Camera 1 is reflected. In particular, Camera 1 generates a person image IMG in which at least a part of the face of the person (especially, at least a part of the face including the eyes) is reflected by imaging the face of the subject (especially, at least a part of the face including the eyes). That is, Camera 1 generates a person image IMG including the eye region EA of the subject. The eye region EA of the subject may, for example, mean a region of a predetermined shape (e.g., a rectangular shape) including the eyes of the subject. The eye region EA of the subject may, for example, mean a region of a predetermined shape located within a certain distance range from the eyes of the subject. As an example, as shown in FIGS. 5 and 6 to be described later, the eye region EA of the subject may mean a region included in the eye image IMG_EYE. Note that as long as the person image IMG includes the eye region of the subject, it may or may not include a region other than the eye region of the subject. Therefore, the "person image IMG in which the subject is reflected" in the present embodiment may mean a person image IMG in which at least the eye region of the subject is reflected, or may mean a person image IMG in which a part of the subject including the eye region (e.g., the face) is reflected, or may mean a person image IMG in which the whole body of the subject is reflected. In any case, an image including at least the eye region of the subject may be used as the person image IMG. Camera 1 outputs the generated person image IMG to the information processing device 2. Specifically, Camera 1 transmits the generated person image IMG to the information processing device 2 via the communication network 3.

[0015] The information processing device 2 is capable of performing an impersonation determination operation. Specifically, the information processing device 2 is capable of receiving the person image IMG transmitted from Camera 1 via the communication network 3. Further, based on the received person image IMG, the information processing device 2 is capable of determining whether the subject reflected in the person image IMG (i.e., the subject in front of Camera 1) is attempting to impersonate another person different from the subject.

[0016] In this embodiment, the information processing apparatus 2 can determine whether a target person is attempting to disguise themselves as another person by wearing (for example, putting on) a mask M that mimics another person. An example of the mask M for which the impersonation determination operation is to be performed is shown in FIG. 2. As shown in FIG. 2, in this embodiment, the information processing apparatus 2 can determine whether a target person is attempting to disguise themselves as another person by wearing a mask M in which an opening AP is formed in a portion corresponding to the eyes of another person. Note that FIG. 2 shows an example in which the opening AP is an opening having a horizontally long circular shape. However, the shape of the opening AP is not limited to the shape shown in FIG. 2. For example, the shape of the opening AP may be any circular shape, any elliptical shape, any polygonal shape, or any shape corresponding to an arbitrary closed curve.

[0017] Note that the "mask" in this embodiment may mean an object (member) costume that can be worn by a target person and can cover (in other words, hide) at least a part of the target person's face when the target person is wearing it. For example, a mask that completely covers the head of the target person may be used as the mask M. For example, a mask (so-called, face) that partially covers the head of the target person (for example, covers the front surface of the target person's face) may be used as the mask M.

[0018] Here, as an example of a method for determining whether a person in front of camera 1 is impersonating another person, there is a method of determining whether the person being imaged in the person image IMG blinks according to a predetermined instruction to determine whether the person is impersonating another person. The information processing apparatus in the comparative example using this method can appropriately determine whether the person is impersonating another person when the person causes a photo in which another person is reflected or a display showing another person to be imaged by camera 1. This is because it is difficult for another person reflected in the photo or displayed on the display to blink according to a predetermined instruction. Therefore, the information processing apparatus in the comparative example typically determines that the person being imaged in the person image IMG is a different person from the person in front of camera 1 (as a result, the person is impersonating another person) when the person being imaged in the person image IMG does not blink according to a predetermined instruction. However, when the person wears a mask M in which an aperture AP is formed in a portion corresponding to the eyes of another person, the eyes of the person wearing the mask M will be exposed from the aperture AP. For this reason, if the person wearing the mask M blinks, the information processing apparatus in the comparative example will determine that the person imaged in the person image IMG blinks according to a predetermined instruction. As a result, the information processing apparatus in the comparative example has a technical problem that it erroneously determines that the person is not impersonating another person even though the person is impersonating another person.

[0019] Therefore, in the present embodiment, an information processing apparatus 2 capable of solving the above-described technical problem will be described. That is, in the present embodiment, an information processing apparatus 2 capable of appropriately determining that a person is impersonating another person in a situation where the person wears a mask M in which an aperture AP is formed in a portion corresponding to the eyes of another person will be described.

[0020] (1-2) Configuration of Information Processing Apparatus 2 Next, with reference to FIG. 3, the configuration of the information processing apparatus 2 according to the present embodiment will be described. FIG. 3 is a block diagram showing the configuration of the information processing apparatus 2 according to the present embodiment.

[0021] As shown in FIG. 3, the information processing apparatus 2 includes an arithmetic unit 21, a storage device 22, and a communication device 23. Further, the information processing apparatus 2 may include an input device 24 and an output device 25. However, the information processing apparatus 2 may not include at least one of the input device 24 and the output device 25. The arithmetic unit 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0022] The arithmetic unit 21 includes at least one of, for example, a CPU (Central Processing Unit), a GPU (Graphics Proecssing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic unit 21 reads a computer program. For example, the arithmetic unit 21 may read a computer program stored in the storage device 22. For example, the arithmetic unit 21 may read a computer program stored in a computer-readable and non-temporary recording medium using a recording medium reading device (not shown) provided in the information processing apparatus 2. The arithmetic unit 21 may acquire a computer program from a device (not shown) disposed outside the information processing apparatus 2 via the communication device 23 (or another communication device) (that is, it may be downloaded or read). The arithmetic unit 21 executes the read computer program. As a result, logical functional blocks for executing operations (for example, the forgery determination operation described above) to be performed by the information processing apparatus 2 are realized in the arithmetic unit 21. That is, the arithmetic unit 21 can function as a controller for realizing logical functional blocks for executing operations (in other words, processing) to be performed by the information processing apparatus 2.

[0023] FIG. 3 shows an example of logical function blocks realized in the arithmetic unit 21 to execute the forgery determination operation. As shown in FIG. 3, in the arithmetic unit 21, an image acquisition unit 211, which is a specific example of the "acquisition means", and a forgery determination unit 212, which is a specific example of the "determination means", are realized. The image acquisition unit 211 can acquire a person image IMG from the camera 1. The forgery determination unit 212 can determine whether the subject person (i.e., the subject person in front of the camera 1) shown in the person image IMG acquired by the image acquisition unit 211 is attempting to impersonate another person different from the subject person.

[0024] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic unit 21. The storage device 22 may temporarily store data that the arithmetic unit 21 temporarily uses when executing the computer program. The storage device 22 may store data that the information processing device 2 stores long-term. Note that the storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. That is, the storage device 22 may include a non-temporary recording medium.

[0025] The communication device 23 can communicate with the camera 1 via the communication network 3. In the present embodiment, the communication device 23 receives (i.e., acquires) a person image IMG from the camera 1 via the communication network 3.

[0026] The input device 24 is a device that receives input of information to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operating device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on an externally attachable recording medium with respect to the information processing device 2.

[0027] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called display) that can display an image indicating the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include a sound device (so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (so-called printer) that can print desired information on paper.

[0028] (2) Spoofing determination operation performed by the information processing device 2 Subsequently, with reference to FIG. 4, the spoofing determination operation performed by the information processing device 2 will be described. FIG. 4 is a flowchart showing the flow of the spoofing determination operation performed by the information processing device 2.

[0029] As shown in FIG. 4, the image acquisition unit 211 acquires a person image IMG from the camera 1 using the communication device 23 (step S11).

[0030] Thereafter, the spoofing determination unit 212 determines, based on the person image IMG acquired in step S11, whether the subject captured in the person image IMG (that is, the subject in front of the camera 1) is attempting to impersonate another person different from the subject (steps S12 to S14).

[0031] Here, FIG. 5 shows an example of a person image IMG in which a subject wearing a mask M with an opening AP formed therein is reflected, together with an enlarged view of a partial image portion (hereinafter referred to as "eye image IMG_EYE") including the eye region EA of the subject in the person image IMG. As shown in FIG. 5, when the subject wears a mask M with an opening AP formed therein, the person image IMG shows the eyes of the subject with at least a part thereof exposed from the opening AP, together with the mask M imitating a person other than the subject. That is, in the person image IMG, at least a part of the eyes of the subject is exposed from the opening AP.

[0032] In other words, when the subject wears a mask M with an opening AP formed therein, the person image IMG shows the eyes of the subject at least partially surrounded by the opening AP. That is, in the person image IMG, at least a part of the eyes of the subject is surrounded by the opening AP. Here, the state where "at least a part of the eyes of the subject is surrounded by the opening AP" may mean, for example, the state where "at least a part of the eyes of the subject is surrounded by the outer edge AP_edge of the opening AP (that is, the outer edge of the mask M forming the opening AP)".

[0033] Note that FIG. 5 shows an example where the entire eyes of the subject are exposed from the opening AP (that is, the entire eyes of the subject are surrounded by the opening AP). The state where "the entire eyes of the subject are exposed from the opening AP" may mean, for example, the state where "the entire lower edge ul_edge of the upper eyelid and the entire upper edge ul_edge of the lower eyelid of the subject are exposed from the opening AP". The state where "the entire eyes of the subject are surrounded by the opening AP" may mean, for example, the state where "the entire lower edge ul_edge of the upper eyelid and the entire upper edge ul_edge of the lower eyelid of the subject are surrounded by the opening AP".

[0034] However, it is not necessary for a part of the subject's eye to be exposed from the opening AP. That is, a part of the subject's eye does not have to be surrounded by the opening AP. For example, when the horizontal size of the subject's eye (e.g., the length from the outer corner to the inner corner of the eye) is larger than the horizontal size of the opening AP, a part of the subject's eye is covered by the mask M without being exposed from the opening AP. Similarly, when the vertical size of the subject's eye (e.g., the distance between the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid) is larger than the vertical size of the opening AP, a part of the subject's eye is covered by the mask M without being exposed from the opening AP. Therefore, in these cases, while a part of the subject's eye is exposed from the opening AP, another part of the subject's eye does not have to be exposed from the opening AP. That is, while a part of the subject's eye is exposed from the opening AP, another part of the subject's eye may be covered by the mask M. In other words, while a part of the subject's eye is surrounded by the opening AP, another part of the subject's eye does not have to be surrounded by the opening AP. In any case, when the subject wears the mask M having the opening AP formed therein, at least a part (i.e., the whole or a part) of the subject's eye is still exposed from the opening AP (i.e., surrounded by the opening AP).

[0035] Considering such a state where the subject's eye is surrounded by the opening AP, in the present embodiment, the "eye" may mean a part of the subject including not only the eyeball but also the appendages associated with the eyeball (particularly, appendages visible from the outside, and as an example, eyelids including the upper and lower eyelids described above). Therefore, the state of "at least a part of the subject's eye being surrounded by the opening AP" in the present embodiment may include a state where at least a part of the subject's eyeball is surrounded by the opening AP, or may include a state where at least a part of the appendages associated with the subject's eyeball is surrounded by the opening AP. However, in the present embodiment, the "eye" may mean a part of the subject that includes the eyeball but does not include the appendages associated with the eyeball.

[0036] On the other hand, FIG. 6 shows an example of a person image IMG in which a subject not wearing a mask M with an opening AP formed thereon is captured, together with an enlarged view of an eye image IMG_EYE including the eye region EA of the subject in the person image IMG. As shown in FIG. 6, when the subject is not wearing the mask M with the opening AP formed thereon, at least a part of the subject's eyes does not appear to be exposed from the opening AP in the person image IMG. That is, in the person image IMG, at least a part of the subject's eyes is not surrounded by the opening AP (for example, not surrounded by the outer edge AP_edge of the opening AP).

[0037] In the present embodiment, the forgery determination unit 212 determines whether the subject is attempting to impersonate another person based on the difference between the person image IMG in which the subject wearing such a mask M is captured and the person image IMG in which the subject not wearing the mask M is captured. Specifically, the forgery determination unit 212 determines whether at least a part of the subject's eyes is surrounded by the opening AP (for example, surrounded by the outer edge AP_edge of the opening AP) in the person image IMG acquired in step S11 (step S12). That is, the forgery determination unit 212 determines whether there is an opening AP surrounding at least a part of the subject's eyes (for example, whether there is an outer edge AP_edge of the opening AP surrounding at least a part of the subject's eyes) in the person image IMG acquired in step S11 (step S12). In other words, the forgery determination unit 212 determines whether there is an object (in this embodiment, the mask M) with an opening AP formed thereon surrounding at least a part of the subject's eyes in the person image IMG acquired in step S11 (step S12).

[0038] As described above, when at least a part of the subject's eyes is surrounded by the opening AP, at least a part of the subject's eyes will be exposed from the opening AP. Therefore, the operation of determining whether at least a part of the subject's eyes is surrounded by the opening AP in step S12 may be regarded as substantially equivalent to the operation of determining whether at least a part of the subject's eyes is exposed from the opening AP. The operation of determining whether at least a part of the subject's eyes is surrounded by the opening AP in step S12 may be regarded as substantially equivalent to the operation of determining whether there is an opening AP from which at least a part of the subject's eyes is exposed. The operation of determining whether at least a part of the subject's eyes is surrounded by the opening AP in step S12 may be regarded as substantially equivalent to the operation of determining whether there is an object (in this embodiment, the mask M) in which an opening AP from which at least a part of the subject's eyes is exposed is formed.

[0039] The forgery determination unit 212 may determine whether at least a part of the subject's eyes is surrounded by the opening AP by performing an image analysis process for analyzing at least a part (for example, the above-described eye image IMG_EYE) of the person image IMG. For example, the forgery determination unit 212 may detect the subject's eyes from the person image IMG and determine whether there is an opening AP surrounding the subject's eyes in the vicinity of the subject's eyes. In this case, the forgery determination unit 212 may determine whether there is an opening AP in the person image IMG by performing template matching using a template corresponding to the opening AP.

[0040] The forgery determination unit 212 may perform edge extraction processing on at least a part of the person image IMG, and determine whether there is an edge surrounding the target person's eyes (that is, the contour of any object surrounding the target person's eyes, and here, the contour of the mask M forming the outer edge AP_edge of the opening AP), thereby determining whether at least a part of the target person's eyes is surrounded by the opening AP. Specifically, the forgery determination unit 212 may generate an edge image representing the edges within at least a part of the person image IMG by performing edge extraction processing on at least a part of the person image IMG (for example, the above-mentioned eye image IMG_EYE). For example, the forgery determination unit 212 may generate an edge image by performing filtering processing using a first-order differential filter on the person image IMG. The forgery determination unit 212 may perform filtering processing using at least one of a vertical first-order differential filter and a horizontal first-order differential filter. In the edge image, usually, edges corresponding to the target person's eyes appear. The edges corresponding to the target person's eyes may include at least one of the edge corresponding to the lower edge ul_edge of the upper eyelid and the edge corresponding to the upper edge ul_edge of the lower eyelid. Further, when the target person is wearing the mask M, in the edge image, an edge corresponding to the opening AP appears. The edge corresponding to the opening AP may include an edge corresponding to the outer edge AP_edge of the opening AP (that is, the edge of a part of the mask M forming the outer edge AP_edge of the opening AP, and typically, an edge forming a closed loop). Therefore, the forgery determination unit 212 may determine whether there is an edge corresponding to the opening AP surrounding the edge corresponding to the target person's eyes in the edge image. When there is an edge corresponding to the opening AP surrounding the edge corresponding to the target person's eyes, the forgery determination unit 212 may determine that at least a part of the target person's eyes is surrounded by the opening AP.

[0041] The forgery determination unit 212 may determine whether at least a part of the target person's eyes is surrounded by the opening AP using a learnable arithmetic model. For example, when at least a part of the person image IMG is input, the forgery determination unit 212 may determine whether at least a part of the target person's eyes is surrounded by the opening AP using an arithmetic model that outputs a determination result as to whether at least a part of the target person's eyes is surrounded by the opening AP. For example, when an edge image generated by performing edge extraction processing on at least a part of the person image IMG is input, the forgery determination unit 212 may determine whether at least a part of the target person's eyes is surrounded by the opening AP using an arithmetic model that outputs a determination result as to whether at least a part of the target person's eyes is surrounded by the opening AP.

[0042] As an example of a learnable arithmetic model, an arithmetic model using a neural network can be cited. The forgery determination unit 212 may determine whether at least a part of the target person's eyes is surrounded by the opening AP using an arithmetic model that has been learned by machine learning based on at least one of supervised learning and unsupervised learning.

[0043] If, as a result of the determination in step S12, it is determined that at least a part of the target person's eyes is surrounded by the opening AP (step S12: Yes), it is assumed that the target person shown in the person image IMG is highly likely to be wearing the mask M in which the opening AP is formed. Therefore, in this case, the forgery determination unit 212 determines that the target person is impersonating another person (step S13).

[0044] On the other hand, if, as a result of the determination in step S12, it is determined that at least a part of the target person's eyes is not surrounded by the opening AP (step S12: No), it is assumed that the target person shown in the person image IMG is less likely to be wearing the mask M in which the opening AP is formed. Therefore, in this case, the forgery determination unit 212 determines that the target person is not impersonating another person (step S14).

[0045] When the impersonation determination unit 212 determines that the subject is not impersonating another person, the information processing apparatus 2 may authenticate the subject using the person image IMG. For example, when the face of the subject is captured in the person image IMG, the information processing apparatus 2 may authenticate the subject by performing face authentication using the face of the subject captured in the person image IMG. Further, since the eye region EA of the subject is included in the person image IMG, the information processing apparatus 2 may authenticate the subject by performing iris authentication using the iris of the subject captured in the person image IMG. Further, the information processing apparatus 2 may authenticate the subject by performing any authentication different from face authentication and iris authentication. Further, the information processing apparatus 2 may authenticate the subject by performing any authentication different from the authentication using the person image IMG.

[0046] (3) Technical effects of the information processing apparatus 2 As described above, when at least a part of the eyes of the subject is surrounded by the opening AP in the person image IMG (that is, there is an opening AP surrounding at least a part of the eyes of the subject), the information processing apparatus 2 determines that the subject is impersonating another person. Therefore, the information processing apparatus 2 can appropriately determine that the subject is impersonating another person in a situation where the subject is impersonating another person by wearing the mask M in which the opening AP is formed in a part corresponding to the eyes of another person.

[0047] As described above, in order to determine whether at least a part of the eyes of the target person is surrounded by the opening AP in the person image IMG, the information processing apparatus 2 may use an edge image generated by performing edge extraction processing on at least a part of the person image IMG. That is, the information processing apparatus 2 uses an edge image in which the contours of the objects existing in the person image IMG (for example, the eyes of the target person and the mask M in which the opening AP is formed) are emphasized to determine whether at least a part of the eyes of the target person is surrounded by the opening AP. In this case, compared with the case where the edge image is not used, the information processing apparatus 2 can more accurately determine whether at least a part of the eyes of the target person is surrounded by the opening AP. As a result, the information processing apparatus 2 can more accurately determine whether the target person is impersonating another person.

[0048] (4) Modification example (4-1) First modification example When the target person is wearing the mask M, as shown in FIG. 5, in the person image IMG, at least a part of the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid of the target person is located at a position away from the outer edge AP_edge of the opening AP. This is because usually the size of the opening AP formed in the mask M and the size of the eyes of the target person wearing the mask M are unlikely to match. When at least a part of the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid of the target person is located at a position away from the outer edge AP_edge of the opening AP in the person image IMG in this way, the impersonation determination unit 212 is likely to determine that at least a part of the eyes of the target person is surrounded by the opening AP in the person image IMG.

[0049] On the other hand, it cannot be said that the possibility that the size of the opening AP formed in the mask M matches the size of the eyes of the subject wearing the mask M is zero. When the size of the opening AP formed in the mask M matches the size of the eyes of the subject wearing the mask M, in the person image IMG, the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid of the subject may each overlap with the outer edge AP_edge of the opening AP. In this case, the forgery determination unit 212 may determine that at least a part of the subject's eyes in the person image IMG is not surrounded by the opening AP. This is because the outer edge AP_edge of the opening AP where the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid overlap may be misrecognized as the contour of the eyes (that is, the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid). As a result, the determination accuracy of whether the subject is impersonating another person may decrease.

[0050] Therefore, in order to accurately determine whether the subject is impersonating another person even when the subject is wearing the mask M with the aperture AP having a size that matches the size of the subject's eyes, the image acquisition unit 211 may acquire a person image IMG in which the subject with closed eyes is captured in step S11 of FIG. 4. An example of the person image IMG in which the subject wearing the mask M with the aperture AP formed thereon and having closed eyes is captured is shown in FIG. 7. As shown in FIG. 7, when the subject has closed eyes, even if the size of the aperture AP formed in the mask M matches the size of the eyes of the subject wearing the mask M, in the person image IMG, the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid of the subject are less likely to overlap the outer edge AP_edge of the aperture AP. For this reason, the impersonation determination unit 212 can determine that at least a part of the subject's eyes is surrounded by the aperture AP in the person image IMG in a situation where the subject is wearing the mask M with the aperture AP having a size that matches the size of the subject's eyes. As a result, the impersonation determination unit 212 can accurately determine whether the subject is impersonating another person even when the subject is wearing the mask M with the aperture AP having a size that matches the size of the subject's eyes.

[0051] In addition, not limited to the situation where the subject is wearing the mask M with the aperture AP having a size that matches the size of the subject's eyes, the image acquisition unit 211 may acquire a person image IMG in which the subject with closed eyes is captured in step S11 of FIG. 4.

[0052] In order to acquire the person image IMG in which the subject with closed eyes is captured, the information processing device 2 may instruct the subject to close their eyes. For example, when the information processing device 2 includes a display near the camera 1, the information processing device 2 may display a GUI (Graphical User Interface) for instructing the subject to close their eyes.

[0053] Alternatively, the camera 1 may continuously photograph the subject over time. That is, the camera 1 may generate a plurality of person images IMG as time-series data. In this case, the impersonation determination unit 212 may extract at least one person image IMG in which a subject with closed eyes is captured from among the plurality of person images IMG, and determine whether at least a part of the subject's eyes is surrounded by the opening AP in the extracted at least one person image IMG.

[0054] In addition, the information processing apparatus 2 may perform an impersonation determination operation of determining whether the subject captured in the person image IMG is blinking, in the same manner as the information processing apparatus of the above-described comparative example, in parallel with the impersonation determination operation shown in FIG. 4, to determine whether the subject is impersonating another person. In this case, in order to determine whether the subject is blinking, the camera 1 usually continuously photographs the subject over time. That is, the camera 1 generates a plurality of person images IMG as time-series data. Also in this case, the impersonation determination unit 212 may extract at least one person image IMG in which a subject with closed eyes is captured from among the plurality of person images IMG, and determine whether at least a part of the subject's eyes is surrounded by the opening AP in the extracted at least one person image IMG. As a result, the information processing apparatus 2 can perform the impersonation determination operation shown in FIG. 4 and the impersonation determination operation using blinking in parallel.

[0055] (4-2) Second Modified Example The image acquisition unit 211 may acquire both a person image IMG in which a subject with closed eyes is captured and a person image IMG in which a subject with open eyes is captured in step S11 of FIG. 4. In this case, the impersonation determination unit 212 may use the person image IMG in which a subject with closed eyes is captured and the person image IMG in which a subject with open eyes is captured to determine whether the subject is impersonating another person.

[0056] As an example, the impersonation determination unit 212 may determine whether the subject is impersonating another person based on the difference between the person image IMG in which the subject with closed eyes is reflected and the person image IMG in which the subject with open eyes is reflected. Specifically, when the subject is wearing the mask M, part of the subject that was reflected in the person image IMG in the situation where the subject has closed eyes may not be reflected in the person image IMG in the situation where the subject has open eyes. For example, in the situation where the subject has closed eyes, the upper eyelid of the subject is reflected in the person image IMG, while in the situation where the subject has open eyes, the upper eyelid of the subject may be hidden by the mask M. As a result, in the situation where the subject has open eyes, the upper eyelid of the subject may not be reflected in the person image IMG. In this case, there is a difference regarding the presence or absence of the reflection of the upper eyelid between the person image IMG in which the subject with closed eyes is reflected and the person image IMG in which the subject with open eyes is reflected. Therefore, the impersonation determination unit 212 can determine whether the subject is impersonating another person based on the difference between the person image IMG in which the subject with closed eyes is reflected and the person image IMG in which the subject with open eyes is reflected. Specifically, for example, the impersonation determination unit 212 may determine whether part of the subject that was reflected in the person image IMG in the situation where the subject has closed eyes is not reflected in the person image IMG in the situation where the subject has open eyes. When part of the subject that was reflected in the person image IMG in the situation where the subject has closed eyes is not reflected in the person image IMG in the situation where the subject has open eyes, the impersonation determination unit 212 may determine that the subject is impersonating another person.

[0057] Even in this case, the reason for the difference between the person image IMG in which the subject with closed eyes is reflected and the person image IMG in which the subject with open eyes is reflected remains the aperture AP surrounding the subject's eyes (specifically, the mask M in which the aperture AP is formed). That is, a part of the subject that was reflected in the person image IMG when the subject had closed eyes no longer appears in the person image IMG when the subject has open eyes because of the aperture AP surrounding the subject's eyes (specifically, the mask M in which the aperture AP is formed). Therefore, the operation of determining whether or not a part of the subject that was reflected in the person image IMG when the subject had closed eyes no longer appears in the person image IMG when the subject has open eyes may be regarded as substantially equivalent to the operation of determining whether or not at least a part of the subject's eyes in the person image IMG is surrounded by the aperture AP.

[0058] (5) Supplementary Note Regarding the embodiments described above, the following supplementary notes are further disclosed. [Supplementary Note 1] An acquisition means for acquiring a person image including the eye region of the subject, A determination means for determining that the subject is impersonating another person when there is an aperture in the person image that surrounds at least a part of the subject's eyes included in the eye region An information processing apparatus comprising the same. [Supplementary Note 2] The acquisition means acquires the person image including the eye region of the subject with closed eyes The information processing apparatus according to Supplementary Note 1. [Supplementary Note 3] The determination means (i) performs edge extraction processing on at least a part of the person image, and (ii) determines that there is an aperture surrounding at least a part of the subject's eyes included in the eye region when an edge surrounding at least a part of the subject's eyes included in the eye region is extracted The information processing apparatus according to Supplementary Note 1 or 2. [Supplementary Note 4] The acquisition means acquires a first image which is a person image including the eye region of the subject with eyes closed and a second image which is a person image including the eye region of the subject with eyes open. The determination means determines whether the subject is impersonating another person based on the difference between the first image and the second image. The information processing apparatus according to any one of Appendices 1 to 3. [Appendix 5] Acquire a person image including the eye region of the subject. In the person image, when there is an opening surrounding at least a part of the eyes of the subject included in the eye region, it is determined that the subject is impersonating another person. Information processing method. [Appendix 6] Cause a computer to Acquire a person image including the eye region of the subject, and In the person image, when there is an object in which an opening is formed where at least a part of the eyes of the subject included in the eye region is exposed, it is determined that the subject is impersonating another person. A recording medium on which a computer program for executing an information processing method is recorded.

[0059] At least a part of the constituent elements of each of the above-described embodiments can be appropriately combined with at least another part of the constituent elements of each of the above-described embodiments. It is not necessary to use some of the constituent elements of each of the above-described embodiments. Also, to the extent permitted by law, the disclosures of all the documents (for example, published gazettes) cited in this disclosure are incorporated by reference to form part of the description of this disclosure.

[0060] This disclosure can be appropriately modified within a range not contrary to the technical idea that can be read from the claims and the entire specification. An information processing apparatus, an information processing method, a computer program, and a recording medium involving such modifications are also included in the technical idea of this disclosure.

Explanation of Reference Numerals

[0061] 1 Camera 2 Information processing device 21 Arithmetic unit 211 Image acquisition unit 212 Impersonation determination unit 3 Communication network SYS Impersonation determination system M Mask AP Aperture IMG Portrait image

Claims

1. An acquisition means for acquiring a person image including the eye region of a target person, A determination means for determining, based on the contour of an ornament located outside the target person's eyes in the horizontal or vertical direction of the target person's eyes in the person image, that the target person is disguising as another person An information processing apparatus comprising the above.

2. The determination means determines that the target person is disguising as another person when the contour surrounds at least a part of the target person's eyes The information processing apparatus according to claim 1.

3. The ornament is a mask, and the contour is the contour of an opening formed in the mask. The information processing apparatus according to claim 1 or 2.

4. The determination means determines that the target person is disguising as another person by using the person image including the eye region of the target person with the eyes closed The information processing apparatus according to any one of claims 1 to 3.

5. The determination means detects the contour of the ornament by performing edge extraction processing on at least a part of the person image The information processing apparatus according to any one of claims 1 to 4.

6. The acquisition means acquires a first image which is the person image including the eye region of the target person with the eyes closed and a second image which is the person image including the eye region of the target person with the eyes open, The determination means determines whether the target person is disguising as another person based on the difference between the first image and the second image The information processing apparatus according to any one of claims 1 to 5.

7. Acquire a person image including the eye region of a target person, In the person image, based on the contour of an ornament located outside the target person's eyes in the horizontal or vertical direction of the target person's eyes, determine that the target person is disguising as another person An information processing method.

8. A computer program that causes a computer to Acquire a person image in which a target person is captured, In the person image, based on the contour of an ornament located outside the target person's eyes in the horizontal or vertical direction of the target person's eyes, determine that the target person is disguising as another person Execute an information processing method.

Images