Information processing device, information processing method, and recording medium

Abstract

To provide an information processing device capable of determining whether or not a target person in a portrait image is pretending to be another person.SOLUTION: An information processing device (2) is provided, comprising acquisition means (211) for acquiring a portrait image (IMG) including an eye area (EA) of a target person, and determination means (212) configured to determine that the target person is impersonating another person when apertures (AP) surrounding at least portions of eyes of the target person are present in the portrait image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This disclosure relates to the technical field of an information processing apparatus, an information processing method, and a recording medium that can determine whether a person shown in a person image is impersonating another person.

Background Art

[0002] An example of an information processing apparatus that can determine whether a person shown in a person image is impersonating another person is described in Patent Document 1. Patent Document 1 describes an information processing apparatus that can determine whether a face shown in an image captured using a camera is a real face of a person or a fake face.

[0003] In addition, Patent Documents 2 to 5 are cited as prior art documents related to this disclosure.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure aims to provide an information processing apparatus, an information processing method, and a recording medium for improving the technologies described in the prior art documents.

Means for Solving the Problems

[0006] One aspect of the information processing device includes an acquisition means for acquiring a person image in which a subject is captured, and a determination means for determining that the subject is impersonating another person if there is an opening in the person image that surrounds at least a part of the subject's eyes included in the eye region.

[0007] One aspect of the information processing method involves acquiring a person image in which the subject is captured, and determining that the subject is impersonating another person if there is an opening in the person image that surrounds at least a part of the subject's eyes included in the eye region.

[0008] One embodiment of a recording medium is a recording medium on which a computer program is recorded that causes a computer to execute an information processing method that acquires a person image in which a subject is captured, and determines that the subject is impersonating another person if there is an opening in the person image that surrounds at least a part of the subject's eyes included in the eye region. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing the overall configuration of the information processing system in this embodiment. [Figure 2] Figure 2 shows an example of a mask worn by a subject in order to impersonate another person. [Figure 3] Figure 3 is a block diagram showing the configuration of the information processing device according to this embodiment. [Figure 4] Figure 4 is a flowchart showing the flow of the impersonation detection operation performed by the information processing device of this embodiment. [Figure 5] Figure 5 is a plan view showing an example of a person image in which a person wearing a mask with an opening is captured, along with an enlarged view of a portion of the image, including the person's eyes. [Figure 6]Figure 6 is a plan view showing an example of a person image in which the subject is not wearing a mask with an opening, along with an enlarged view of a portion of the image, including the subject's eyes. [Figure 7] Figure 7 is a plan view showing an example of a person image in which the subject is wearing a mask with an opening and has their eyes closed, along with an enlarged view of a portion of the image, including the subject's eyes. [Modes for carrying out the invention]

[0010] The following describes embodiments of the information processing device, information processing method, and recording medium. The embodiments of the information processing device, information processing method, and recording medium will be described using the SYS information processing system to which these embodiments are applied.

[0011] (1) Configuration of the information processing system SYS First, the configuration of the information processing system SYS in this embodiment will be described.

[0012] (1-1) Overall configuration of the SYS information processing system First, the overall configuration of the information processing system SYS of this embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the overall configuration of the information processing system SYS of this embodiment.

[0013] As shown in Figure 1, the information processing system SYS comprises a camera 1 and an information processing device 2. The information processing system SYS may have a single camera 1 or multiple cameras 1. The camera 1 and the information processing device 2 can communicate with each other via a communication network 3. The communication network 3 may include a wired communication network or a wireless communication network.

[0014] Camera 1 is an imaging device capable of imaging a subject (i.e., a person) located within the imaging target range of Camera 1. By imaging the subject, Camera 1 generates a person image IMG in which the subject imaged by Camera 1 is reflected. In particular, Camera 1 generates a person image IMG in which at least a part of the face of the person (especially including the eyes) is reflected by imaging the face of the subject (especially at least a part of the face including the eyes). That is, Camera 1 generates a person image IMG including the eye area EA of the subject. The eye area EA of the subject may, for example, mean an area of a predetermined shape (e.g., a rectangular shape) including the eyes of the subject. The eye area EA of the subject may, for example, mean a predetermined-shaped area located within a certain distance range from the eyes of the subject. As an example, as shown in FIGS. 5 and 6 to be described later, the eye area EA of the subject may mean the area included in the eye image IMG_EYE. Note that the person image IMG may or may not include an area other than the eye area of the subject as long as it includes the eye area of the subject. Therefore, the "person image IMG in which the subject is reflected" in the present embodiment may mean a person image IMG in which at least the eye area of the subject is reflected, or may mean a person image IMG in which a part of the subject including the eye area (e.g., the face) is reflected, or may mean a person image IMG in which the whole body of the subject is reflected. In any case, an image including at least the eye area of the subject may be used as the person image IMG. Camera 1 outputs the generated person image IMG to the information processing device 2. Specifically, Camera 1 transmits the generated person image IMG to the information processing device 2 via the communication network 3.

[0015] The information processing device 2 is capable of performing an impersonation determination operation. Specifically, the information processing device 2 can receive the person image IMG transmitted from Camera 1 via the communication network 3. Further, based on the received person image IMG, the information processing device 2 can determine whether the subject reflected in the person image IMG (i.e., the subject in front of Camera 1) is attempting to impersonate another person different from the subject.

[0016] In this embodiment, the information processing apparatus 2 can determine whether a target person is attempting to impersonate another person by wearing (e.g., putting on) a mask M that mimics another person. An example of the mask M for the impersonation determination operation is shown in FIG. 2. As shown in FIG. 2, in this embodiment, the information processing apparatus 2 can determine whether a target person is attempting to impersonate another person by wearing a mask M in which an opening AP is formed in a portion corresponding to the eyes of another person. Note that FIG. 2 shows an example in which the opening AP is an opening having a horizontally long circular shape. However, the shape of the opening AP is not limited to the shape shown in FIG. 2. For example, the shape of the opening AP may be any circular shape, any elliptical shape, any polygonal shape, or any shape corresponding to an arbitrary closed curve.

[0017] Note that the "mask" in this embodiment may mean an object (member) accessory that can be worn by a target person and can cover (in other words, hide) at least a part of the target person's face when the target person wears it. For example, a mask that completely covers the target person's head may be used as the mask M. For example, a mask that partially covers the target person's head (e.g., covers the front of the target person's face) (so-called, a face mask) may be used as the mask M.

[0018] Here, as an example of a method for determining whether a subject in front of camera 1 is impersonating another person, one method is to determine whether the subject, captured in the person image IMG, is blinking according to a predetermined instruction. The information processing device of the comparative example using this method can appropriately determine whether a subject is impersonating another person, mainly when the subject is having camera 1 capture a photograph containing another person or a display showing another person in order to impersonate another person. This is because it is difficult for another person captured in a photograph or displayed on a screen to blink according to a predetermined instruction. For this reason, the information processing device of the comparative example typically determines that the person captured in the person image IMG is a different person from the subject in front of camera 1 (and therefore the subject is impersonating another person) if the subject captured in the person image IMG is not blinking according to a predetermined instruction. However, if the subject is wearing a mask M with an aperture AP formed in the area corresponding to the eyes of the other person, the eyes of the subject wearing the mask M will be exposed through the aperture AP. Therefore, if a subject wearing mask M blinks, the comparative example's information processing device will determine that the subject captured in the person image IMG is blinking in accordance with the prescribed instructions. As a result, the comparative example's information processing device has a technical problem in that it incorrectly determines that the subject is not impersonating another person, even though the subject is actually impersonating another person.

[0019] Therefore, in this embodiment, we will describe an information processing device 2 that can solve the above-mentioned technical problems. Specifically, in this embodiment, we will describe an information processing device 2 that can appropriately determine that a subject is impersonating another person when the subject is wearing a mask M in which an opening AP is formed in the part corresponding to the eye of another person.

[0020] (1-2) Configuration of the information processing device 2 Next, the configuration of the information processing device 2 of this embodiment will be described with reference to Figure 3. Figure 3 is a block diagram showing the configuration of the information processing device 2 of this embodiment.

[0021] As shown in Figure 3, the information processing device 2 comprises an arithmetic unit 21, a storage device 22, and a communication device 23. Furthermore, the information processing device 2 may also include an input device 24 and an output device 25. However, the information processing device 2 does not have to include at least one of the input device 24 and the output device 25. The arithmetic unit 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0022] The arithmetic unit 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic unit 21 reads a computer program. For example, the arithmetic unit 21 may read a computer program stored in the storage device 22. For example, the arithmetic unit 21 may read a computer program stored in a computer-readable and non-temporary recording medium using a recording medium reading device (not shown) provided by the information processing device 2. The arithmetic unit 21 may obtain a computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or other communication device) (i.e., it may download or read the program). The arithmetic unit 21 executes the read computer program. As a result, logical functional blocks for performing the operations that the information processing device 2 should perform (for example, the impersonation detection operation described above) are realized within the arithmetic unit 21. In other words, the arithmetic unit 21 can function as a controller for realizing the logical functional blocks necessary to perform the operations (in other words, processing) that the information processing device 2 should perform.

[0023] Figure 3 shows an example of a logical functional block implemented within the arithmetic unit 21 to perform the impersonation detection operation. As shown in Figure 3, the arithmetic unit 21 implements an image acquisition unit 211, which is a specific example of an "acquisition means," and an impersonation detection unit 212, which is a specific example of a "determination means." The image acquisition unit 211 can acquire a person image IMG from camera 1. Based on the person image IMG acquired by the image acquisition unit 211, the impersonation detection unit 212 can determine whether the subject captured in the person image IMG (i.e., the subject in front of camera 1) is attempting to impersonate someone other than the subject.

[0024] The storage device 22 is capable of storing desired data. For example, the storage device 22 may temporarily store computer programs executed by the arithmetic unit 21. The storage device 22 may temporarily store data that the arithmetic unit 21 uses temporarily when it is executing a computer program. The storage device 22 may store data that the information processing device 2 stores long-term. The storage device 22 may include at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, magneto-optical disk drive, SSD (Solid State Drive), and disk array device. In other words, the storage device 22 may include non-temporary recording media.

[0025] The communication device 23 can communicate with the camera 1 via the communication network 3. In this embodiment, the communication device 23 receives (i.e., acquires) a person image IMG from the camera 1 via the communication network 3.

[0026] The input device 24 is a device that receives information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operating device (for example, at least one of a keyboard, mouse, and touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be attached externally to the information processing device 2.

[0027] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called display) capable of displaying an image representing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (so-called speaker) capable of outputting sound. For example, the output device 25 may output information onto paper. That is, the output device 25 may include a printing device (so-called printer) capable of printing desired information onto paper.

[0028] (2) Impersonation detection operation performed by the information processing device 2 Next, we will explain the impersonation detection operation performed by the information processing device 2, referring to Figure 4. Figure 4 is a flowchart showing the flow of the impersonation detection operation performed by the information processing device 2.

[0029] As shown in Figure 4, the image acquisition unit 211 acquires a person image IMG from the camera 1 using the communication device 23 (step S11).

[0030] Subsequently, the impersonation detection unit 212 determines, based on the person image IMG acquired in step S11, whether the subject captured in the person image IMG (i.e., the subject in front of camera 1) is attempting to impersonate someone other than the subject (steps S12 to S14).

[0031] Here, Figure 5 shows an example of a person image IMG in which a subject wearing a mask M with an opening AP is captured, along with a magnified view of a portion of the image IMG that includes the subject's eye region EA (hereinafter referred to as "eye image IMG_EYE"). As shown in Figure 5, when a subject is wearing a mask M with an opening AP, the person image IMG captures the subject's eyes, at least partially exposed through the opening AP, along with a mask M that mimics another person. In other words, in the person image IMG, at least a portion of the subject's eyes is exposed through the opening AP.

[0032] In other words, if the subject is wearing a mask M with an aperture AP formed on it, the person image IMG will show the subject's eyes, which are at least partially surrounded by the aperture AP. That is, in the person image IMG, at least a portion of the subject's eyes are surrounded by the aperture AP. Note that the state "at least a portion of the subject's eyes are surrounded by the aperture AP" may also mean, for example, "at least a portion of the subject's eyes are surrounded by the outer edge AP_edge of the aperture AP (i.e., the outer edge of the mask M that forms the aperture AP)."

[0033] Figure 5 shows an example where the entire eye of the subject is exposed from the opening AP (i.e., the entire eye of the subject is surrounded by the opening AP). The state of "the entire eye of the subject is exposed from the opening AP" may also mean, for example, "the entire lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the lower eyelid are exposed from the opening AP." The state of "the entire eye of the subject is surrounded by the opening AP" may also mean, for example, "the entire lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the lower eyelid are surrounded by the opening AP."

[0034] However, it is not necessary for a portion of the subject's eye to be exposed from the opening AP. In other words, it is not necessary for a portion of the subject's eye to be surrounded by the opening AP. For example, if the lateral size of the subject's eye (e.g., the length from the outer corner to the inner corner of the eye) is greater than the lateral size of the opening AP, a portion of the subject's eye will be covered by the mask M without being exposed from the opening AP. Similarly, if the vertical size of the subject's eye (e.g., the distance between the lower edge of the upper eyelid ul_edge and the upper edge of the lower eyelid ul_edge) is greater than the vertical size of the opening AP, a portion of the subject's eye will be covered by the mask M without being exposed from the opening AP. Therefore, in these cases, a portion of the subject's eye may be exposed from the opening AP while another portion of the subject's eye is not exposed from the opening AP. In other words, a portion of the subject's eye may be surrounded by the opening AP while another portion of the subject's eye is not surrounded by the opening AP. In any case, when a subject is wearing a mask M with an opening AP formed therein, at least a part of the subject's eyes (i.e., all or part of them) are exposed to the opening AP (i.e., surrounded by the opening AP).

[0035] Considering the state in which the subject's eyes are surrounded by the opening AP, the term "eye" in this embodiment may refer to a part of the subject that includes not only the eyeball but also the appendages associated with the eyeball (in particular, appendages that are visible from the outside, such as the eyelids including the upper and lower eyelids mentioned above). Therefore, the state in this embodiment that "at least a part of the subject's eyes is surrounded by the opening AP" may include a state in which at least a part of the subject's eyeball is surrounded by the opening AP, or a state in which at least a part of the appendages associated with the subject's eyeball is surrounded by the opening AP. However, the term "eye" in this embodiment may refer to a part of the subject that includes the eyeball but does not include the appendages associated with the eyeball.

[0036] On the other hand, Figure 6 shows an example of a person image (IMG) in which a subject is not wearing a mask M with an aperture AP formed on it, along with a magnified view of the eye image (IMG_EYE) within the person image (IMG) which includes the subject's eye region (EA). As shown in Figure 6, when a subject is not wearing a mask M with an aperture AP formed on it, at least a portion of the subject's eyes will not be exposed from the aperture AP in the person image (IMG). In other words, at least a portion of the subject's eyes will not be surrounded by the aperture AP in the person image (for example, not surrounded by the outer edge (AP_edge) of the aperture AP).

[0037] In this embodiment, the impersonation detection unit 212 determines whether a subject is attempting to impersonate another person based on the difference between a person image IMG in which a subject wearing such a mask M is captured and a person image IMG in which a subject not wearing the mask M is captured. Specifically, the impersonation detection unit 212 determines in step S12 whether at least a part of the subject's eyes are surrounded by an aperture AP in the person image IMG acquired in step S11 (for example, whether they are surrounded by the outer edge AP_edge of the aperture AP). In other words, the impersonation detection unit 212 determines in step S12 whether an aperture AP surrounding at least a part of the subject's eyes exists in the person image IMG acquired in step S11 (for example, whether the outer edge AP_edge of the aperture AP surrounding at least a part of the subject's eyes exists). In other words, the impersonation detection unit 212 determines whether or not an object (in this embodiment, a mask M) with an opening AP surrounding at least a part of the subject's eyes exists in the person image IMG acquired in step S11 (step S12).

[0038] Furthermore, as described above, if at least a portion of the subject's eye is surrounded by the aperture AP, then at least a portion of the subject's eye will be exposed from the aperture AP. For this reason, the operation in step S12 to determine whether or not at least a portion of the subject's eye is surrounded by the aperture AP can be considered substantially equivalent to the operation to determine whether or not at least a portion of the subject's eye is exposed from the aperture AP. The operation in step S12 to determine whether or not at least a portion of the subject's eye is surrounded by the aperture AP can be considered substantially equivalent to the operation to determine whether or not there is an aperture AP that exposes at least a portion of the subject's eye. The operation in step S12 to determine whether or not at least a portion of the subject's eye is surrounded by the aperture AP can be considered substantially equivalent to the operation to determine whether or not there is an object (in this embodiment, a mask M) on which an aperture AP that exposes at least a portion of the subject's eye is formed.

[0039] The impersonation detection unit 212 may determine whether at least a portion of the subject's eyes is surrounded by an aperture AP by performing image analysis processing to analyze at least a portion of the person image IMG (for example, the eye image IMG_EYE described above). For example, the impersonation detection unit 212 may detect the subject's eyes from the person image IMG and determine whether or not there is an aperture AP surrounding the subject's eyes. In this case, the impersonation detection unit 212 may determine whether or not an aperture AP exists in the person image IMG by performing template matching using a template corresponding to the aperture AP.

[0040] The impersonation detection unit 212 may perform edge extraction processing on at least a portion of the person image IMG and determine whether or not there is an edge surrounding the subject's eye (i.e., the contour of any object surrounding the subject's eye, in this case the contour of the mask M that forms the outer edge AP_edge of the aperture AP), thereby determining whether or not at least a portion of the subject's eye is surrounded by the aperture AP. Specifically, the impersonation detection unit 212 may generate an edge image representing the edges within at least a portion of the person image IMG by performing edge extraction processing on at least a portion of the person image IMG (for example, the eye image IMG_EYE described above). For example, the impersonation detection unit 212 may generate an edge image by performing filtering processing on the person image IMG using a first-order differential filter. The impersonation detection unit 212 may perform filtering processing using at least one of a vertical first-order differential filter and a horizontal first-order differential filter. Within the edge image, edges corresponding to the subject's eye usually appear. The edge corresponding to the subject's eye may include at least one of the edges corresponding to the lower edge of the upper eyelid ul_edge and the upper edge of the lower eyelid ul_edge. Furthermore, if the subject is wearing a mask M, an edge corresponding to the aperture AP appears in the edge image. The edge corresponding to the aperture AP may include the edge corresponding to the outer edge AP_edge of the aperture AP (i.e., a portion of the edge that forms the outer edge AP_edge of the aperture AP in the mask M, and typically an edge that forms a closed loop). For this reason, the impersonation detection unit 212 may determine whether or not there is an edge corresponding to the aperture AP surrounding the edge corresponding to the subject's eye in the edge image. If there is an edge corresponding to the aperture AP surrounding the edge corresponding to the subject's eye, the impersonation detection unit 212 may determine that at least a portion of the subject's eye is surrounded by the aperture AP.

[0041] The impersonation detection unit 212 may use a learnable computational model to determine whether or not at least a portion of the subject's eyes are surrounded by an aperture AP. For example, the impersonation detection unit 212 may use a computational model that outputs a determination result of whether or not at least a portion of the subject's eyes are surrounded by an aperture AP when at least a portion of a person image IMG is input. For example, the impersonation detection unit 212 may use a computational model that outputs a determination result of whether or not at least a portion of the subject's eyes are surrounded by an aperture AP when an edge image generated by performing edge extraction processing on at least a portion of a person image IMG is input.

[0042] An example of a learnable computational model is a computational model using a neural network. The impersonation detection unit 212 may use a computational model trained by machine learning based on at least one of supervised learning and unsupervised learning to determine whether at least a portion of the target person's eyes are surrounded by the aperture AP.

[0043] If the determination in step S12 determines that at least a portion of the subject's eyes are surrounded by an aperture AP (step S12: Yes), then it is assumed that the subject captured in the person image IMG is likely wearing a mask M with an aperture AP formed on it. In this case, the impersonation detection unit 212 determines that the subject is impersonating another person (step S13).

[0044] On the other hand, if the determination in step S12 determines that at least a portion of the subject's eyes are not surrounded by the aperture AP (step S12: No), then it is assumed that the subject captured in the person image IMG is unlikely to be wearing a mask M with an aperture AP formed on it. In this case, the impersonation determination unit 212 determines that the subject is not impersonating another person (step S14).

[0045] If the impersonation detection unit 212 determines that the subject is not impersonating another person, the information processing device 2 may authenticate the subject using the person image IMG. For example, if the subject's face is captured in the person image IMG, the information processing device 2 may authenticate the subject by performing facial recognition using the subject's face captured in the person image IMG. Alternatively, if the person image IMG includes the subject's eye region EA, the information processing device 2 may authenticate the subject by performing iris recognition using the subject's iris captured in the person image IMG. Furthermore, the information processing device 2 may authenticate the subject by performing any authentication method different from facial recognition and iris recognition.

[0046] (3) Technical effects of the information processing device 2 As explained above, the information processing device 2 determines that a person is impersonating another person if, in the person image IMG, at least a portion of the subject's eyes is surrounded by an aperture AP (i.e., there is an aperture AP surrounding at least a portion of the subject's eyes). Therefore, the information processing device 2 can appropriately determine that a person is impersonating another person when the subject is wearing a mask M in which an aperture AP is formed in the portion corresponding to the eyes of another person.

[0047] As described above, the information processing device 2 may use an edge image generated by performing edge extraction on at least a portion of the person image IMG in order to determine whether at least a portion of the subject's eyes in the person image IMG is surrounded by an aperture AP. In other words, the information processing device 2 may use an edge image in which the contours of objects present in the person image IMG (for example, the subject's eyes and the mask M on which the aperture AP is formed) are emphasized to determine whether at least a portion of the subject's eyes is surrounded by an aperture AP. In this case, the information processing device 2 can determine with greater accuracy whether at least a portion of the subject's eyes is surrounded by an aperture AP compared to when an edge image is not used. As a result, the information processing device 2 can determine with greater accuracy whether the subject is impersonating another person.

[0048] (4) Variations (4-1) First variation When the subject is wearing a mask M, as shown in Figure 5, at least a portion of the lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the lower eyelid are located away from the outer edge AP_edge of the opening AP in the person image IMG. This is because, under normal circumstances, the size of the opening AP formed in the mask M is unlikely to match the size of the eyes of the subject wearing the mask M. In this way, when at least a portion of the lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the lower eyelid are located away from the outer edge AP_edge of the opening AP in the person image IMG, the impersonation detection unit 212 can easily determine that at least a portion of the subject's eyes are surrounded by the opening AP in the person image IMG.

[0049] On the other hand, it cannot be said that there is zero possibility that the size of the opening AP formed in the mask M matches the size of the eyes of the person wearing the mask M. If the size of the opening AP formed in the mask M matches the size of the eyes of the person wearing the mask M, then in the person image IMG, the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid may overlap with the outer edge AP_edge of the opening AP. In this case, the impersonation detection unit 212 may determine that at least a part of the person's eyes are not surrounded by the opening AP in the person image IMG. This is because the outer edge AP_edge of the opening AP, where the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid overlap, may be mistakenly recognized as the outline of the eye (i.e., the lower edge ul_edge of the upper eyelid and the upper edge ul_edge of the lower eyelid). As a result, the accuracy of determining whether or not the person is impersonating another person may decrease.

[0050] Therefore, in order to accurately determine whether a subject is impersonating another person even when the subject is wearing a mask M with an opening AP formed thereon that matches the size of the subject's eyes, the image acquisition unit 211 may acquire a person image IMG in step S11 of Figure 4 in which the subject has their eyes closed. An example of a person image IMG in which a subject wearing a mask M with an opening AP formed thereon and with their eyes closed is shown in Figure 7. As shown in Figure 7, when the subject has their eyes closed, even if the size of the opening AP formed in the mask M matches the size of the subject's eyes wearing the mask M, it is unlikely that the lower edge ul_edge of the subject's upper eyelid and the upper edge ul_edge of the lower eyelid will overlap with the outer edge AP_edge of the opening AP in the person image IMG. For this reason, the impersonation determination unit 212 can determine that at least a part of the subject's eyes are surrounded by the opening AP in the person image IMG when the subject is wearing a mask M with an opening AP formed thereon that matches the size of the subject's eyes. As a result, the impersonation detection unit 212 can accurately determine whether a person is impersonating another person, even when the person is wearing a mask M with an opening AP having a size that matches the size of the person's eyes.

[0051] Furthermore, the image acquisition unit 211 may acquire a person image IMG in step S11 of Figure 4 that includes a subject with their eyes closed, even if the subject is not wearing a mask M with an opening AP having a size that matches the size of the subject's eyes.

[0052] In order to obtain a person image (IMG) in which the subject has their eyes closed, the information processing device 2 may instruct the subject to close their eyes. For example, if the information processing device 2 has a display near the camera 1, the information processing device 2 may display a GUI (Graphical User Interface) to instruct the subject to close their eyes.

[0053] Alternatively, camera 1 may continuously photograph the subject over time. In other words, camera 1 may generate multiple person images (IMG) as time-series data. In this case, the impersonation detection unit 212 may extract at least one person image (IMG) from the multiple person images (IMG) that includes a subject with their eyes closed, and determine whether at least a portion of the subject's eyes are surrounded by an aperture (AP) in the extracted person image (IMG).

[0054] Furthermore, the information processing device 2 may, in parallel with the impersonation detection operation shown in Figure 4, perform an impersonation detection operation similar to that of the information processing device in the comparative example described above, by determining whether or not the subject captured in the person image IMG is blinking, thereby determining whether or not the subject is impersonating another person. In this case, in order to determine whether or not the subject is blinking, the camera 1 usually photographs the subject in a series of timeframes. That is, the camera 1 generates multiple person image IMGs as time-series data. Even in this case, the impersonation detection unit 212 may extract at least one person image IMG from the multiple person image IMGs that captures a subject with their eyes closed, and determine whether or not at least a part of the subject's eyes are surrounded by an aperture AP in the extracted at least one person image IMG. As a result, the information processing device 2 can perform the impersonation detection operation shown in Figure 4 and the impersonation detection operation using blinking in parallel.

[0055] (4-2) Second variation In step S11 of Figure 4, the image acquisition unit 211 may acquire both a person image IMG in which the subject has their eyes closed and a person image IMG in which the subject has their eyes open. In this case, the impersonation detection unit 212 may use the person image IMG in which the subject has their eyes closed and the person image IMG in which the subject has their eyes open to determine whether or not the subject is impersonating another person.

[0056] For example, the impersonation detection unit 212 may determine whether a person is impersonating another person based on the difference between a person image IMG in which the person has their eyes closed and a person image IMG in which the person has their eyes open. Specifically, if the person is wearing a mask M, a part of the person that was visible in the person image IMG when the person has their eyes closed may not be visible in the person image IMG when the person has their eyes open. For example, when the person has their eyes closed, the person's upper eyelids are visible in the person image IMG, while when the person has their eyes open, the person's upper eyelids may be hidden by the mask M. As a result, when the person has their eyes open, the person's upper eyelids may not be visible in the person image IMG. In this case, there is a difference regarding the presence or absence of the upper eyelids between the person image IMG in which the person has their eyes closed and the person image IMG in which the person has their eyes open. Therefore, the impersonation detection unit 212 can determine whether a subject is impersonating another person based on the difference between a person image IMG in which the subject has their eyes closed and a person image IMG in which the subject has their eyes open. Specifically, for example, the impersonation detection unit 212 may determine whether a part of the subject that was visible in the person image IMG when the subject had their eyes closed is no longer visible in the person image IMG when the subject had their eyes open. If a part of the subject that was visible in the person image IMG when the subject had their eyes closed is no longer visible in the person image IMG when the subject had their eyes open, the impersonation detection unit 212 may determine that the subject is impersonating another person.

[0057] Even in this case, the reason for the difference between a person image (IMG) in which the subject has their eyes closed and a person image (IMG) in which the subject has their eyes open remains the same: the aperture AP surrounding the subject's eyes (specifically, the mask M on which the aperture AP is formed). In other words, the reason why a part of the subject that is visible in the person image (IMG) when the subject has their eyes closed disappears from the person image (IMG) when the subject has their eyes open remains the same: the aperture AP surrounding the subject's eyes (specifically, the mask M on which the aperture AP is formed). For this reason, the operation of determining whether a part of the subject that is visible in the person image (IMG) when the subject has their eyes closed disappears from the person image (IMG) when the subject has their eyes open can be considered to be substantially equivalent to the operation of determining whether at least a part of the subject's eyes in the person image (IMG) is surrounded by the aperture AP.

[0058] (5) Note The following additional information is disclosed regarding the embodiments described above. [Note 1] A means for acquiring a person image including the eye region of the subject, A determination means for determining that the subject is impersonating another person if, in the aforementioned person image, there is an opening surrounding at least a part of the subject's eye included in the eye region. An information processing device equipped with the following features. [Note 2] The acquisition means acquires the person image, including the eye region of the subject with their eyes closed. The information processing device described in Appendix 1. [Note 3] The determination means (i) performs edge extraction processing on at least a portion of the person image, and (ii) determines that if edges surrounding at least a portion of the subject's eyes included in the eye region are extracted, there is an opening surrounding at least a portion of the subject's eyes included in the eye region. The information processing device described in Appendix 1 or 2. [Note 4] The acquisition means acquires a first image which is a person image including the eye region of the subject with their eyes closed, and a second image which is a person image including the eye region of the subject with their eyes open. The determination means determines whether or not the subject is impersonating another person based on the difference between the first image and the second image. An information processing device as described in any one of the items 1 to 3 of the appendix. [Note 5] We obtain a person image that includes the eye area of ​​the subject. If, in the aforementioned person image, there is an opening surrounding at least a portion of the subject's eye included in the eye region, it is determined that the subject is impersonating another person. Information processing methods. [Note 6] On the computer, When a person image including the eye area of ​​the subject is obtained, If, in the aforementioned person image, there is an object in which an opening is formed that exposes at least a portion of the subject's eye within the eye region, it is determined that the subject is impersonating another person. A recording medium on which a computer program that executes an information processing method is stored.

[0059] At least some of the constituent elements of each embodiment described above can be appropriately combined with at least some other constituent elements of each embodiment described above. Some of the constituent elements of each embodiment described above may not be used. Furthermore, to the extent permitted by law, all of the disclosures of the documents cited in this disclosure above (e.g., published gazettes) shall be incorporated as part of the description of this disclosure.

[0060] This disclosure may be modified from time to time, insofar as it does not contradict the technical idea that can be inferred from the claims and the entire specification. Information processing devices, information processing methods, computer programs and recording media that include such modifications are also included in the technical idea of ​​this disclosure. [Explanation of Symbols]

[0061] 1 Camera 2. Information Processing Device 21 Arithmetic unit 211 Image acquisition unit 212 Impersonation detection unit 3. Communication Network SYS impersonation detection system M Mask AP aperture IMG People Images

Claims

1. We obtain a person image that includes the eye area of ​​the subject. In the aforementioned image of a person, it is determined that the person is impersonating another person based on the outline of an accessory located outside at least a portion of the person's eye in the horizontal or vertical direction, when the person's eyes are closed. The information processing method performed by computers.

2. If the outline surrounds at least a portion of the subject's closed eye, it is determined that the subject is impersonating another person. The information processing method performed by the computer described in claim 1.

3. The accessory is a mask, and the contour is the contour of an opening formed in the mask. The information processing method performed by the computer according to claim 1 or 2.

4. A means for acquiring a person image including the eye region of the subject, In the aforementioned image of a person, a determination means determines that the subject is impersonating another person based on the outline of an accessory located outside the subject's eyes in the horizontal or vertical direction, at least a portion of the subject's eyes, when the subject has their eyes closed. An information processing device equipped with the following features.

5. The determination means determines that the subject is impersonating another person if the outline surrounds at least a portion of the subject's closed eye. The information processing apparatus according to claim 4.

6. The accessory is a mask, and the contour is the contour of an opening formed in the mask. The information processing apparatus according to claim 4 or 5.

7. A means for acquiring a person image including the eye region of the subject, In the aforementioned image of a person, a determination means determines that the subject is impersonating another person based on the outline of an accessory located outside the subject's eyes in the horizontal or vertical direction, at least a portion of the subject's eyes, when the subject has their eyes closed. An information processing system equipped with the following features.

8. The determination means determines that the subject is impersonating another person if the outline surrounds at least a portion of the subject's closed eye. The information processing system according to claim 7.

9. The accessory is a mask, and the contour is the contour of an opening formed in the mask. The information processing system according to claim 7 or 8.

10. On the computer, We obtain images of people that include the subject, In the aforementioned image of a person, it is determined that the person is impersonating another person based on the outline of an accessory located outside at least a portion of the person's eye in the horizontal or vertical direction, when the person's eyes are closed. A computer program that executes information processing methods.

11. On the computer, If the aforementioned contour surrounds at least a portion of the subject's closed eye, the system will execute an information processing method that determines that the subject is impersonating another person. The computer program according to claim 10.

12. The accessory is a mask, and the contour is the contour of an opening formed in the mask. The computer program according to claim 10 or 11.

Images