Information processing apparatus, information processing method, and computer program

Abstract

To provide an information processing apparatus, an information processing method, and a program capable of reducing the possibility of erroneous determinations related to impersonation.SOLUTION: An information processing apparatus 100 functions as: an authentication image acquisition unit 102b that acquires an image including a person to be authenticated; an authentication contribution region decision unit 103b that specifies a contribution region in the image used for authentication of the person; an impersonation determination region decision unit 104b that decides a determination region used for impersonation determination processing on the basis of the contribution region specified by the authentication contribution region decision unit 103b; and a first impersonation determination unit 105b that executes impersonation determination processing for determining impersonation on the basis of the determination region determined by the impersonation determination region decision unit 104b.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a computer program, etc. [Background technology]

[0002] In the past, there have been attacks on biometric authentication in which an artificial object imitating a living body is presented to attempt fraudulent authentication. For example, in the case of facial recognition, an attacker could present a printed photograph of a face to impersonate another person and attempt authentication.

[0003] One technique for detecting such attacks is spoofing detection technology. In Patent Document 1, whether an input face image is a biological image is determined to detect whether it is a spoofed image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6932150 [Non-patent literature]

[0005] [Non-Patent Document 1] Jonathan R Williford, Brandon B May, and Jeffrey Byrne.Explainable face recognition. In ECCV, pages 248-263.Springer, 2020. [Non-patent document 2] A. George and S. Marcel, ”On the Effectiveness of Vision Transformers for Zero-shot Face Anti-Spoofing,” 2021 IEEE International Joint Conference on Biometrics (IJCB), Shenzhen, China, 2021, pp. 1-8 [Non-Patent Document 3] C. -Y. Wang, Y. -D. Lu, S. -T. Yang and S. -H. Lai, ”PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognition,” 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 20249-20258 [Non-Patent Document 4] Z. Yu, J. Wan, Y. Qin, X. Li, S. Z. Li and G. Zhao, ”NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 9, pp. 3005-3023, 1 Sept. 2021 [Non-Patent Document 5] RR Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh and D. Batra, “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,” 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 618-626 Summary of the Invention [Problem to be solved by the invention]

[0006] However, there is a problem in that accidental occlusion without any intention of impersonation can lead to a false determination of impersonation. For example, even if a person wears a mask, sunglasses, face paint, or a fake beard without any intention of impersonation, the facial area is occluded and the area where the skin is visible is reduced, leading to a false determination of impersonation.

[0007] Therefore, one of the objects of the present invention is to reduce the possibility of erroneous determination regarding impersonation. [Means for solving the problem]

[0008] In the information processing device, an acquisition means for acquiring an image including a person to be authenticated; means for identifying a contributing region within the image for use in authenticating the person; a determination means for determining a determination area to be used in the spoofing determination process based on the contributing area identified by the identification means; a spoofing determination means for executing a spoofing determination process for determining spoofing based on the determination area determined by the determination means; The present invention is characterized by having the following. [Effects of the Invention]

[0009] According to the present invention, the possibility of erroneous determination regarding impersonation can be reduced. [Brief explanation of the drawings]

[0010] [Figure 1] 1A is a block diagram showing an example of the hardware configuration of information processing devices 100 and 300, and FIG. 1B is a block diagram showing an example of the functional configuration of the information processing device 100. FIG. [Figure 2] 10A to 10D are flowcharts illustrating the operation of the information processing device 100. [Figure 3] 1A is a block diagram showing an example of the functional configuration of an information processing device 300. FIGS. 1B and 1C are flowcharts illustrating the operation of the information processing device 300. [Figure 4] 10A to 10D are flowcharts showing the flow of display processing and learning processing by information processing device 300. [Figure 5] 10A to 10D are diagrams showing examples of data formats and displays stored by the information processing device 300. FIG. [Figure 6] 10 is a flowchart showing the flow of an impersonation determination process. [Figure 7] 10(A) to 10(E) are explanatory diagrams of examples of creating contributing regions. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions will be omitted or simplified.

[0012] <Embodiment 1> An example of the hardware configuration of an information processing apparatus according to this embodiment will be described with reference to the block diagram of Fig. 1(A). Fig. 1(A) is a block diagram showing an example of the hardware configuration of an information processing apparatus 100 according to the first embodiment.

[0013] Reference numeral 101a denotes a CPU (Central Processing Unit) that controls the entire information processing device 100, and 102a denotes a ROM (Read Only Memory) that stores programs and parameters that do not require modification.

[0014] Reference numeral 103a denotes a RAM (Random Access Memory) that temporarily stores programs and data supplied from an external device, and reference numeral 104a denotes an external storage device such as a hard disk or memory card that is fixedly installed in the information processing device 100.

[0015] The external storage device 104a may include an optical disk such as a compact disk (CD) that is detachable from the information processing device 100, a magnetic or optical card, an IC card, a memory card, or the like.

[0016] Reference numeral 105a denotes an input device interface connected to an input device 109a, such as a pointing device, mouse, or keyboard, which receives user operations and inputs data, and 106a denotes an output device interface connected to a monitor 110a for displaying data held by the information processing device 100 or data supplied thereto.

[0017] Reference numeral 107a denotes a communication interface for connecting to a network line 111a such as the Internet, and is connected to an NW (network) camera 112a via the network line 111a. The NW camera 112a is an imaging device that captures video. A system bus 108a is a transmission path that connects the units 101a to 107a so that they can communicate with each other.

[0018] Next, the functions of the information processing apparatus of this embodiment will be described with reference to the block diagram of Fig. 1(B) Fig. 1(B) is a functional block diagram showing an example of the configuration of an information processing apparatus 100 according to the first embodiment.

[0019] Note that some or all of the functional blocks shown in FIG. 1B are realized by causing a CPU 101a or the like serving as a computer included in the information processing device 100 to execute a computer program stored in a memory serving as a storage medium.

[0020] However, some or all of these functions may be implemented by hardware separate from the CPU 101a, such as a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP).

[0021] Furthermore, the functional blocks shown in Fig. 1(B) do not have to be built into the same housing, but may be configured as separate devices connected to each other via signal paths. The above explanation regarding Fig. 1 also applies to Fig. 3(A).

[0022] The information processing device 100 of this embodiment performs impersonation determination on face images. The registered image acquisition unit 101b acquires a plurality of face images registered in advance. These images are called registered images. The registered images are stored in advance in the external storage device 104a. The registered image acquisition unit 101b functions as registered image acquisition means for acquiring registered images of a person to be authenticated.

[0023] Each registered image may be associated with a person ID for identifying the person and stored. Other information, such as the person's name, may also be associated and stored. The methods for saving and acquiring registered images are not limited to these.

[0024] The authentication image acquisition unit 102b acquires an image such as a facial image including the person to be authenticated. This image is called an authentication image. The authentication image may be acquired via a NW (network) camera 112a or the like. Alternatively, a facial image previously stored in the external storage device 104a may be used. Note that the method for acquiring the authentication image is not limited to these. Here, the authentication image acquisition unit 102b functions as an acquisition means for acquiring an image including the person to be authenticated.

[0025] The authentication contributing area determination unit 103b determines which area of the authentication image acquired by the authentication image acquisition unit 102b is an area that contributes to face authentication (contributing area). The authentication contributing area determination unit 103b functions as a specifying unit that specifies a contributing area in the image used for person authentication. In this embodiment, the contributing area is determined by combining two main methods.

[0026] The first method is to remove areas that do not contribute to face recognition based on heuristics (rules of thumb) or machine learning, etc. Areas of the face that are occluded by masks such as surgical masks or sunglasses are areas that do not contribute to face recognition. Therefore, this method considers areas other than these as contributing areas.

[0027] The second method is to find the grounds area that determines the similarity between the registered image and the authentication image, i.e., to determine the contributing area based on the area that contributes to determining the similarity between the authentication image acquired by the authentication image acquisition unit 102b and the registered image acquired by the registered image acquisition unit 101b.

[0028] Specifically, one method of facial recognition is to convert an image into a feature vector using a neural network or the like, and determine whether the same person appears in the two images based on the similarity between the feature vectors of the registered image and the authentication image. In such cases, an area that determines the similarity between the two feature vectors may be calculated and used as the basis area.

[0029] For example, Non-Patent Document 1 (Jonathan R Williford, Brandon B May, and Jeffrey Byrne. Explainable face recognition. In ECCV, pages 248-263. Springer, 2020.) describes a method of adding noise to an authentication image to determine a ground truth region.

[0030] The method described in Non-Patent Document 1 may also be used. Note that the method for calculating the ground region is not limited to these. In addition, in this embodiment, an example will be mainly described in which the regions calculated by the above-mentioned two methods are integrated to obtain the final contributing region, but a detailed processing example will be described later using FIG. 2(B) etc.

[0031] Alternatively, a face image of a natural face without any decorations such as a mask may be acquired from the registration image acquisition unit 101b. When obtaining a grounds area for determining the similarity between a registration image and an authentication image, if the registration image includes an attachment such as a mask, the attachment will contribute to determining the similarity.

[0032] Therefore, it is desirable to use an image of a person's face as the registration image. That is, it is desirable for the registration image acquisition unit 101b to acquire an image of a person not wearing any covering as the registration image. This prevents elements other than the face, such as clothing, from being included in the grounds area.

[0033] The spoofing detection area determination unit 104b generates a spoofing detection area to be used for spoofing detection based on the contributing area obtained by the authentication contributing area determination unit 103b. Here, the spoofing detection area determination unit 104b functions as a determination means for determining a detection area to be used for spoofing detection processing based on the contributing area identified by the identification means.

[0034] Specifically, the spoofing determination area determining unit 104b adds the spoofing characteristic area to the spoofing determination area, and excludes from the spoofing determination area any area that may induce an erroneous spoofing determination.

[0035] That is, the spoofing determination area determination unit 104b determines the determination area by adding an spoofing characteristic area having spoofing characteristics to the contributing area. Also, the spoofing determination area determination unit 104b determines the determination area by excluding an erroneous spoofing determination area that satisfies a predetermined condition that induces an erroneous spoofing determination from the contributing area.

[0036] A spoofing feature region is a region that shows the likelihood of spoofing. When a printed photograph of a person is held up to a camera, light is reflected in the photograph. On the other hand, regions that are blown out due to reflections are unlikely to become evidence regions in face recognition, and are unlikely to remain as contributing regions.

[0037] Therefore, by adding a spoofing feature region to the spoofing determination region, it becomes easier to detect spoofing. Furthermore, the spoofing feature region may include, in addition to a reflective region, an area where wrinkles, peeling, dirt, etc., of paper or the like are present. In other words, the spoofing feature region includes at least one of reflection, wrinkles, peeling, and dirt of the image of the person to be authenticated. Note that the spoofing feature region is preferably an area that represents an artificial appearance that does not appear on a living body.

[0038] Note that reflections may occur due to glasses or the like, and such reflections are often not spoofing. Therefore, reflection areas other than the eyes may be used as spoofing feature areas. However, spoofing feature areas are not limited to these.

[0039] Furthermore, the spoofing erroneous judgment area that induces the spoofing erroneous judgment also includes areas with poor conditions such as blurred areas and brightness. In the case of face recognition, highly robust recognition is possible even in areas with poor conditions such as blurred areas and brightness, but spoofing judgment is less robust than face recognition because it judges whether an object is a living body or an artificial object based on minute changes in the image.

[0040] Therefore, even if an area has poor conditions for spoofing detection, it may still contribute to face authentication. Therefore, in this embodiment, the spoofing detection error area is excluded from the spoofing detection area, thereby reducing the number of spoofing detection errors.

[0041] That is, the spoofing erroneous determination area is excluded from the spoofing determination area based on the blur or brightness of the image of the person to be authenticated. Note that the spoofing erroneous determination area that induces the erroneous determination is not limited to these.

[0042] The first spoofing determination unit 105b determines whether the authentication image obtained by the authentication image acquisition unit 102b is a spoofed image or not, using the spoofing determination area determined by the spoofing determination area determination unit 104b. Specifically, there are several methods.

[0043] For example, a method for calculating the spoofing score of an image is described in Non-Patent Document 2 (A. George and S. Marcel, "On the Effectiveness of Vision Transformers for Zero-shot Face Anti-Spoofing," 2021 IEEE International Joint Conference on Biometrics (IJCB), Shenzhen, China, 2021, pp. 1-8).

[0044] Non-Patent Document 2 describes a method in which a neural network is trained in advance so that it can classify spoofed images from non-spoofed images, and the trained neural network is used to determine the spoofing score of an image.

[0045] In such a "method for calculating an image spoofing score," one possible method for using the spoofing determination area is to black out the authentication image outside the spoofing determination area and input it into a neural network.

[0046] Alternatively, when training the neural network, both the image and the spoofing detection area are given as inputs to the neural network for training, and then both the spoofing detection area and the authentication image are input to the neural network to calculate the spoofing score.

[0047] On the other hand, examples of "methods for calculating spoofing scores for each region of an image" include Non-Patent Document 3 (C. -Y. Wang, Y. -D. Lu, S. -T. Yang and S. -H. Lai, "PatchNet: A Simple Face Anti-Spoofing Framework via Fine-Grained Patch Recognition," 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 20249-20258).

[0048] In this non-patent document 3, an input image is divided into patches, and a spoofing score is calculated for each patch.

[0049] In addition, there is also a method for calculating an spoofing score for each region of an image, such as Non-Patent Document 4 (Z. Yu, J. Wan, Y. Qin, X. Li, SZ Li and G. Zhao, "NAS-FAS: Static-Dynamic Central Difference Network Search for Face Anti-Spoofing," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 9, pp. 3005-3023, 1 Sept. 2021).

[0050] Non-Patent Document 4 describes a method for calculating a spoofing score on a pixel-by-pixel basis, although at a lower resolution than the input image. In such a "method for calculating a spoofing score for each region of an image," a method using a spoofing judgment region can be considered in which the spoofing score of an image is determined using the spoofing scores of regions in the spoofing judgment region.

[0051] For example, it is possible to calculate statistics such as the average or maximum value of the spoofing score within the spoofing determination region. Alternatively, when dividing the input image into patches, it is possible to arrange the patches so that many of the patches include the spoofing determination region.

[0052] The statistics of the spoofing scores obtained for each patch may then be calculated as the spoofing score for the image.However, the method of spoofing determination using the spoofing determination area is not limited to these.

[0053] Next, the spoofing determination process of this embodiment will be described with reference to Figures 2(A) to 2(D). Figures 2(A) to 2(D) are flowcharts showing an example of a spoofing determination process method in the information processing method according to this embodiment.

[0054] The CPU 101a, which serves as a computer within the information processing device, executes a computer program stored in memory, thereby sequentially performing the operations of the steps in the flowcharts of FIGS. 2(A) to 2(D).

[0055] In step S201a, the authentication image acquisition unit 102b acquires an authentication image. Step S201a functions as an acquisition step for acquiring an image including a person to be authenticated.

[0056] In step S202a, a calculation process of a contributing area for face authentication is performed on the authentication image obtained in step S201a. Here, step S202a functions as a specifying step for specifying a contributing area in the image used for person authentication. A specific processing example in step S202a will be described with reference to FIG. 2(B). The processing in FIG. 2(B) is performed by the authentication contributing area determination unit 103b.

[0057] In step S201b of FIG. 2(B), a mask region is extracted from the authentication image. In step S202b, a sunglasses region is extracted from the authentication image. To extract the mask region and sunglasses region, a neural network trained to segment the mask region and sunglasses region is used to extract the region. However, the method for extracting the mask region and sunglasses region is not limited to these.

[0058] In step S203b, face authentication of the authentication image is performed. Specifically, registered images are acquired from registered image acquisition unit 101b, and the similarity between each registered image and the authentication image is calculated. In step S204b, the top N registered images with the highest similarity and their similarities are identified using the similarity for each registered image acquired in step S203b.

[0059] Alternatively, when sorting by similarity, if the difference between the similarities before and after the image becomes a predetermined value or more, the registered image before the difference appears may be used. Alternatively, only the highest similarity may be used. Note that the method of selecting the highest similarity is not limited to these.

[0060] Step S205b is the start of a loop for searching the top images. The registered images obtained in step S204b are assumed to be assigned numbers in order starting from 1. To refer to these numbers using the variable i, i is first initialized to 1. If i is equal to or less than a predetermined number of top registered images, the process proceeds to step S206b. If i is greater than the predetermined number of top registered images, the process exits step S207b, which is the end of the loop, and proceeds to step S208b.

[0061] In step S206b, the basis region that determines the similarity between the i-th registered image and the authentication image is calculated. The basis region can be calculated using the method described in the aforementioned Non-Patent Document 1. Step S207b is the end of the top search loop, where 1 is added to the variable i and the process returns to step S205b.

[0062] In step S208b, the mask region obtained in step S201b, the sunglasses region obtained in step S202b, and the ground region obtained for each registered image in step S206b are integrated. A specific processing example of step S208b will be described with reference to the flowchart in FIG. 2(D).

[0063] In this embodiment, the mask area, sunglasses area, and grounds area are binary mask images. A binary mask image is an image in which each pixel has a value of 0 or 1, and when it is in the relevant area, it has a value of 1.

[0064] In step S201d of Fig. 2(D), the evidence regions obtained in step S206b are integrated. Figs. 7(A) to 7(E) are explanatory diagrams of an example of creating a contributing region in an embodiment. For example, if two evidence regions 701a and 701b as shown in Figs. 7(A) and 7(B) are obtained in step S206b, a region is obtained by taking the logical sum of the two binary masks.

[0065] That is, a binary mask is created in which pixels having either one of the values 1 are set to 1. As a result, an integrated grounds region 701c shown in Fig. 7(C) is obtained.

[0066] In step S202d, a process is performed to exclude the mask region obtained in step S201b and the sunglasses region obtained in step S202b from the integrated ground region 701c. For example, assume that the mask region 701d shown in Figure 7(D) is obtained. Since sunglasses are not being worn, the sunglasses region is not obtained.

[0067] "No region was obtained" means that the binary mask of the sunglasses region was all zero. Therefore, here, mask region 701d is excluded from integrated ground region 701c. Specifically, when the binary mask indicating the mask region has a value of 1, processing is performed to set the pixels of the integrated ground region to zero. As a result, step S202d obtains contributing region 701e shown in FIG. 7(E).

[0068] When the process of step S202d is completed, the process of step S20b is completed, and step S202a in Fig. 2A is completed. Next, in step S203a, a spoofing determination area is created using the contributing area 701e obtained in step S202a, for example, as shown in Fig. 7E.

[0069] Here, step S203a functions as a determination step for determining a determination area to be used in the spoofing determination process based on the contributing area identified in step S202a, which serves as the identification step. Specific processing of step S203a will be described using the flowchart in Fig. 2(C). The processing in Fig. 2(C) is performed by spoofing determination area determination unit 104b.

[0070] In step S201c, spoof feature regions are detected. Specifically, spoof feature regions are extracted using a neural network or the like that has been trained to segment regions other than the eyes where reflections, wrinkles, peeling, dirt, etc. are present. However, the method for extracting spoof feature regions is not limited to this.

[0071] In step S202c, regions where spoofing has been erroneously determined are detected. Specifically, regions with strong blurring or regions with poor brightness conditions, such as regions that are too bright or too dark, are detected. A method for detecting regions with strong blurring may be, for example, to sequentially extract regions using window shift, and apply a Laplacian filter to determine that a region is strongly blurred if the variance of the Laplacian values within the region is equal to or less than a predetermined value.

[0072] Another method for detecting areas with poor brightness conditions is to sequentially extract areas by window shifting, and determine that an area has poor brightness conditions if the variance of the brightness histogram is small or the average is too high or too low, etc. However, the methods for extracting areas where spoofing is erroneously determined are not limited to these.

[0073] In step S203c, the contributing regions determined in step S202a are corrected and integrated using the spoofing characteristic regions determined in step S201c and the spoofing erroneous determination regions determined in step S202c. Specifically, the contributing regions are integrated by adding the spoofing characteristic regions to the contributing regions and removing the spoofing erroneous determination regions from the contributing regions.

[0074] When the process of step S203c is completed, the process of step S203a in Fig. 2A is completed. In step S204a, first spoofing determination unit 105b performs spoofing determination using the spoofing determination area obtained in step S203c. Here, first spoofing determination unit 105b functions as a first spoofing determination means that determines whether the image is an attack attempting fraudulent authentication based on the determination area.

[0075] Here, step S204a functions as a spoofing determination step for executing a spoofing determination process for determining spoofing based on the determination area determined in the determination step.

[0076] In this embodiment, the method of determining the contributing region by the authentication contributing region determining unit 103b is a combination of the heuristic method and the method of determining the grounds region.

[0077] That is, an example has been described in which a method of removing areas that do not contribute to face authentication from the contributing area using a heuristic method is combined with a method of determining the grounds area that determines the similarity between the registered image and the authentication image. However, a method using only one of the methods may be used. By focusing on only one of the methods, processing speed can be increased. The method of determining the contributing area is not limited to these.

[0078] In addition, in this embodiment, an example has been described in which multiple registered images are acquired from the registered image acquisition unit 101b. However, a configuration in which only one registered image is acquired from the registered image acquisition unit 101b may also be used. Alternatively, a configuration in which the registered image acquisition unit 101b is not provided may also be used.

[0079] In this case, the processing of steps S203b to S207b in Fig. 2B performed by the authentication contributing area determination unit 103b is omitted. Therefore, the area excluding the mask and sunglasses area can be used as the contributing area. In this case, the face area can be segmented, and then the area excluding the mask and sunglasses area from the face area can be used as the contributing area.

[0080] In this embodiment, the spoofing detection area determination unit 104b obtains the spoofing detection area by correcting the contributing area, but the contributing area may be used as the spoofing detection area as is.

[0081] In this embodiment, the mask area, sunglasses area, and ground area appearing in the process of Fig. 2B are described as binary masks. However, they may be images in which each pixel has a continuous value between 0 and 1.

[0082] In this case, threshold processing may be performed to convert the mask into a binary mask before performing the process shown in Fig. 2(D). Alternatively, in step S201d, an average may be calculated for each pixel of multiple evidence regions, and pixels exceeding a threshold may be used as the integrated evidence region.

[0083] In this embodiment, the contributing region has been described as a region without a value. However, each pixel in the contributing region may have a contribution score that indicates the degree of contribution. For example, the mask region, sunglasses region, and grounds region obtained in step S201b, step S202b, and step S206b may each have a contribution score value.

[0084] Then, in integrating the regions in step S208b, the regions may be integrated by adding or subtracting the values of each region. For example, the values of the multiple evidence regions obtained in step S206b are averaged for each pixel and then integrated into one evidence region.

[0085] Then, the values of the mask region and the sunglasses region may be subtracted from the integrated evidence region to obtain the contributing region. Note that weights may be assigned to the mask region and the sunglasses region before subtraction. Note that the region integration process is not limited to these.

[0086] Similarly, the spoofing determination area determined by the spoofing determination area determination unit 104b may also have a value. The spoofing characteristic area determined in step S201c and the spoofing erroneous determination area determined in step S202c may also be areas in which each pixel has a value.

[0087] In such a case, in the region integration process of step S203c, the spoofing characteristic region may be weighted and added to the contributing region, and the spoofing erroneous determination region may be weighted and subtracted from the contributing region. The region integration process is not limited to these.

[0088] Also, when the spoofing error judgment area has a value, if the first spoofing judgment unit 105b uses the ``method of calculating the spoofing score of an image,'' the input image may be created by blacking out pixels in the authentication image where the value of the spoofing judgment area is less than a predetermined value.

[0089] Alternatively, the brightness of the input image may be changed depending on the magnitude of the value, or a neural network may be trained in advance using the spoofing determination area having a value and the authentication image as input.

[0090] On the other hand, when using the "method of calculating the spoofing score for each region of the image," the spoofing score may be calculated by weighting the spoofing judgment score using the value of the spoofing judgment region as a weight and then calculating a statistical quantity such as an average to calculate the spoofing score of the image. The spoofing judgment method using the spoofing judgment region having a value is not limited to these.

[0091] In the above embodiment, when the first spoofing determination unit 105b obtains the result of the spoofing determination in step S204a, the determination result is displayed on the monitor 110a as an image display means to notify the user, or is recorded in the RAM 103a or the external storage device 104a.

[0092] The use of the spoofing determination result is not limited to these. Also, an spoofing score may be obtained as the spoofing determination result. Alternatively, the spoofing score may be subjected to threshold processing, and if it exceeds a predetermined threshold, it may be determined to be an "attack," and if it does not, it may be determined to be a "non-attack." The spoofing determination result is not limited to these.

[0093] By doing so, it is possible to determine whether someone is spoofing something by focusing on areas that affect authentication. If there is a spoofed printed matter or other material in the area that affects authentication, the authentication results will be fraudulently distorted. On the other hand, even if there is something obscuring the skin, as long as it is outside the area that affects authentication, the authentication results will not be distorted by the obscuring material.

[0094] Therefore, it is effective to detect spoofing by focusing on areas that affect authentication. In particular, it is common for people to cover their skin with masks, sunglasses, glasses, face paint, fake beards, etc., and this has the effect of reducing false positives due to such decorations.

[0095] In addition, the area used for spoofing detection is modified by the spoofing detection area determination unit 104b. That is, by adding obvious spoofing characteristic areas such as reflections, wrinkles, peeling, and dirt other than the eyes to the spoofing detection area, it is possible to prevent missed spoofing detections.

[0096] In addition, areas that are heavily blurred or areas that are too bright or too dark can lead to erroneous spoofing judgments, so excluding these erroneous spoofing judgment areas from the spoofing judgment area can have the effect of preventing erroneous spoofing judgments.

[0097] <Embodiment 2> In the second embodiment, after the spoofing determination, the determination result and images are saved and presented to the user. The saved data is also used to perform machine learning to detect "spoofing characteristic regions" and "spoofing erroneous determination regions" used in the spoofing determination region determination unit 104b.

[0098] The configuration of the information processing device 300 according to the second embodiment will be described with reference to the block diagram of Fig. 3(A). Fig. 3(A) is a functional block diagram showing an example of the configuration of the information processing device 300 according to the second embodiment. The difference from the first embodiment is that elements 306a to 311a have been added.

[0099] The second spoofing determination unit 306a performs spoofing determination using the entire face area. Specifically, it performs spoofing determination using the methods described in the above-mentioned Non-Patent Documents 2 to 4. While the first spoofing determination unit 105b uses the contributing area of face authentication, the second spoofing determination unit 306a is characterized in that it performs spoofing determination regardless of the contributing area.

[0100] The second spoofing determination unit 306a functions as a second spoofing determination means for determining whether the image is an image (attack) attempting fraudulent authentication based on the acquired image. Although the entire area is used in the second embodiment, it is not necessary to use the entire area. For example, it may be configured to exclude areas that may induce erroneous determination based on criteria such as blurring.

[0101] However, the configuration of second spoofing determination unit 306a is not limited to these. In the second embodiment, first spoofing determination unit 105b and second spoofing determination unit 306a obtain an spoofing determination result of "attack" or "non-attack."

[0102] The attack area creation unit 307a creates an attack area. The attack area is an area that indicates an attack object. For example, in the case of an authentication image in which an attacker holds up a photo with the mouth of another person cut out, data indicating the area of the photo held up to the mouth is set as the attack area. The attack area creation unit 307a functions as attack area creation means that creates an attack area based on the contributing area. Details of the processing of the attack area creation unit 307a will be described later using FIG. 4(A).

[0103] The non-attack area creation unit 308a creates a non-attack area. The non-attack area is an area that indicates an obstruction that has no offensive intent, such as a mask, sunglasses, glasses, face paint, or a false beard. The obstruction includes at least one of a mask, sunglasses, glasses, face paint, or a false beard.

[0104] The non-attack area creating unit 308a functions as a non-attack area creating unit that creates a non-attack area based on the contributing area. Details of the processing in the non-attack area creating unit 308a will be described later with reference to FIG. 4(B).

[0105] Image storage unit 309a stores the spoofing determination results. Specifically, it stores the first and second spoofing determination results, image data, attack area data, non-attack area data, etc. Here, image storage unit 309a functions as image storage means that stores images, the determination results of the first spoofing determination means, and the determination results of the second spoofing determination means.

[0106] In particular, the image storage unit 309a stores attack area data when the first spoofing judgment result is “attack” and the second spoofing judgment result is “non-attack.” This allows attack area data that may induce an erroneous judgment in the second spoofing judgment to be stored.

[0107] In addition, non-attack area data is saved when the first spoofing judgment result is "non-attack" and the second spoofing judgment result is "attack." This makes it possible to save non-attack area data that may induce an erroneous judgment in the second spoofing judgment.

[0108] The details of the stored contents will be described later with reference to Fig. 5(A). In this embodiment, these data are stored in the external storage device 104a, but they may be configured to be stored in the RAM 103a or the like.

[0109] The image display unit 310a performs processing to display the data stored in the image storage unit 309a. In this embodiment, data for which the first spoofing determination result is determined to be "attack" is displayed on the monitor 110a.

[0110] In particular, data that is judged as "attack" by the first spoofing judgment result but "non-attack" by the second spoofing judgment result is highlighted. This is because data that is judged as "attack" by both the first and second spoofing judgment results is often a normal attack that presents a photo of another person's face, etc.

[0111] On the other hand, data where the first result is "attack" and the second result is "non-attack" indicates a more sophisticated attack, such as holding up a photo of another person's mouth cut out over the attacker's mouth. Therefore, to warn the user of such sophisticated attacks, this data is displayed with more emphasis. Details of this process will be explained later using Figure 4(C).

[0112] The spoofing judgment area detection learning unit 311a performs machine learning to detect "spoofing characteristic areas" and "spoofing erroneous judgment areas" used in the spoofing judgment area determination unit 104b. Specifically, using the attack area data and non-attack area data stored in the image storage unit 309a, learning is performed so that these areas can be segmented (cut out) using a neural network or the like.

[0113] The spoofing determination region determination unit 104b then operates to detect "spoofing characteristic regions" and "spoofing erroneous determination regions" using these machine-learned detectors. Detailed processing will be described later with reference to FIG. 4(D).

[0114] FIG. 3B is a flowchart showing an example of the spoofing determination process according to the second embodiment, and FIG. 3C is a flowchart showing a detailed example of the process of step S305b.

[0115] The CPU 101a, which serves as a computer within the information processing device, executes a computer program stored in the memory, thereby sequentially performing the operations of the steps in the flowcharts of FIGS. 3B and 3C. The flow in Fig. 3(B) adds step S305b of saving data at the end to the flow of the spoofing determination process in Fig. 2(A) of embodiment 1. Details of step S305b will be described using Fig. 3(C).

[0116] In step S301c of FIG. 3C, second spoofing determination unit 306a performs second spoofing determination processing on the authentication image.

[0117] In step S302c, it is determined whether only the first spoofing determination result is "attack." That is, if the first spoofing determination result is "attack" and the second spoofing determination result is "non-attack," the determination is YES and the process proceeds to step S303c. Otherwise, the process proceeds to step S304c.

[0118] In step S303c, the attack area is created by the attack area creation unit 307a. Figures 4(A) to 4(D) are flowcharts showing the flow of the display process and learning process in embodiment 2, and Figure 4(A) is a flowchart showing the flow of the process in step S303c in embodiment 2.

[0119] The CPU 101a, which serves as a computer within the information processing device, executes a computer program stored in memory, thereby sequentially performing the operations of the steps in the flowcharts of FIGS. 4(A) to 4(D).

[0120] In step S401a, the contributing region created in step S202a is acquired. In step S402a, the grounds region of the determination result obtained by first spoofing determination unit 105b in step S203a is obtained, and a first spoofing determination grounds region is created.

[0121] For example, a general method for finding the ground truth region of a neural network is described in Non-Patent Document 5 (RR Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh and D. Batra, "Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 618-626).

[0122] In step S402a, a first spoofing determination basis region is created by thresholding the basis region obtained using GradCAM or the like described in Non-Patent Document 5. Note that the method for creating the first spoofing determination basis region is not limited to this.

[0123] In step S403a, the evidence region within the contributing region is obtained, thereby obtaining the evidence region that was the basis for the impersonation that affected the authentication.

[0124] In step S404a, the area of the attacker (attack area) is obtained based on the grounds area obtained in step S403a. For example, the area obtained in step S403a may be expanded by obtaining the outline or edges surrounding the area obtained in step S403a. Alternatively, after dividing the area into small areas using edges, etc., the area obtained in step S403a may be expanded by linking ranges of similar color using a color histogram or the like.

[0125] Alternatively, a segmentation region may be obtained by a neural network that has learned the segmentation of general objects, and the region of the attacking object (attack region) may be obtained by using the segmentation region that includes the region obtained in step S403a.

[0126] That is, the image storage device may have a learning means for learning a detection means for detecting a spoofing determination area using an image that the first spoofing determination means has determined to be an attack and the second spoofing determination means has determined to be a non-attack, among the images stored in the image storage device. Also, the detection means learned by the learning means may be used to determine the area.

[0127] However, the method for acquiring the area of the attacking object is not limited to these. After the processing of step S404a, the flow of Fig. 4(A) ends, step S303c of Fig. 3(C) ends, and the process proceeds to step S304c.

[0128] In step S304c, it is determined whether only the first spoofing determination result is "non-attack." If the first spoofing determination result is "non-attack" and the second spoofing determination result is "attack," the determination is YES and the process proceeds to step S305c.

[0129] Otherwise, the process proceeds to step S306c. In step S305c, the non-attack area creation unit 308a creates a non-attack area. A detailed example of the process in step S305c will be described with reference to FIG.

[0130] In step S401b of Fig. 4(B), the contributing region created in step S202a is obtained. In step S402b, a second spoofing judgment ground region is created by determining the ground region of the judgment result obtained by second spoofing judgment unit 306a in step S301c. In step S402b, the spoofing judgment ground region can be determined using GradCAM or the like in the same manner as in step S402a.

[0131] In step S403b, a ground truth region outside the contributing region is obtained. This makes it possible to obtain a region that does not affect authentication but is likely to be determined as spoofing. In step S404b, a region of an obstruction is obtained based on the region obtained in step S403b.

[0132] In step S404b, the area of the obstruction is expanded in the same manner as in step S404a. The area other than the area of the obstruction in the image of the person to be authenticated is then regarded as the contributing area. After processing step S404b, the flow in FIG. 4B ends, step S305c in FIG. 3C ends, and the process proceeds to step S306c.

[0133] In step S306c, the determination result is stored in the image storage unit 309a and displayed. Specifically, first, one row of data in the table shown in Fig. 5(A) is stored. Figs. 5(A) to 5(D) are diagrams showing examples of the format of data stored in step S306c of the second embodiment and examples of display.

[0134] Fig. 5(A) shows an example of data stored in image storage unit 309a. Determination date and time 501a holds the date and time when the process of Fig. 3(B) was performed. First spoofing determination result 502a holds the determination result obtained in step S203a.

[0135] The second spoofing determination result 503a holds the determination result obtained in step S301c. The image data 504a holds the authentication image obtained by the authentication image acquisition unit 102b processed in the flow of FIG.

[0136] The attack area data 505a holds the attack area obtained in step S303c. The non-attack area data 506a holds the non-attack area obtained in step S305c. Note that NONE in 505a and 506a indicates that no corresponding data exists.

[0137] Next, the attack image display process in step S306c of embodiment 2 will be described with reference to Fig. 4(C). Note that the display of the process in Fig. 4(C) is performed on the monitor 110a or the like, and operations such as pressing buttons on the screen are performed using a mouse or the like connected to the input device 109a.

[0138] In step S401c, an attack sample is acquired from the image storage unit 309a. Specifically, a record in which the judgment result of the first spoofing judgment result 502a in Fig. 5A is "attack" is acquired.

[0139] In step S402c, the attack samples obtained in step S401c are displayed. Specifically, they are displayed as shown in the display example in Fig. 5(B). 501b and 502b display the attack samples, and these attack samples are displayed in Z-shaped order from the top left of the screen, starting from the most recent assessment date.

[0140] 5(B), the frames of attack samples are thickened to highlight samples whose second spoofing judgment result is "non-attack." That is, the frame of attack sample 501b is not thickened because the second spoofing judgment result is "attack."

[0141] On the other hand, the frame of attack sample 502b is thicker because the second spoofing determination result is "non-attack." That is, when the image display means displays an image determined to be an attack by the first spoofing determination means, it highlights an image determined to be a non-attack by the second spoofing determination means.

[0142] By highlighting in this way, it is possible to more clearly highlight sophisticated attacks that may result in a false positive in the second spoofing judgment result, and to warn the user. In addition, by pressing (clicking) button 503b that says "Partial attacks", the attack samples are narrowed down to those with bold frames and are visualized for the user.

[0143] The method of highlighting a sample in which the first spoofing judgment result is "attack" but the second spoofing judgment result is "non-attack" is not limited to the above example.

[0144] In step S403c, when the user selects one of the attack samples shown in step S402c, the selected sample is displayed in more detail on the screen. For example, when attack sample 502b in Fig. 5(B) is selected, the display is as shown in Fig. 5(C).

[0145] Attack area 501c is a visualization of the attack area data in Figure 5(A). In this example, a piece of paper with a print of another person's eyes is held up in front of the face. The area of the paper around the eyes is displayed in color as attack area 501c.

[0146] Furthermore, when the user selects the attack area 501c with a mouse or the like, it may be enlarged and displayed as shown in Figure 5(D), allowing for detailed observation of the material used to create the paper area around the eyes.

[0147] In addition to coloring the attack area, it is also possible to surround the area with a red line, etc. Also, when 502b is selected, an enlarged display like that shown in Figure 5(D) may be displayed. The method of displaying the attack area is not limited to these.

[0148] Next, an example of the learning process for spoofing determination area detection in the spoofing determination area detection learning unit 311a of the second embodiment will be described with reference to Fig. 4(D). The process of Fig. 4(D) is executed when a learning instruction is received from the user via the input device 109a or the like. Alternatively, it may be executed when the amount of data in the image storage unit 309a exceeds a predetermined amount. However, the execution timing of the learning process of Fig. 4(D) is not limited to these.

[0149] In step S401d, a partial attack sample is obtained from the image storage unit 309a. Specifically, a record in which the first spoofing judgment result 502a in Fig. 5A is "attack" and the second spoofing judgment result is "non-attack" is obtained.

[0150] In step S402d, the attack area data 505a and image data 504a of the record obtained in step S401d are obtained. In step S403d, the attack area is learned using the data obtained in step S402d.

[0151] That is, the learning means trains the detection means to detect the attack area. Specifically, the learning means may train the area detector to perform segmentation that inputs image data using a neural network or the like and outputs attack area data.

[0152] In step S404d, a partial biometric sample is obtained from the image storage unit 309a. Specifically, for example, a record in which the first spoofing determination result 502a in Fig. 5A is "non-attack" and the second spoofing determination result is "attack" is obtained.

[0153] In step S405d, the non-attack area data 506a and the image data 504a of the record obtained in step S404d are obtained. In step S406d, the non-attack area is learned using the data obtained in step S405d.

[0154] That is, the learning means trains the detection means to detect non-attack areas. Specifically, the learning means may train the area detector to perform segmentation that inputs image data using a neural network or the like and outputs non-attack area data.

[0155] In this way, the learning means uses images stored in the image storage means that are judged as non-attack by the first spoofing judgment means and as attack by the second spoofing judgment means to learn the detection means for the spoofing judgment area.

[0156] In step S407d, the detection method (determination method) of the spoofing determination area determination unit 104b is updated. Specifically, the area detector learned in step S403d is used to detect the spoofing characteristic area in step S201c of FIG. 2(C).

[0157] That is, in step S201c, spoofing feature areas such as reflections are detected, but in addition, in step S403d, areas obtained by the trained area detector are also detected, and these areas are added to the spoofing feature determination areas.

[0158] Similarly, the area detector trained in step S406d is used in step S202c in Fig. 2(C). That is, in step S202c, areas where spoofing has been erroneously determined, such as blur, are detected, but in addition, areas obtained by the trained area detector in step S406d are also detected, and these areas are configured to be excluded from the spoofing feature determination areas.

[0159] In the above embodiment, only attack samples are visualized, but non-attack samples may also be visualized. In other words, samples for which the first spoofing judgment result is "non-attack" may be visualized.

[0160] In particular, it may be possible to display only non-attack samples where the first spoofing judgment result is "non-attack" but the second spoofing judgment result is "attack." This allows users to know what kind of accessories are likely to induce erroneous spoofing judgments.

[0161] In addition, non-biological areas may also be displayed, as in the case of the attack area shown in Fig. 5(C), which makes it easier to find non-biological areas on the face that are likely to lead to false spoofing detection.

[0162] Similarly, the non-biological area may be enlarged, as in the case of the attack area shown in Fig. 5(D), allowing for detailed observation of the material in the non-biological area that is likely to induce false spoofing judgments.

[0163] In this embodiment, the image is displayed after being saved, but it is also possible to only save the image. In this case, the image display unit 310a and the spoofing judgment area detection learning unit 311a are not required. Also, it is possible to configure the system so that the attack area and non-attack area are not saved, in which case the attack area creation unit 307a and the non-attack area creation unit 308a are not required.

[0164] Also, the samples to be stored in the image storage unit 309a may be limited to a certain number. For example, a sample for which the first spoofing judgment result is "non-attack" and the second spoofing judgment result is "non-attack" may not be stored in the image storage unit 309a.

[0165] Since such obvious biological samples are generated frequently during the operation of the system, capacity can be saved by not storing such data.Furthermore, it is also possible to store only when the first spoofing judgment result and the second spoofing judgment result differ.

[0166] Alternatively, it is also possible to store only samples of sophisticated attacks, such as when the first spoofing judgment result is "attack" but the second spoofing judgment result is "non-attack." However, the method of selecting samples to store is not limited to this.

[0167] In this embodiment, both "displaying an attack image" and "learning to detect an area for spoofing" are performed, but it is also possible to configure the system to perform only one of them. When "displaying an attack image" is not performed, the image display unit 310a is not required.

[0168] In addition, if a biometric sample is not displayed, the processes of steps S304c and S305c may be omitted. Similarly, if "learning of spoofing determination area detection" is not performed, spoofing determination area detection learning unit 311a is not necessary.

[0169] In addition, in this embodiment, in "learning for detecting spoofing judgment areas," both "learning of attack areas" and "learning of non-attack areas" are performed, but it is also possible to perform only one of them. If "learning of attack areas" is not performed, steps S401d to S403d can be omitted. If "learning of non-attack areas" is not performed, steps S404d to S406d can be omitted.

[0170] The above-described embodiment utilizes the case where the results of a normal spoofing judgment (second spoofing judgment) and a spoofing judgment based on a contributing area (first spoofing judgment) differ, thereby making it possible to extract sophisticated attack samples that normal spoofing judgments would miss (such as partial attacks, such as presenting a photo of someone else's mouth).

[0171] By displaying the extracted results preferentially, it is possible to more clearly warn users of sophisticated attacks. Also, by learning the attack areas of these sophisticated attack samples, it becomes possible to extract spoofing characteristic areas that can serve as clues for spoofing detection. Furthermore, by adding these areas to the detection area for the first spoofing detection, it is possible to reduce the number of missed spoofing detections.

[0172] In addition, it is possible to extract non-attack samples with decorations that would cause a false positive in a normal spoofing detection. By learning the non-attack areas of these non-attack samples, it is possible to extract areas that would induce a false positive in the spoofing detection. Furthermore, by excluding these areas from the detection area of the first spoofing detection, it is possible to reduce false positives in the spoofing detection.

[0173] <Embodiment 3> In the third embodiment, when the second spoofing determination determines that an attack has occurred, the first spoofing determination is performed. The configuration of the information processing device in the third embodiment may be the same as that in the second embodiment. Alternatively, 307a to 311a may be deleted.

[0174] The spoofing determination process of the third embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of the spoofing determination process of the third embodiment. Note that the CPU 101a as a computer in the information processing device executes a computer program stored in memory, thereby sequentially performing the operations of the steps in the flowchart of Fig. 6.

[0175] In step S601, second spoofing determination unit 306a performs spoofing determination. In step S602, it is determined whether the determination result in step S601 is "attack." If it is "attack," the process proceeds to step S603. Otherwise, the flow in FIG. 6 ends.

[0176] In step S603, the spoofing determination process shown in Fig. 3(B) is performed. Note that the data saving process in step S305b may be configured not to be performed. As described above, in the third embodiment, in step S602, the first spoofing determination is performed only when the second spoofing determination process determines an "attack." In other words, the first spoofing determination means performs spoofing determination when the second spoofing determination means determines an attack, thereby making the process more efficient.

[0177] In step S204a, if the area of the contributing region obtained in step S202a is smaller than a predetermined threshold, it may be determined to be spoofing. That is, if the area of the contributing region is equal to or smaller than a predetermined value, it may be determined to be an attack. In this case, the contributing region may be subjected to threshold processing or the like before the area is calculated.

[0178] This is because, in situations where a person is wearing a mask, sunglasses, and a hat and the skin area is almost completely hidden, it is better to judge the area as spoofing even if it is not in the contributing area, as this is suspicious. This reduces the number of false positives that occur when spoofing is judged using only the slightly visible area.

[0179] Although the above-described embodiment aims to detect spoofing of face authentication, other biometric authentication may be used. For example, spoofing of other biometric authentication such as iris authentication or fingerprint authentication may be detected.

[0180] Although the present invention has been described in detail above based on the preferred embodiments, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible based on the spirit of the present invention, and are not excluded from the scope of the present invention. The present invention also includes the following combinations.

[0181] (Configuration 1) An information processing device characterized by having an acquisition means for acquiring an image including a person to be authenticated, an identification means for identifying a contributing area within the image to be used for authenticating the person, a determination means for determining a judgment area to be used for impersonation judgment processing based on the contributing area identified by the identification means, and an impersonation judgment means for executing an impersonation judgment processing for determining impersonation based on the judgment area determined by the determination means.

[0182] (Configuration 2) The information processing device according to configuration 1, wherein the determining means determines the judgment area by adding an spoofing characteristic area having an spoofing characteristic to the contributing area.

[0183] (Configuration 3) The information processing device according to configuration 1 or 2, wherein the spoofing characteristic region includes at least one of reflection, wrinkles, peeling, and dirt in the image of the person to be authenticated.

[0184] (Configuration 4) An information processing device described in any one of configurations 1 to 3, characterized in that the determination means determines the judgment area by excluding from the contributing area an area that is an erroneous spoofing judgment that falls under predetermined conditions that induce an erroneous spoofing judgment.

[0185] (Configuration 5) The information processing device according to Configuration 4, wherein the determining means excludes the spoofing erroneously determined region from the determination region based on blur or brightness of the image of the person to be authenticated.

[0186] (Configuration 6) The information processing device according to any one of configurations 1 to 5, wherein the determining means determines the contributing region to be a region other than an area of an obstruction in the image of the person to be authenticated.

[0187] (Configuration 7) The information processing device according to Configuration 6, wherein the shielding object includes at least one of a mask, sunglasses, glasses, face paint, and a false beard.

[0188] (Configuration 8) A registered image acquisition means for acquiring a registered image of the person to be authenticated; The information processing device according to any one of configurations 1 to 7, wherein the determination means determines the contributing area based on an area that contributes to determining the similarity between the image acquired by the acquisition means and the registered image.

[0189] (Configuration 9) The information processing device according to configuration 8, wherein the registration image acquisition means acquires an image of a person not wearing a shield as the registration image.

[0190] (Configuration 10) The information processing device according to any one of configurations 1 to 9, further comprising a first spoofing determination means for determining whether the image is an attack attempting fraudulent authentication based on the determination area.

[0191] (Configuration 11) The information processing device according to configuration 10, wherein the first spoofing determination means determines that an attack has occurred when the area of the contributing region is equal to or smaller than a predetermined value.

[0192] (Configuration 12) The information processing device according to configuration 10 or 11, characterized in that it has a second impersonation determination means for determining whether the image is an attack attempting fraudulent authentication based on the image acquired by the acquisition means.

[0193] (Configuration 13) The information processing device according to configuration 12, wherein the first spoofing determination means performs the spoofing determination when the second spoofing determination means determines that an attack has occurred.

[0194] (Configuration 14) An information processing device according to configuration 12 or 13, characterized in that it has an image storage means for storing the image, the judgment result of the first spoofing judgment means, and the judgment result of the second spoofing judgment means.

[0195] (Configuration 15) An information processing device as described in Configuration 14, characterized in that it has a learning means for learning a detection means for detecting the judgment area using images among the images stored in the image storage means that are judged to be attacks by a first impersonation judgment means and judged to be non-attacks by a second impersonation judgment means, and makes the decision using the detection means of the judgment area obtained by the learning means.

[0196] (Configuration 16) The method further includes an attack area creation means for creating an attack area based on the contributing area, 16. The information processing device according to configuration 15, wherein the learning means trains the detection means to detect the attack area.

[0197] (Configuration 17) An information processing device described in any one of configurations 14 to 16, characterized in that it has a learning means for learning the detection means for detecting spoofing judgment areas using images that are judged as non-attack by the first spoofing judgment means and as attack by the second spoofing judgment means, among the images stored in the image storage means.

[0198] (Configuration 18) An information processing device as described in Configuration 17, further comprising a non-attack area creation means for creating a non-attack area based on the contributing area, wherein the learning means trains the detection means to detect the non-attack area.

[0199] (Configuration 19) An information processing device according to any one of configurations 14 to 18, characterized in that it has an image display means for highlighting an image that the second impersonation determination means has determined not to be an attack when displaying an image that the first impersonation determination means has determined to be an attack.

[0200] (Method) An information processing method comprising: an acquisition step of acquiring an image including a person to be authenticated; an identification step of identifying a contributing area within the image to be used for authenticating the person; a determination step of determining a judgment area to be used for impersonation judgment processing based on the contributing area identified by the identification step; and an impersonation judgment step of performing an impersonation judgment processing to judge impersonation based on the judgment area determined by the determination step.

[0201] (Program) A computer program for controlling each means of the information processing device according to any one of configurations 1 to 19 by a computer.

[0202] In order to realize part or all of the control in the above-described embodiments, a computer program that realizes the functions of the above-described embodiments may be supplied to an information processing device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the information processing device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0203] 101b: Registration image acquisition unit 102b: Authentication image acquisition unit 103b: Authentication contribution area determination unit 104b: Spoofing detection area determination unit 105b: First spoofing determination unit

Claims

1. an acquisition means for acquiring an image including a person to be authenticated; means for identifying a contributing region within the image for use in authenticating the person; a determination means for determining a determination area to be used in the spoofing determination process based on the contributing area identified by the identification means; a spoofing determination means for executing a spoofing determination process for determining spoofing based on the determination area determined by the determination means; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the determining means determines the judgment area by adding an spoofing characteristic area having an spoofing characteristic to the contributing area.

3. The information processing apparatus according to claim 2 , wherein the spoofing characteristic region includes at least one of reflection, wrinkles, peeling, and dirt in the image of the person to be authenticated.

4. 2. The information processing apparatus according to claim 1, wherein the determining means determines the determination area by excluding from the contributing area an area that satisfies a predetermined condition that induces an erroneous determination of spoofing.

5. The information processing apparatus according to claim 4 , wherein the determining unit excludes the spoofing erroneously determined region from the determination region based on blur or brightness of the image of the person to be authenticated.

6. The information processing apparatus according to claim 1 , wherein the determining unit determines an area of the image of the person to be authenticated other than an area of an obstruction as the contributing area.

7. 7. The information processing apparatus according to claim 6, wherein the shielding object includes at least one of a mask, sunglasses, glasses, face paint, and a false beard.

8. a registration image acquisition means for acquiring a registration image of the person to be authenticated; 2. The information processing apparatus according to claim 1, wherein the determining means determines the contributing area based on an area that contributes to determining the degree of similarity between the image acquired by the acquiring means and the registered image.

9. 9. The information processing apparatus according to claim 8, wherein the registration image acquisition means acquires an image of a person not wearing a shield as the registration image.

10. 2. The information processing apparatus according to claim 1, further comprising a first spoofing determination means for determining whether the image is an attack attempting fraudulent authentication based on the determination area.

11. 11. The information processing apparatus according to claim 10, wherein the first spoofing determination means determines that an attack has occurred when the area of the contributing region is equal to or smaller than a predetermined value.

12. 11. The information processing apparatus according to claim 10, further comprising a second spoofing determination unit that determines whether the image obtained by the obtaining unit represents an attack attempting fraudulent authentication.

13. 13. The information processing apparatus according to claim 12, wherein the first spoofing determination means performs the spoofing determination when the second spoofing determination means determines that an attack has occurred.

14. 13. The information processing apparatus according to claim 12, further comprising image storage means for storing the image, the determination result of the first spoofing determination means, and the determination result of the second spoofing determination means.

15. An information processing device as described in claim 14, characterized in that it has a learning means for learning a detection means for detecting the judgment area using images among the images stored in the image storage means that are judged to be attacks by a first spoofing judgment means and judged to be non-attacks by a second spoofing judgment means, and the decision is made using the detection means of the judgment area obtained by the learning means.

16. The attack area generating means further includes an attack area generating means for generating an attack area based on the contributing area, 16. The information processing apparatus according to claim 15, wherein the learning means trains the detection means to detect the attack area.

17. 15. The information processing device according to claim 14, further comprising a learning means for learning the detection means for detecting spoofing judgment areas using images among the images stored in the image storage means that are judged as non-attack by the first spoofing judgment means and as attack by the second spoofing judgment means.

18. The method further includes a non-attack area creating means for creating a non-attack area based on the contributing area, 18. The information processing apparatus according to claim 17, wherein the learning means trains the detection means to detect the non-attack area.

19. 15. The information processing apparatus according to claim 14, further comprising image display means for highlighting an image determined as a non-attack by said second spoofing determination means when displaying an image determined as an attack by said first spoofing determination means.

20. an acquisition step of acquiring an image including a person to be authenticated; identifying a contributing region within the image for use in authenticating the person; a determination step of determining a determination area to be used in the spoofing determination process based on the contributing area identified in the identification step; a spoofing determination step of executing a spoofing determination process for determining spoofing based on the determination area determined in the determination step; An information processing method comprising:

21. A computer program for controlling each means of the information processing device according to any one of claims 1 to 19 by a computer.

Images