Face authentication device and face authentication method

Abstract

To provide a face authentication device that controls light applied to a face of a subject and improves accuracy of face authentication.SOLUTION: A face authentication device includes: an image data reception unit that receives image data reflecting a subject from a camera; an image generation unit that generates, on the basis of the image data, projection image data of a projection image projected onto the subject; a projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; a projection subject image data reception unit that receives projection subject image data of the subject on which the projected image is projected from the camera; and a face authentication unit that performs, on the basis of the projection subject image data, face authentication processing of the subject.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a face authentication device and a face authentication method.

Background Art

[0002] Patent Document 1 discloses a face authentication device that performs face authentication on a user passing through a gate or the like.

[0003] Patent Document 2 discloses an illumination system that illuminates a subject photographed by a photographing camera. The illumination system of Patent Document 2 is a system that illuminates while avoiding the pupils of the subject and reduces the glare of the illumination light on the subject. Patent Document 2 discloses that only the brightness adjustment of the light applied to the subject is performed in accordance with the ambient light.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In face authentication, the authentication accuracy may vary due to the influence of the light hitting the face of the subject, and it is important to control the light applied to the face.

[0006] Non-limiting embodiments of the present disclosure contribute to providing a face authentication device and a face authentication method that control the light applied to the face of the subject and improve the authentication accuracy of face authentication.

Means for Solving the Problems

[0007] A face recognition device according to an embodiment of the present disclosure includes an image data receiving unit that receives image data in which a subject is reflected from a camera, an image generation unit that generates projection image data of a projection image projected onto the subject based on the image data, a projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject, a projection subject image data receiving unit that receives projection subject image data of the subject onto which the projection image is projected from the camera, and a face recognition unit that performs face recognition processing of the subject based on the projection subject image data.

[0008] A face recognition method according to an embodiment of the present disclosure includes receiving image data in which a subject is reflected from a camera, generating projection image data of a projection image projected onto the subject based on the image data, transmitting the projection image data to a projector that projects the projection image onto the subject, receiving projection subject image data of the subject onto which the projection image is projected from the camera, and performing face recognition processing of the subject based on the projection subject image data.

[0009] These general or specific aspects may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium, or may be implemented in any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0010] According to an embodiment of the present disclosure, the face recognition device can control the light applied to the face of the subject and improve the authentication accuracy of face recognition.

[0011] Further advantages and effects in an embodiment of the present disclosure will be clarified from the specification and drawings. Such advantages and / or effects are provided by some embodiments and the features described in the specification and drawings respectively, but it is not necessarily required that all be provided to obtain one or more of the same features.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, a more detailed description than necessary may be omitted. For example, a detailed description of already well-known matters or a redundant description of substantially the same configuration may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art.

[0014] Note that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] (First Embodiment) FIG. 1 is a diagram showing an example of a face authentication system according to the first embodiment. As shown in FIG. 1, the face authentication system includes a face authentication device 1, a camera 2, and a projector 3. FIG. 1 also shows a subject 4 to be face-authenticated.

[0016] The face authentication system is installed, for example, inside or outside a building such as a building, and performs face authentication processing on the subject 4 passing through a predetermined location inside or outside the building. The face authentication system is used, for example, in an entrance / exit system that determines the entrance / exit authority of the subject 4 using the face authentication result of the face authentication system, or in an attendance management system that manages the attendance of the subject 4. Note that the face authentication device 1 may be installed at a location different from the location where the camera 2 and the projector 3 are installed.

[0017] The face authentication device 1, the camera 2, and the projector 3 are each connected to a network such as a LAN (Local Area Network) or the Internet. The face authentication device 1, the camera 2, and the projector 3 communicate via the network. The face authentication device 1 may be an information processing device such as a server or a personal computer, for example.

[0018] The face authentication device 1 receives the image data captured by the camera 2. The image data captured by the camera 2 may be moving image data. Based on the image data including the subject 4, the face authentication device 1 generates the image data of the image to be projected from the projector 3 onto the subject 4. The face authentication device 1 transmits the generated image data to the projector 3.

[0019] The projector 3 projects an image based on the image data transmitted from the face authentication device 1 onto the subject 4. Note that the image projected by the projector 3 onto the subject 4 may be regarded as illumination light for face authentication.

[0020] The camera 2 captures the subject 4 onto which the image (light) of the projector 3 is projected, and transmits the image data to the face authentication device 1.

[0021] The face recognition device 1 receives image data including the subject 4 onto which the image of the projector 3 is projected, from the camera 2. The face recognition device 1 performs face recognition of the subject 4 based on the image data of the subject 4 onto which the image of the projector 3 is projected.

[0022] That is, in the face recognition system shown in FIG. 1, the camera 2 is used as a camera for face recognition and also as a sensor for controlling the light projected onto the subject 4. The projector 3 is used as a flexible lighting device that changes the brightness or color of the light projected onto the subject 4.

[0023] Then, the face recognition device 1 controls the image (light) projected by the projector 3 based on the image data of the subject 4 captured by the camera 2, and performs face recognition processing of the subject 4 based on the image data of the camera 2 that has captured the subject 4 onto which the image of the projector 3 is projected.

[0024] FIG. 2 is a diagram showing an example of the block configuration of the face recognition device 1. As shown in FIG. 2, the face recognition device 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and an output unit 15.

[0025] The control unit 11 controls the entire face recognition device 1. The control unit 11 may be configured by a processor such as a CPU (Central Processing Unit), for example.

[0026] The control unit 11 includes an image generation unit 11a and a face recognition unit 11b. The image generation unit 11a generates image data to be output to the projector 3 based on the image data of the image including the subject 4 captured by the camera 2. The face recognition unit 11b performs face recognition of the subject 4 based on the image data of the camera 2 that has captured the subject 4 onto which the image is projected by the projector 3. The control unit 11 may execute, for example, a program stored in the storage unit 12 to realize the functions of the image generation unit 11a and the face recognition unit 11b.

[0027] The storage unit 12 stores a program for the control unit 11 to operate. Also, the storage unit 12 stores data for the control unit 11 to perform calculation processing and data for controlling each unit.

[0028] The communication unit 13 is connected to a network such as a LAN or the Internet. The communication unit 13 communicates with the camera 2 via the network. The communication unit 13 communicates with the projector 3 via the network. The communication unit 13 communicates with an entrance / exit system and an attendance management system via the network.

[0029] The communication unit 13 includes an image data receiving unit 13a, a projection image data transmitting unit 13b, and a projected subject image data receiving unit 13c. The image data receiving unit 13a receives image data from the camera 2. The projection image data transmitting unit 13b transmits projection image data to the projector 3. The projected subject image data receiving unit 13c receives projected subject image data from the camera 2.

[0030] The input unit 14 is connected to an input device such as a keyboard or a touch panel. The input unit 14 outputs a signal received from the input device to the control unit 11.

[0031] The output unit 15 is connected to a display device such as a display or a touch panel. The output unit 15 outputs a signal received from the control unit 11 to the display device.

[0032] An example of the image generation process and the face authentication process of the projector 3 in the face authentication device 1 will be described. Hereinafter, the image data of the image transmitted from the face authentication device 1 to the projector 3 and projected onto the subject 4 may be referred to as projection image data. Also, the image data of the camera 2 that captures the subject 4 onto which the image of the projector 3 is projected may be referred to as projected subject image data.

[0033] ·Example 1 The face recognition device 1 generates projection image data for correcting the brightness or color of the area around the eyes of the subject 4 wearing glasses based on the image data of the subject 4 captured by the camera 2. In this case, the color may include hue, color tone, and pattern.

[0034] For example, the face recognition device 1 determines whether the subject 4 is wearing glasses from the image data of the subject 4 captured by the camera 2. The determination of whether the subject 4 is wearing glasses can be realized, for example, by using a machine learning model trained by machine learning such as deep learning with images of faces wearing glasses, or by confirming through pattern matching whether a pattern corresponding to glasses is included in the image data. When the face recognition device 1 determines that the subject 4 is wearing glasses, it generates projection image data in which the image of the area around the eyes (glasses area) of the subject 4 is masked. More specifically, the face recognition device 1 generates projection image data in which no image (light) is projected onto the area around the eyes of the subject 4, and an image is projected onto the area other than the eyes.

[0035] The face recognition device 1 outputs the generated projection image data to the projector 3. The projector 3 projects an image based on the projection image data received from the face recognition device 1 onto the subject 4.

[0036] FIG. 3 is a diagram showing an example of the face of the subject 4 onto which the image of the projector 3 is projected. In FIG. 3, the subject 4 is wearing glasses.

[0037] When the subject 4 is wearing glasses, no image (light) of the projector 3 is projected onto the area around the eyes of the face of the subject 4. Therefore, the area around the eyes of the subject 4 becomes darker than the area other than the eyes.

[0038] The camera 2 captures the subject 4 onto which the image of the projector 3 is projected, and transmits projection subject image data to the face recognition device 1. The face recognition device 1 performs face recognition of the subject 4 based on the projection subject image data received from the camera 2.

[0039] Here, the image of the projector 3 is not projected onto the eye area of the subject 4. Therefore, the reflection of light by the glasses is suppressed. That is, in the face authentication process, the reflected light of the glasses, which is noise, is suppressed. Also, in the face authentication process, since eyes and pupils may be used as characteristic parts of the subject 4, if the highlights of the glasses hide these parts, the accuracy of face authentication will decrease. In the present embodiment, since the reflected light of the glasses can be suppressed, the decrease in accuracy due to such a cause can also be suppressed. As a result, the face authentication device 1 can improve the authentication accuracy of face authentication.

[0040] Note that the glasses assumed in the present embodiment are not limited to general vision correction glasses, and various types are conceivable. For example, glasses with colored lenses such as sunglasses, or those for eye protection like goggles are also included in the "glasses" of the present embodiment. In the case of sunglasses, etc., the eyes of the subject 4 may be hidden and not be able to be photographed by the camera 2. However, in the present embodiment, even in this case, the reflected light, which is noise in the face authentication process, can be suppressed, so the accuracy of the face authentication process is improved compared to the case of simply projecting illumination.

[0041] Note that in the above, the face authentication device 1 was assumed to generate projection image data with the eye area of the subject 4 masked, but it is not limited to this. When the face authentication device 1 determines that the subject 4 is wearing glasses, it may generate projection image data in which the brightness of the image of the eye area of the subject 4 is smaller than the brightness of the image of the area other than the eye area of the subject 4.

[0042] Also, when the subject 4 is wearing glasses, the face authentication device 1 may generate projection image data of a color in which the reflection of light by the glasses is suppressed. For example, the face authentication device 1 may generate projection image data in which the eye area of the subject 4 becomes gray.

[0043] Also, regardless of whether the subject 4 is wearing glasses or not, the face authentication device 1 may generate projection image data with the pupil area of the subject 4 masked. Thereby, the face authentication device 1 can prevent the subject 4 from feeling dazzled by the light of the projector 3.

[0044] Also, when the subject 4 feels dazzled by the light of the projector 3, the subject may close their eyes, making it difficult to utilize the eye features in the face authentication process. In this embodiment, by generating projection image data with the pupil portion of the subject 4 masked, it is possible to suppress a decrease in the accuracy of the face authentication process due to the subject 4 closing their eyes.

[0045] ·Example 2 The face authentication device 1 generates projection image data for correcting the brightness or color of the mouth area of the masked subject 4 based on the image data of the subject 4 captured by the camera 2.

[0046] For example, the face authentication device 1 determines whether the subject 4 is wearing a mask from the image data of the subject 4 captured by the camera 2. The determination of whether the subject 4 is wearing a mask can be realized, for example, by using a machine learning model trained by machine learning such as deep learning for an image of a face wearing a mask, or by confirming through pattern matching whether a pattern corresponding to a mask is included in the image data. When the face authentication device 1 determines that the subject 4 is wearing a mask, it generates projection image data based on the color of the mask. For example, when the color of the mask is white, the face authentication device 1 generates projection image data of a predetermined color different from white, such as gray, in the mouth area of the subject 4.

[0047] The face authentication device 1 outputs the generated projection image data to the projector 3. The projector 3 projects an image based on the projection image data received from the face authentication device 1 onto the subject 4.

[0048] FIG. 4 is a diagram showing an example of the face of the subject 4 onto which the image of the projector 3 is projected. In FIG. 4, the subject 4 is wearing a white mask.

[0049] When the subject 4 wears a white mask, a gray image is projected, for example, by the projector 3 onto the mouth area of the face of the subject 4. Therefore, the color of the mask worn by the subject 4 becomes gray.

[0050] The camera 2 captures the subject 4 onto which the image of the projector 3 is projected, and transmits the projected subject image data to the face authentication device 1. The face authentication device 1 performs face authentication of the subject 4 based on the projected subject image data received from the camera 2.

[0051] Here, a gray image is projected by the projector 3 onto the mouth area of the subject 4. For this reason, the white-out of the image due to the white mask is suppressed.

[0052] Here, white-out is a phenomenon in which, because the luminance of the subject is too high, the imaging element of the camera 2 exceeds the width by which color gradations can be distinguished, and thus it is recognized as a white image. When the subject 4 wears a mask, although part of the face is hidden, since the distribution of the shadows resulting from the three-dimensional shape of the face differs for each individual, the color corresponding to this shadow can be used for face authentication. However, when white-out occurs, the color information corresponding to these shadows is also lost, so the accuracy of face authentication decreases. Therefore, in the present embodiment, when a mask is detected, the color projected onto the mouth area is darkened to suppress the occurrence of white-out. Thereby, the face authentication device 1 can improve the authentication accuracy of face authentication.

[0053] Note that, in the above, the face authentication device 1 was described as generating projection image data of a color different from white, such as gray, onto the mouth area of the subject 4, but it is not limited to this. The face authentication device 1 may generate projection image data of an image obtained by masking the mouth area of the subject 4.

[0054] Also, even if the color of the mask is other than white, the face authentication device 1 may generate projection image data. For example, the face authentication device 1 may select a color that suppresses white-out and reflection according to the color of the mask, and generate projection image data for projecting an image of the selected color onto the mouth area of the subject 4.

[0055] In addition, the above-described embodiment may be applied to a mask that is not entirely white. This is because white bleeding can occur in any bright color.

[0056] Also, although the mask in the present embodiment has been described as covering the mouth area, a mask that covers other parts of the face may also be targeted. Even in this case, when white bleeding occurs, the color tone of that part is lost, and the accuracy of the face authentication process decreases. When assuming a mask that covers other parts than the mouth area, the color of the portion corresponding to the part covered by the mask in the projection image data may be darkened.

[0057] ·Example 3 For example, depending on the installation location of the camera 2, the brightness of the face of the subject 4 may become uneven. For example, when the camera 2 is installed in a place where outdoor light shines in from one direction, or when it is installed in a place where illumination light is irradiated from one direction, the brightness of the face of the subject 4 may become uneven. Therefore, based on the image data captured by the camera 2, the face authentication device 1 analyzes the intensity of the brightness on the face of the subject 4 and generates projection image data so that the brightness of the entire face of the subject 4 becomes uniform.

[0058] FIG. 5 is a diagram showing an example in which the brightness of the face of the subject 4 is uneven. In FIG. 5, with respect to the face of the subject 4, when viewed from the front, light is hitting from the right side. For this reason, when the face of the subject 4 is viewed from the front, the right side is brighter than the left side. That is, the brightness of the face of the subject 4 is uneven.

[0059] The face authentication device 1 analyzes the intensity of the light hitting the face of the subject 4 from the image data of the subject 4 captured by the camera 2. Based on the analysis result of the intensity of the light, the face authentication device 1 generates projection image data so that the brightness becomes uniform over the entire face of the subject 4.

[0060] For example, the face recognition device 1 generates projection image data of a color with uniform brightness over the entire face of the subject 4. More specifically, the face recognition device 1 generates projection image data such that a bright-colored image is projected onto the darker part of the light hitting the face of the subject 4. The face recognition device 1 generates projection image data such that a dark-colored image is projected onto the brighter part of the light hitting the face of the subject 4.

[0061] The face recognition device 1 outputs the generated projection image data to the projector 3. The projector 3 projects an image based on the projection image data received from the face recognition device 1 onto the subject 4.

[0062] FIG. 6 is a diagram showing an example of the face of the subject 4 onto which the image of the projector 3 is projected. As described with reference to FIG. 5, when the right side of the face of the subject 4 is brighter than the left side when viewed from the front, the face recognition device 1 generates projection image data such that a dark-colored image is projected onto the right side of the face of the subject 4. Further, the face recognition device 1 generates projection image data such that a bright-colored image is projected onto the left side of the face of the subject 4. Thereby, the face of the subject 4 onto which the image of the projector 3 is projected becomes uniform in brightness as shown in FIG. 6.

[0063] The camera 2 photographs the subject 4 onto which the image of the projector 3 is projected, and transmits projection subject image data to the face recognition device 1. The face recognition device 1 performs face recognition on the subject 4 based on the projection subject image data received from the camera 2.

[0064] Here, the face image data used for face recognition to be compared with the projection subject image data is generally photographed in advance in a place with proper lighting facilities and registered in the storage unit 12. Therefore, the face images included in the face image data used for face recognition are generally uniform in brightness. When face recognition is performed by comparing face image data with uniform face brightness and projection subject image data with non-uniform face brightness, the recognition accuracy of face recognition may decrease.

[0065] As described with reference to FIG. 6, the face of the subject 4 onto which the image of the projector 3 is projected has uniform brightness. Therefore, the face authentication device 1 performs face authentication based on the face image data with uniform brightness registered in advance in the storage unit 12 and the projected subject image data of the subject 4 whose brightness is corrected to be uniform by the projected image of the projector 3. Thereby, the face authentication device 1 can improve the authentication accuracy of face authentication.

[0066] ·Example 4 For example, depending on the color of the illumination light, the facial color and hair color of the subject 4 may change. For example, when the illumination light is red, the facial color and hair color of the subject 4 may have a reddish tint. That is, the facial color and hair color of the subject 4 may be different from the facial color and hair color of the face image data registered in advance in the storage unit 12 depending on the color of the illumination light. Therefore, the face authentication device 1 analyzes the facial color or hair color of the subject 4 based on the image data captured by the camera 2, and generates projection image data so that the facial color or hair color of the subject 4 becomes a predetermined color.

[0067] For example, when the face of the subject 4 is entirely reddish, the face authentication device 1 generates projection image data so that the facial color or hair color of the subject 4 becomes the color under white light.

[0068] The face authentication device 1 outputs the generated projection image data to the projector 3. The projector 3 projects an image based on the projection image data received from the face authentication device 1 onto the subject 4.

[0069] The camera 2 captures the subject 4 onto which the image of the projector 3 is projected, and transmits the projected subject image data to the face authentication device 1. The face authentication device 1 performs face authentication of the subject 4 based on the projected subject image data received from the camera 2.

[0070] Here, the face images for face authentication are generally pre-captured in a place with proper lighting facilities, etc., for example, under a white light. The face authentication device 1 performs face authentication based on the face image data for face authentication captured under a white light and the projected subject image data of the subject 4 whose face color and hair color have been corrected by the projected image of the projector 3. Thereby, the face authentication device 1 can improve the authentication accuracy of face authentication.

[0071] ·Example 5 The face authentication device 1 may generate projected image data based on the image data (background image data) of the background image (the image of the part other than the face or body of the subject 4) of the image captured by the camera 2. For example, when viewed from the camera 2, if there is a background of a specific color, such as blue, behind the subject 4, the authentication accuracy of face authentication may decrease. In the face authentication process, since the processing is performed using the entire image without distinguishing between the face and the background, if the color of the background is significantly different from the color of the background in the registered image, the authentication accuracy may decrease. Therefore, in this case, the face authentication device 1 generates the projected image data so that the background behind the subject 4 is a color different from the specific color, such as blue. Thereby, the face authentication device 1 can improve the authentication accuracy of face authentication.

[0072] Also, when the background colors of the face image data for face authentication registered in the storage unit 12 are almost the same color, the projected image data may be generated so that the background behind the subject 4 approaches that color. For example, when a large number of people take registration images at almost the same location, such as at the time of enrollment or employment, the background colors are likely to be similar to each other. Therefore, by adjusting the color of the projected image data so that the background color at the time of authentication approaches this color, an improvement in the accuracy of face authentication can be expected. In this case, the background color of the registered image data may be specified by the user in advance, or may be determined by analyzing the registered image data by the face authentication device 1 or the like.

[0073] Note that the above colors are not limited to single colors and may be patterns composed of multiple colors. That is, projection image data may be generated such that the pattern of the background becomes a pattern different from a specific pattern. Similar to the case when colors were described, a pattern significantly different from the background of the registered image may reduce the authentication accuracy. Therefore, it is beneficial to generate projection image data such that it becomes a pattern different from such a pattern. Also, when there is a pattern similar to a face or a part thereof (for example, an image that looks like an eye), projection image data may be generated to change that pattern. This is because if such a pattern exists in the background, there is a risk that the pattern will be misrecognized as the face of the subject 4 or a part thereof, resulting in a decrease in the face authentication accuracy.

[0074] Note that the generation of the projection image data described in the above Examples 1 to 5 is an example and is not limited to the above examples. Also, the methods for generating the projection image data in Examples 1 to 5 may be combined.

[0075] FIG. 7 is a flowchart showing an operation example of the face authentication device 1. The face authentication device 1 repeatedly executes the processing of the flowchart in FIG. 7, for example, at a predetermined cycle.

[0076] The face authentication device 1 receives image data from the camera 2 (S1).

[0077] The face authentication device 1 acquires attribute information regarding the face of the subject 4 from the image data received in S1 (S2). For example, the face authentication device 1 acquires face attribute information such as glasses, mask, face brightness, face color, and hair color from the face image of the subject 4.

[0078] The face authentication device 1 generates projection image data based on the attribute information of the face of the subject 4 acquired in S2 (S3).

[0079] The face authentication device 1 transmits the projection image data generated in S3 to the projector 3 (S4).

[0080] The projector 3 projects an image based on the projection image data transmitted from the face recognition device 1. The image of the projector 3 is projected onto the subject 4.

[0081] The face recognition device 1 receives projection subject image data from the camera 2 (S5).

[0082] The face recognition device 1 performs face recognition processing based on the face image of the subject 4 included in the projection subject image data received in S5 (S6). Note that the face recognition device 1 may perform face recognition processing using existing face recognition processing. For example, the face recognition device 1 may compare the face image data for face recognition registered in the storage unit 12 in advance with the image data of the face image of the subject 4 included in the projection subject image data received in S5, and calculate the degree of coincidence. The face recognition device 1 may calculate the degree of coincidence using machine learning such as deep learning, for example.

[0083] The face recognition device 1 determines whether the degree of coincidence calculated in S6 is equal to or greater than a threshold value (S7).

[0084] When the face recognition device 1 determines that the degree of coincidence calculated in S6 is equal to or greater than the threshold value (''YES'' in S7), it outputs an OK determination result to, for example, an attendance management system or an entrance / exit system (S8). Then, the face recognition device 1 ends the processing of the flowchart in FIG. 7. Note that the OK determination result indicates that, for example, the face image of the subject 4 in the projection subject image data received in S5 matches the face image of the face image data registered in the storage unit 12 in advance.

[0085] On the other hand, when the face recognition device 1 determines that the degree of coincidence calculated in S6 is not equal to or greater than the threshold value (''NO'' in S7), it outputs an NG determination result to, for example, an attendance management system or an entrance / exit system (S9). Then, the face recognition device 1 ends the processing of the flowchart in FIG. 7. Note that the NG determination result indicates that, for example, the face image of the subject 4 in the projection subject image data received in S5 does not match the face image of the face image data registered in the storage unit 12 in advance.

[0086] As described above, the image data receiving unit 13a of the face authentication device 1 receives image data in which the subject 4 is reflected from the camera 2. The image generation unit 11a generates projection image data of a projection image to be projected onto the subject 4 based on the image data. The projection image data transmitting unit 13b transmits the projection image data to the projector 3 that projects the projection image onto the subject 4. The projected subject image data receiving unit 13c receives the projected subject image data of the subject 4 onto which the projection image is projected from the camera 2. Then, the face authentication unit 11b performs face authentication processing of the subject based on the projected subject image data.

[0087] In this way, since the face authentication device 1 controls the image (light) projected onto the subject 4 of the projector 3 based on the image data of the subject 4 photographed by the camera 2, the authentication accuracy of face authentication can be improved.

[0088] Further, since the face authentication device 1 controls the light projected onto the subject 4 of the projector 3 based on the image data of the subject 4 photographed by the camera 2, it is not necessary to correct the projected subject image data by controlling the parameters of the camera 2. For example, the face authentication device 1 does not need to correct the brightness or color tone of the image of the projected subject image data by controlling the parameters of the camera 2. Thereby, the face authentication device 1 can easily improve the authentication accuracy of face authentication.

[0089] In addition, the face authentication device 1 uses the camera 2 as a camera for face authentication and also uses it as a sensor for controlling the light projected onto the subject 4. Thereby, the cost of the face authentication system is reduced.

[0090] Note that the face authentication device 1 may calculate the distance between the lighting device and the subject 4 from the image data of the camera 2. The face authentication device 1 may generate projection image data for correcting the brightness or color of the face of the subject 4 based on the calculated distance.

[0091] In addition, the face authentication device 1 may store a plurality of different projection image data in the storage unit 12 in advance. The face authentication device 1 may select the projection image data stored in the storage unit 12 in advance based on the image data of the camera 2.

[0092] In addition, the image captured by the camera 2 may include a plurality of subjects 4. In this case, the face authentication device 1 generates projection image data for each subject 4 based on the image data of the plurality of subjects 4. Then, the face authentication device 1 performs face authentication processing on the faces of the plurality of subjects on which the images are projected by the projector 3.

[0093] In this case, the projection image data may be one piece of image data in which images for each subject are included in the area corresponding to each subject. By doing so, one projector 3 can handle a large number of subjects 4. Also, each of the plurality of projectors 3 may project projection image data for one or more subjects 4. By doing so, parameters such as the focal length of each projector 3 can be adjusted individually, so that the projection image data can be accurately projected onto each subject.

[0094] In addition, the face authentication device 1 may be composed of a plurality of devices. For example, the face authentication device 1 may be composed of a server that generates projection image data and a server that performs face authentication processing.

[0095] (Second Embodiment) In the first embodiment, the face authentication device 1 generated one piece of projection image data from the image data of one image captured by the camera 2. In the second embodiment, the face authentication device 1 generates N (N is an integer of 1 or more) different projection image data (N types of projection image data) from the image data of one image captured by the camera 2. After the images of the N pieces of projection image data are projected onto the subject 4, the face authentication device 1 performs face authentication processing based on the N pieces of projected subject image data.

[0096] Note that the system configuration and the block configuration of the face authentication device 1 in the second embodiment are the same as those described in the first embodiment, and the description thereof is omitted.

[0097] FIG. 8 is a flowchart showing an operation example of the face authentication device 1 according to the second embodiment. The face authentication device 1 repeatedly executes the processing of the flowchart in FIG. 8 at a predetermined cycle, for example.

[0098] The face authentication device 1 receives image data from the camera 2 (S21).

[0099] The face authentication device 1 acquires attribute information regarding the face of the subject 4 from the image data received in S21 (S22). For example, the face authentication device 1 acquires face attribute information such as glasses, mask, face brightness, face color, and hair color from the face image of the subject 4.

[0100] The face authentication device 1 generates N different projection image data based on the attribute information of the face of the subject 4 acquired in S22 (S23). For example, the face authentication device 1 generates N projection image data with different brightnesses. Alternatively, the face authentication device 1 generates N projection image data with different colors. Alternatively, the face authentication device 1 generates N projection image data with different brightnesses and colors.

[0101] The face authentication device 1 initializes the variable i to 0 (S24).

[0102] The face authentication device 1 transmits the i-th projection image data to the projector 3 (S25).

[0103] Note that the projector 3 projects an image based on the projection image data transmitted from the face authentication device 1. The image of the projector 3 is projected onto the subject 4.

[0104] The face authentication device 1 receives projection subject image data from the camera 2 (S26).

[0105] The face recognition device 1 determines whether it has received N pieces of projected subject image data (S27). That is, the face recognition device 1 determines whether it has received the projected subject image data of the subject 4 onto which the images of N pieces of different projected image data are projected, from the camera 2.

[0106] When the face recognition device 1 determines that it has not received N pieces of projected subject image data from the camera 2 (``NO'' in S27), it adds 1 to the variable i (S28). Then, the face recognition device 1 transfers the process to S25, and transmits the i-th projected image data among the N pieces of projected image data generated in S23 to the projector 3.

[0107] On the other hand, when the face recognition device 1 determines that it has received N pieces of projected subject image data from the camera 2 (``YES'' in S27), it performs face recognition processing on each of the N pieces of projected subject image data received from the camera 2 (S29). The face recognition device 1 performs face recognition processing, for example, by the method described in S6 of FIG. 7.

[0108] The face recognition device 1 determines whether the degree of coincidence exceeds a threshold value in any of the N pieces of projected subject image data (S30).

[0109] When the face recognition device 1 determines that the degree of coincidence is equal to or greater than the threshold value in any of the projected subject image data (``YES'' in S30), it outputs an OK determination result to, for example, an attendance management system or an entrance / exit system (S31). Then, the face recognition device 1 ends the processing of the flowchart in FIG. 8.

[0110] On the other hand, when the face recognition device 1 determines that the degree of coincidence is not equal to or greater than the threshold value in any of the projected subject image data (``NO'' in S30), it outputs an NG determination result to, for example, an attendance management system or an entrance / exit system (S32). Then, the face recognition device 1 ends the processing of the flowchart in FIG. 8.

[0111] As described above, the image generation unit 11a of the face authentication device 1 generates N different projection image data based on the image data of the image captured by the camera 2. The face authentication unit 11b performs face authentication processing of the subject 4 based on the N projection subject image data after the images of the N projection image data are projected onto the subject 4.

[0112] In this way, since the face authentication device 1 generates N different projection image data based on the image data of the image captured by the camera 2, various images are projected onto the subject 4. Then, the face authentication device 1 performs face authentication of the subject 4 based on the projection subject image data of the subject 4 onto which various images are projected. Thereby, the face authentication device 1 can further improve the authentication accuracy of face authentication.

[0113] For example, when the accuracy of face authentication of the subject 4 is reduced due to a plurality of factors, there may be a case where the reduction in accuracy due to other factors cannot be solved even if the projection image data focusing on a certain factor is used. As an example, when the subject 4 is wearing glasses and a mask at the same time, it is difficult to suppress the white bleeding of the mask part even if the projection image data for glasses is used. On the other hand, if projection image data for which countermeasures against all factors are taken is generated, the accuracy of face authentication may conversely decrease. For example, when preventing a decrease in accuracy by reducing the brightness of the projection image data for both glasses and a mask, taking countermeasures against both will darken most of the face, resulting in insufficient light quantity and making it easier to fail in face authentication itself.

[0114] Therefore, in the present embodiment, by generating projection image data corresponding to each of the N factors, the possibility of successful face authentication of the subject 4 is increased. Note that among these N factors, a pattern including a plurality of factors simultaneously may also be included.

[0115] Further, in the present embodiment, after acquiring the N projection subject image data at once, face authentication of the subject 4 is performed. Therefore, when the determination results using the N projection subject image data are divided, it is possible to take measures to prevent misjudgment, such as determining the determination result by majority vote.

[0116] Note that in S30 of FIG. 8, the face authentication device 1 may output an OK determination result when it is determined that the degree of coincidence is equal to or greater than a threshold value in a predetermined number or more (for example, three or more) of the N projected subject image data among the N projected subject image data. Thereby, the face authentication device 1 can further improve the authentication accuracy of face authentication.

[0117] (Third Embodiment) In the third embodiment, the face authentication device 1 generates N (N is an integer of 1 or more) different projected image data based on the image data of the camera 2. Each time the image of one piece of projected image data is projected onto the subject 4, the face authentication device 1 receives the projected subject image data of the subject 4 from the camera 2 and performs face authentication processing on the subject 4.

[0118] Note that the system configuration and the block configuration of the face authentication device 1 in the third embodiment are the same as those of the system configuration and the block configuration described in the first embodiment, and the description thereof is omitted.

[0119] FIG. 9 is a flowchart showing an operation example of the face authentication device 1 according to the third embodiment. The face authentication device 1 repeatedly executes the processing of the flowchart in FIG. 9, for example, at a predetermined cycle. Note that the processing of S41 to S44 in FIG. 9 is the same as the processing of S21 to S24 described in FIG. 8, and the description thereof is omitted.

[0120] The face authentication device 1 transmits the i-th projected image data to the projector 3 (S45).

[0121] Note that the projector 3 projects an image based on the projected image data transmitted from the face authentication device 1. The image of the projector 3 is projected onto the subject 4.

[0122] The face authentication device 1 receives the projected subject image data from the camera 2 (S46).

[0123] The face authentication device 1 performs face authentication processing on the projected subject image data received in S46 (S47). That is, each time the face authentication device 1 projects one projected image onto the subject 4, it receives one piece of projected subject image data from the camera 2 and performs face authentication processing. Note that the face authentication device 1 performs face authentication processing by, for example, the method described in S6 of FIG. 7.

[0124] The face authentication device 1 determines whether or not the degree of coincidence obtained by the face authentication processing in S47 exceeds a threshold value (S48).

[0125] When the face authentication device 1 determines that the degree of coincidence is equal to or higher than the threshold value (``YES'' in S48), it outputs an OK determination result to, for example, an attendance management system or an entrance / exit system (S49). Then, the face authentication device 1 ends the processing of the flowchart in FIG. 9.

[0126] On the other hand, when the face authentication device 1 determines that the degree of coincidence is not equal to or higher than the threshold value (``NO'' in S48), it determines whether or not face authentication processing has been performed on the N pieces of projected subject image data (S50). That is, when the face authentication device 1 determines that the degree of coincidence is not equal to or higher than the threshold value, it determines whether or not face authentication processing has been performed on the N pieces of projected subject image data of the subject 4 onto which the images of the N pieces of projected image data generated in S43 have been projected.

[0127] When the face authentication device 1 determines that face authentication processing has not been performed on the N pieces of projected subject image data (``NO'' in S50), it increments the variable i by 1 (S51). Then, the face authentication device 1 shifts the processing to S45 and transmits the i-th projected image data among the N pieces of projected image data generated in S43 to the projector 3.

[0128] On the other hand, when the face authentication device 1 determines that face authentication processing has been performed on the N pieces of projected subject image data (``YES'' in S50), it outputs an NG determination result to, for example, an attendance management system or an entrance / exit system (S52). Then, the face authentication device 1 ends the processing of the flowchart in FIG. 9.

[0129] As described above, the image generation unit 11a of the face authentication device 1 generates N different projection image data based on the image data of the image captured by the camera 2. Each time an image of one piece of projection image data is projected onto the subject 4, the face authentication unit 11b receives the projected subject image data of the subject 4 from the camera 2 and performs a face authentication process on the subject 4.

[0130] In this way, since the face authentication device 1 generates N different projection image data based on the image data of the image captured by the camera 2, various images are projected onto the subject 4. Then, the face authentication device 1 performs face authentication on the subject 4 based on the projected subject image data of the subject 4 onto which various images are projected. Thereby, the face authentication device 1 can further improve the authentication accuracy of face authentication.

[0131] Also, each time an image of one piece of projection image data is projected onto the subject 4, the face authentication device 1 receives the projected subject image data of the subject 4 from the camera 2 and performs a face authentication process on the subject 4. Therefore, if face authentication processing is successful with any of the projected subject image data, the projection and face authentication processing of subsequent projection image data can be aborted. Thus, there may be cases where the determination result of face authentication can be output earlier than in the face authentication processing in the second embodiment.

[0132] In the second and third embodiments, N different projection image data are generated based on the attribute information of the face of the subject 4, but the evaluation of the attribute information of the face of the subject 4 may be omitted. For example, all types of projection image data that the face authentication device 1 can generate may be generated, and these may be used as N pieces of projection image data to perform the processing of the second and third embodiments. In this case, for parts where color or the like should be adjusted in the projection image data, predetermined default values may be used. For example, since it can be inferred without referring to the face attribute information that glasses appear in the upper half region of the face and a mask appears in the lower half region of the face, an image with reduced brightness in these regions can be used as the default projection image data.

[0133] Note that, in S48 of FIG. 9, the face recognition device 1 may output an OK determination result when it is determined that the degree of match is equal to or greater than a threshold value in a predetermined number or more (for example, three or more) of projection object images. Thereby, the face recognition device 1 can further improve the authentication accuracy of face recognition.

[0134] In the above-described embodiments, the notation "··· section" used for each component may be replaced with other notations such as "··· circuitry", "··· assembly", "··· device", "··· unit", or "··· module".

[0135] As described above, the embodiments have been described with reference to the drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive various modification examples or correction examples within the scope described in the claims. Such modification examples or correction examples are also understood to belong to the technical scope of the present disclosure. Further, within the scope not departing from the spirit of the present disclosure, the components in the embodiments may be arbitrarily combined.

[0136] The present disclosure can be realized by software, hardware, or software in cooperation with hardware. Each functional block used in the description of the above embodiments is realized as an LSI which is an integrated circuit, partially or entirely, and each process described in the above embodiments may be controlled partially or entirely by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip so as to include a part or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may also be referred to as an IC, a system LSI, a super LSI, or an ultra LSI.

[0137] The method of integrating circuits is not limited to LSI, and it may also be realized by dedicated circuits, general-purpose processors, or dedicated processors. Further, after manufacturing the LSI, an FPGA (Field Programmable Gate Array) that can be programmed, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0138] Furthermore, if a technology for integrating circuits that replaces LSI appears due to the progress of semiconductor technology or another derived technology, naturally, the integration of functional blocks may be performed using that technology. The application of biotechnology or the like is a possibility.

Industrial Applicability

[0139] The present disclosure is useful for a system that controls light applied to the face of a subject and performs face authentication.

Explanation of Signs

[0140] 1 Face authentication device 2 Camera 3 Projector 4 Subject 11 Control unit 11a Image generation unit 11b Face authentication unit 12 Storage unit 13 Communication unit 14 Input unit 15 Output unit

Claims

1. An image data receiving unit that receives image data of a subject from a camera; An image generation unit that generates projection image data of a projection image projected onto the subject based on the image data; A projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; A projected subject image data receiving unit that receives projected subject image data of the subject onto which the projection image is projected from the camera; A face authentication unit that performs face authentication processing of the subject based on the projected subject image data; Comprising: Based on the image data, the image generation unit generates the projection image data for correcting the brightness or color of the eye area of the subject wearing glasses; A face authentication device.

2. An image data receiving unit that receives image data of a subject from a camera; An image generation unit that generates projection image data of a projection image projected onto the subject based on the image data; A projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; A projected subject image data receiving unit that receives projected subject image data of the subject onto which the projection image is projected from the camera; A face authentication unit that performs face authentication processing of the subject based on the projected subject image data; Comprising: Based on the image data, the image generation unit generates the projection image data for correcting the brightness or color of the mouth area of the subject wearing a mask; A face authentication device.

3. An image data receiving unit that receives image data of a subject from a camera; An image generation unit that generates projection image data of a projection image projected onto the subject based on the image data; A projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; A projected subject image data receiving unit that receives projected subject image data of the subject onto which the projection image is projected from the camera; A face authentication unit that performs face authentication processing of the subject based on the projected subject image data; Comprising: Based on the image data, the image generation unit generates the projection image data so that the facial color or hair color of the subject becomes a predetermined color; A face authentication device.

4. An image data receiving unit that receives image data of a subject from a camera; An image generation unit that generates projection image data of a projection image to be projected onto the subject based on the image data; A projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; A projected subject image data reception unit that receives, from the camera, projected subject image data of the subject onto which the projection image has been projected; A face authentication unit that performs face authentication processing of the subject based on the projected subject image data; comprising; The image generation unit generates the projection image data based on background image data of the background of the subject included in the image data; Face authentication device.

5. An image data reception unit that receives image data of a subject captured by a camera; An image generation unit that generates projection image data of a projection image to be projected onto the subject based on the image data; A projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; A projected subject image data reception unit that receives, from the camera, projected subject image data of the subject onto which the projection image has been projected; A face authentication unit that performs face authentication processing of the subject based on the projected subject image data; comprising; The image generation unit generates N (N is an integer of 1 or more) different pieces of the projection image data based on the image data; After the face authentication unit receives, from the camera, N pieces of the projected subject image data in which the images of the N pieces of the projection image data have been respectively projected onto the subject, the face authentication unit performs face authentication processing of the subject based on the N pieces of the projected subject image data; Face authentication device.

6. An image data reception unit that receives image data of a subject captured by a camera; An image generation unit that generates projection image data of a projection image to be projected onto the subject based on the image data; A projection image data transmission unit that transmits the projection image data to a projector that projects the projection image onto the subject; A projected subject image data reception unit that receives, from the camera, projected subject image data of the subject onto which the projection image has been projected; A face authentication unit that performs face authentication processing of the subject based on the projected subject image data; comprising; The image generation unit generates N (N is an integer of 1 or more) different pieces of the projection image data based on the image data; Each time an image of the one piece of the projection image data is projected onto the subject, the face authentication unit receives the projection subject image data of the subject from the camera and performs face authentication processing on the subject. Face authentication device.

7. Receiving image data in which a subject is reflected from a camera, generating projection image data of a projection image to be projected onto the subject based on the image data, transmitting the projection image data to a projector that projects the projection image onto the subject, receiving projection subject image data of the subject onto which the projection image has been projected from the camera, performing face authentication processing on the subject based on the projection subject image data, generating the projection image data for correcting the brightness or color of the eye area of the subject wearing glasses based on the image data. Face authentication method.

8. Receiving image data in which a subject is reflected from a camera, generating projection image data of a projection image to be projected onto the subject based on the image data, transmitting the projection image data to a projector that projects the projection image onto the subject, receiving projection subject image data of the subject onto which the projection image has been projected from the camera, performing face authentication processing on the subject based on the projection subject image data, generating the projection image data for correcting the brightness or color of the mouth area of the subject wearing a mask based on the image data. Face authentication method.

9. Receiving image data in which a subject is reflected from a camera, generating projection image data of a projection image to be projected onto the subject based on the image data, transmitting the projection image data to a projector that projects the projection image onto the subject, receiving projection subject image data of the subject onto which the projection image has been projected from the camera, performing face authentication processing on the subject based on the projection subject image data, generating the projection image data so that the facial color or hair color of the subject becomes a predetermined color based on the image data. Face authentication method.

10. Receiving image data in which a subject is reflected from a camera, generating projection image data of a projection image to be projected onto the subject based on the image data, transmitting the projection image data to a projector that projects the projection image onto the subject, receiving projection subject image data of the subject onto which the projection image has been projected from the camera, Perform face authentication processing of the subject based on the projected subject image data. Generate the projection image data based on the background image data of the background of the subject included in the image data. Face authentication method.

11. Receive image data in which a subject is reflected from a camera. Generate projection image data of a projection image to be projected onto the subject based on the image data. Transmit the projection image data to a projector that projects the projection image onto the subject. Receive projection subject image data of the subject onto which the projection image is projected from the camera. Perform face authentication processing of the subject based on the projection subject image data. Generate N (N is an integer of 1 or more) different pieces of the projection image data based on the image data. After receiving N pieces of projection subject image data in which images of the N pieces of projection image data are respectively projected onto the subject from the camera, perform face authentication processing of the subject based on the N pieces of projection subject image data. Face authentication method.

12. Receive image data in which a subject is reflected from a camera. Generate projection image data of a projection image to be projected onto the subject based on the image data. Transmit the projection image data to a projector that projects the projection image onto the subject. Receive projection subject image data of the subject onto which the projection image is projected from the camera. Perform face authentication processing of the subject based on the projection subject image data. Generate N (N is an integer of 1 or more) different pieces of the projection image data based on the image data. Each time an image of one piece of the projection image data is projected onto the subject, receive the projection subject image data of the subject from the camera and perform face authentication processing of the subject. Face authentication method.

Images