Authentication device, program, and authentication method

Abstract

To provide an authentication device which uses image data on a spherical surface without requiring a person to cower or reach up in front of a camera so as to bring a face to approach the camera, or setting multiple kinds of devices according to a body height, a program, and an authentication method.SOLUTION: In an authentication system 10 including cameras 210, 220 as fish-eye cameras and a sensor 230 installed near a gate 40, an authentication device 100 which communicates with the cameras over a network 20 comprises: a data acquisition unit which acquires spherical data including a video of a person captured by the fish-eye cameras; and an authentication execution unit which extracts a partial image of a part of a face of the person from the spherical data, adjusts a direction of the face of the person in the partial image, executes distortion correction processing for correcting distortion in the partial image due to the fish-eye cameras, and then executes face authentication on the partial image by means of a convolutional neural network.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an authentication device, a program, and an authentication method. [Background technology]

[0002] Patent Document 1 describes a technology for acquiring a face image of a user and authenticating the user using information on feature points included in the face image. Non-Patent Document 1 describes a spherical convolutional neural network (SCNN) specialized for image data on a sphere. [Prior art document] [Patent documents] [Patent Document 1] JP 2021-170205 A [Non-patent literature] [Non-patent document 1] Taco S. Cohen, Mario Geiger, Jonas Koehler, Max Welling, Spherical CNNs, ICLR 2018 Summary of the Invention [Means for solving the problem]

[0003] According to an embodiment of the present invention, there is provided an authentication device. The authentication device may include a data acquisition unit that acquires spherical data including an image of a person captured by a fisheye camera. The authentication device may include an authentication execution unit that executes face authentication using a spherical convolutional neural network on the spherical data.

[0004] The authentication device may include a tracking unit that tracks a position of a person included in the spherical data continuously acquired by the data acquisition unit. The authentication device may include an identification unit that identifies a person for which authentication processing is to be performed based on a tracking result by the tracking unit. The authentication execution unit may perform face authentication using the spherical convolutional neural network on the spherical data for the person identified by the identification unit. The authentication device may include a determination unit that determines whether to perform face authentication using the spherical convolutional neural network or face authentication using a convolutional neural network for the person identified by the identification unit. The authentication execution unit may perform face authentication using the spherical convolutional neural network on the spherical data when the determination unit determines to perform face authentication using the spherical convolutional neural network. When the determination unit determines to perform face authentication using the convolutional neural network, the authentication execution unit may extract a partial image of a part of the face of the person identified by the identification unit from the spherical data, adjust the orientation of the face of the person in the partial image, perform distortion correction processing on the partial image to correct distortion caused by a fisheye camera, and then perform face authentication on the partial image using the convolutional neural network. The convolutional neural network may be a convolutional neural network that is not specialized for a sphere. The convolutional neural network may be a convolutional neural network that is not the spherical convolutional neural network.

[0005] In any of the authentication devices, the determination unit may determine to perform face authentication using the spherical convolutional neural network when a security level required for the face authentication is a first level, and may determine to perform face authentication using the convolutional neural network when the security level required for the face authentication is a second level higher than the first level.

[0006] In any of the authentication devices, the determination unit may determine to perform face authentication using the spherical convolutional neural network when the moving speed of the person to be subjected to the authentication process is faster than a predetermined threshold, and may determine to perform face authentication using the convolutional neural network when the moving speed is slower than the threshold.

[0007] In any of the authentication devices, the determination unit may determine to perform face authentication using the spherical convolutional neural network when a value related to the number of people within an imaging range of the fisheye camera is greater than a predetermined threshold, and may determine to perform face authentication using the convolutional neural network when the value related to the number of people within an imaging range of the fisheye camera is less than the threshold.

[0008] In any of the authentication devices, the data acquisition unit may acquire first spherical data including an image captured by a first fisheye camera and second spherical data including an image captured by a second fisheye camera positioned at a higher position than the first fisheye camera, and the authentication execution unit may perform face authentication using the spherical convolutional neural network on at least one of the first spherical data and the second spherical data.

[0009] According to an embodiment of the present invention, there is provided an authentication device. The authentication device may include a data acquisition unit that acquires spherical data including an image of a person captured by a fisheye camera. The authentication device may include an authentication execution unit that extracts a partial image of a face of the person from the spherical data, adjusts the orientation of the face of the person in the partial image, performs distortion correction processing on the partial image to correct distortion caused by the fisheye camera, and then executes face authentication on the partial image using a convolutional neural network.

[0010] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the authentication device.

[0011] According to one embodiment of the present invention, there is provided an authentication method executed by a computer. The authentication method may include a spherical data acquisition step of acquiring spherical data including an image of a person captured by a fisheye camera. The authentication method may include an authentication execution step of executing face recognition by a spherical convolutional neural network on the spherical data.

[0012] According to one embodiment of the present invention, there is provided an authentication method executed by a computer. The authentication method may include a spherical data acquisition step of acquiring spherical data including an image of a person captured by a fisheye camera. The authentication method may include an authentication execution step of extracting a partial image of a face of the person from the spherical data, adjusting a direction of the face of the person in the partial image, performing distortion correction processing on the partial image to correct distortion caused by the fisheye camera, and then performing face authentication on the partial image using a convolutional neural network.

[0013] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0014] [Figure 1] 1 illustrates a schematic diagram of an example authentication system 10. [Diagram 2] 2 shows an example of a processing flow by the authentication device 100. [Diagram 3] FIG. 2 is an explanatory diagram for describing pre-processing of CNN by the authentication device 100. [Figure 4] 2 illustrates an example of a functional configuration of the authentication device 100. [Diagram 5] 1 shows an example of a hardware configuration of a computer 1200 that functions as the authentication device 100. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] In conventional face authentication systems, a person needs to bend or stretch in front of the camera to bring their face closer to the camera. In addition, it is necessary to set up multiple types of devices for tall and short people. In the authentication system 10 according to the present embodiment, in order to alleviate this problem, a face authentication method is adopted in which a camera equipped with a fisheye lens is used to acquire a wide-angle image. In the authentication system 10, for example, a fisheye camera (or a sensor for measuring distance, if necessary) is installed on the ceiling and on a stand. A distorted image is obtained by the fisheye camera. The image is converted to spherical data based on the projection law (it is preferable to obtain spherical data directly from the fisheye camera). The position of the person's face is identified from the data on the sphere (by attaching a bounding box, for example) and tracking is performed. If the tracked person crosses a certain boundary line, or if the distance can be estimated and the closest person can be processed directly with SCNN, face authentication is performed using SCNN. If there is a problem with using SCNN, the range of the bounding box is locally defished (projected onto a plane using the central projection law) and face authentication is performed using a normal CNN. In the authentication system 10, the result of the facial authentication may be used to control whether or not the gate is opened, or to notify security personnel or an investigative agency.

[0016] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0017] 1 illustrates an example of an authentication system 10. The authentication system 10 may be a system that performs face authentication of a person.

[0018] The system 10 includes an authentication device 100. The authentication system 10 may include a camera 210. The authentication system 10 may include a camera 220. The authentication system 10 may include a distance measurement sensor 230. Of the cameras 210, 220, and 230, the authentication system 10 may include only the camera 210 and the camera 220, or may include only the camera 210 and the distance measurement sensor 230, or may include only the camera 220 and the distance measurement sensor 230. In another embodiment, the authentication system 10 may include only the camera 210 or only the camera 220 of the cameras 210, 220, and 230.

[0019] The cameras 210 and 220 may be fisheye cameras having fisheye lenses. The camera 220 may be disposed at a higher position than the camera 210. For example, the camera 210 is disposed on the lower side, and the camera 220 is disposed on the upper side. As a specific example, as illustrated in FIG. 1, the camera 210 is disposed on a platform installed on the lower side, and the camera 220 is disposed on the ceiling side. The cameras 210 and 220 may be disposed so that no blind spots are generated on the faces of people passing near the cameras 210 and 220. The camera 210 may be an example of a first fisheye camera, and the camera 220 may be an example of a second fisheye camera.

[0020] The distance measurement sensor 230 measures the distance to a person. The distance measurement sensor 230 may be installed near the camera 210 or the camera 220. The distance measurement sensor 230 may also be installed at the gate 40 or in the vicinity of the gate 40.

[0021] The authentication device 100 acquires spherical data including an image of a person captured by the camera 210. For example, the authentication device 100 receives the spherical data generated by the camera 210 from the camera 210. For example, the authentication device 100 receives data of an image of a person captured by the camera 210 from the camera 210, and generates the spherical data from the received data.

[0022] The authentication device 100 and the camera 210 may communicate with each other via a network 20, as exemplified in Fig. 1. The authentication device 100 may communicate with the camera 210 without going through the network 20. For example, the authentication device 100 is placed near the location where the camera 210 is installed, and communicates with the camera 210 via a wired or wireless connection.

[0023] The authentication device 100 acquires spherical data including an image of a person captured by the camera 220. For example, the authentication device 100 receives the spherical data generated by the camera 220 from the camera 220. For example, the authentication device 100 receives data of an image of a person captured by the camera 220 from the camera 220, and generates the spherical data from the received data.

[0024] The authentication device 100 and the camera 220 may communicate with each other via a network 20, as exemplified in Fig. 1. The authentication device 100 may communicate with the camera 220 without going through the network 20. For example, the authentication device 100 is placed near the location where the camera 220 is installed, and communicates with the camera 220 via a wired or wireless connection.

[0025] The authentication device 100 acquires the distance to the person measured by the distance measurement sensor 230. The authentication device 100 receives the distance to the person measured by the distance measurement sensor 230 from the distance measurement sensor 230.

[0026] The authentication device 100 and the distance measuring sensor 230 may communicate with each other via a network 20, as illustrated in Fig. 1. The authentication device 100 may communicate with the distance measuring sensor 230 without using the network 20. For example, the authentication device 100 is placed near the location where the distance measuring sensor 230 is installed, and communicates with the camera distance measuring sensor 230 via a wired or wireless connection.

[0027] The network 20 may include the Internet. The network 20 may include a LAN (Local Area Network). The network 20 may include a mobile communication network. The mobile communication network may conform to any of the following communication methods: 3G (3rd Generation) communication method, LTE (Long Term Evolution) communication method, 5G (5th Generation) communication method, and 6G (6th Generation) communication method and later.

[0028] The authentication device 100 may be connected to the network 20 by wire. The authentication device 100 may be connected to the network 20 wirelessly. The authentication device 100 may be connected to the network 20 via a wireless base station. The authentication device 100 may be connected to the network 20 via a Wi-Fi (registered trademark) access point.

[0029] The camera 210 may be connected to the network 20 by wire. The camera 210 may be connected to the network 20 wirelessly. The camera 210 may be connected to the network 20 via a wireless base station. The camera 210 may be connected to the network 20 via a Wi-Fi access point.

[0030] The camera 220 may be connected to the network 20 by wire. The camera 220 may be connected to the network 20 wirelessly. The camera 220 may be connected to the network 20 via a wireless base station. The camera 220 may be connected to the network 20 via a Wi-Fi access point.

[0031] The distance measurement sensor 230 may be connected to the network 20 by wire. The distance measurement sensor 230 may be connected to the network 20 by wireless. The distance measurement sensor 230 may be connected to the network 20 via a wireless base station. The distance measurement sensor 230 may be connected to the network 20 via a Wi-Fi access point.

[0032] The authentication device 100 performs face authentication of a person captured by the camera 210 and the camera 220. Conventionally, face authentication using a normal CNN has been performed on a planar image captured by a normal camera, rather than a spherical image captured by a fisheye camera. There is a demand for authentication processing on spherical images, and a method of converting a spherical image into a planar image and then applying a normal CNN is considered, but various distortions occur due to the conversion, which causes problems in authentication accuracy, etc. In addition, the calculation in a normal CNN has translational symmetry and is therefore strong against parallel movements in the vertical and horizontal directions, but does not have rotational symmetry, so in the case of an image in which the orientation of the subject rotates, such as a spherical image, it may erroneously determine that the same object is a different object.

[0033] In contrast, SCNN is specialized for spherical data and has the characteristic of being robust against rotation. In Non-Patent Document 1, tests were conducted using rotated MNIST to compare the performance of normal CNN and SCNN. When the training data and test data were rotated, the accuracy rate of the method using CNN was 11%, which is almost the same as answering at random, while the accuracy rate of SCNN was 95%. This is because CNN has translational symmetry and is strong against up / down and left / right parallel translation, but has no rotational symmetry and is weak against rotational transformation, whereas SCNN has rotational symmetry.

[0034] The authentication device 100 according to this embodiment performs face authentication using SCNN, which makes it possible to achieve sufficient authentication accuracy even for spherical data.

[0035] The authentication device 100 may selectively use face recognition by SCNN and face recognition by CNN.

[0036] 2 shows an example of a processing flow by the authentication device 100. Here, a processing flow will be described in which the authentication device 100 detects a person approaching the gate 40 from images captured by the cameras 210 and 220, performs face authentication on the person approaching the gate 40, and executes a process to open the gate 40 if authentication is successful.

[0037] In step (sometimes abbreviated to S) 102, authentication device 100 acquires spherical data including images of a person captured by camera 210 and camera 220. In S104, authentication device 100 identifies the position of the person's face from the spherical data and tracks it.

[0038] In S106, the authentication device 100 determines whether to perform face authentication of a person based on the tracking status of the person. For example, the authentication device 100 determines to perform face authentication of the person when the person being tracked crosses a boundary line virtually set in front of the gate 40. For example, the authentication device 100 determines to perform face authentication of the person closest to the gate 40 or the camera 210 and the camera 220 among the multiple people included in the spherical data.

[0039] If it is determined that face authentication is not to be performed (NO in S106), the process returns to S102, and if it is determined that face authentication is to be performed (YES in S106), the process proceeds to S108. In S108, the authentication device 100 determines whether face authentication by SCNN or face authentication by CNN is to be performed. If it is determined that face authentication by SCNN is to be performed, the process proceeds to S110, and if it is determined that face authentication by CNN is to be performed, the process proceeds to S112.

[0040] In S110, the authentication device 100 performs face authentication using SCNN on the spherical data acquired in S102.

[0041] In S112, the authentication device 100 extracts a partial image of the face of the person to be authenticated from the spherical data acquired in S102. In S114, the authentication device 100 adjusts the orientation of the person's face in the partial image extracted in S112. For example, the authentication device 100 detects the person's eyes, nose, and mouth, and adjusts the orientation of the person's face so that the chin is on the lower side and the top of the head is on the upper side.

[0042] In S116, the authentication device 100 locally performs Defish eye on the spherical data. Defish eye may be a distortion correction process that corrects distortion caused by a fisheye camera. The authentication device 100 may perform Defish eye on the partial image in which the orientation of the person's face has been adjusted in S114.

[0043] In S118, the authentication device 100 performs face authentication using CNN on the partial image on which Defish eye was performed in S116.

[0044] In S120, the authentication device 100 executes a process according to the authentication result. When the authentication of the person to be authenticated is OK, the authentication device 100 controls the gate 40 to open the gate 40, and when the authentication is NG, the authentication device 100 does not open the gate 40.

[0045] In S122, the authentication device 100 determines whether or not to end the authentication process. For example, the authentication device 100 determines to end the authentication process when receiving an end instruction from an administrator or the like of the authentication system 10. If the authentication process is not to be ended, the process returns to S102.

[0046] FIG. 3 is an explanatory diagram for explaining the pre-processing of the CNN by the authentication device 100. The spherical data 300 includes a face image in a state rotated and distorted with respect to the normal face orientation in which the chin is located at the bottom and the top of the head is located at the top, as exemplified in FIG. 3. The authentication device 100 cuts out a partial image 310 of the person's face from the spherical data 300, adjusts the orientation, converts the image into a planar image according to the projection rule with the area in which the face is captured as the center, and inputs the data into a CNN for normal face authentication to perform authentication. This is an application of the fact that, for example, it is difficult to properly represent a world map on a planar diagram while maintaining the distance and angle, but a map of Japan can be represented on a planar map with better accuracy.

[0047] As a specific example, the authentication device 100 projects a face image onto a tangent plane of a sphere near the person's nose in the partial image 310 (a normal camera central projection method) as shown in Fig. 3. The authentication device 100 uses something like a bounding box, and corrects distortion of the partial image 310 by tracking and using a coordinate system centered on the bounding box for data within that range.

[0048] In conventional face authentication systems, people to be authenticated line up in front of a camera, turn to face the camera in order, and adjust the direction of their faces to be photographed, thereby performing face authentication. In contrast, the authentication device 100 uses a fisheye camera to increase the freedom of the position of people to be authenticated, eliminating the need to line up in front of the camera and turn to face the camera, and by executing SCNN or executing CNN after preprocessing, it is possible to achieve face authentication with sufficient accuracy even for spherical data from a fisheye camera.

[0049] 4 illustrates an example of a functional configuration of the authentication device 100. The authentication device 100 includes a storage unit 102, a data acquisition unit 104, a tracking unit 106, an identification unit 108, a determination unit 110, an authentication execution unit 112, and an authentication result processing unit 114. The storage unit 102 stores various data.

[0050] The data acquisition unit 104 acquires various data. The data acquisition unit 104 stores the acquired data in the storage unit 102.

[0051] The data acquisition unit 104 acquires spherical data including a video of a person captured by a fisheye camera. The data acquisition unit 104 receives, for example, spherical data generated by the camera 210 from the camera 210. The data acquisition unit 104 receives, for example, data of a video of a person captured by the camera 210 from the camera 210, and generates spherical data from the received data based on the projection rule. The data acquisition unit 104 receives, for example, spherical data generated by the camera 220 from the camera 220. The data acquisition unit 104 receives, for example, data of a video of a person captured by the camera 220 from the camera 220, and generates spherical data from the received data based on the projection rule.

[0052] The data acquisition unit 104 acquires the distance to a person within the imaging range of the fisheye camera based on the fisheye camera, the distance measurement sensor 230, etc. The data acquisition unit 104 calculates the distance to a person based on the cameras 210 and 220, for example, using images captured by the cameras 210 and 220. The data acquisition unit 104 may calculate the distance to a person by using the cameras 210 and 220 as a stereo camera. The data acquisition unit 104 acquires, for example, the distance between the distance measurement sensor 230 and a person measured by the distance measurement sensor 230 from the distance measurement sensor 230.

[0053] The tracking unit 106 tracks the position of a human face included in the spherical data continuously acquired by the data acquisition unit 104. The tracking unit 106 may track the human face included in the spherical data by attaching a bounding box to the human face. The tracking unit 106 may identify the position of the human face included in the spherical data by using SCNN on the spherical data. The tracking unit 106 may identify the position of the human face included in the spherical data by using various methods having rotational symmetry, such as a capsule network.

[0054] The tracking unit 106 may determine the moving speed of the person included in the spherical data and store it in the storage unit 102. The tracking unit 106 may determine the moving speed of the person included in the spherical data by analyzing the spherical data continuously acquired by the data acquisition unit 104.

[0055] The tracking unit 106 may calculate a value related to the number of people included in the spherical data and store it in the storage unit 102. For example, the tracking unit 106 stores number of people data indicating the number of people included in the spherical data in the storage unit 102. For example, the tracking unit 106 stores population density information indicating the population density of people included in the spherical data in the storage unit 102.

[0056] The identification unit 108 identifies a person who is to undergo authentication processing based on the tracking result by the tracking unit 106.

[0057] For example, the identification unit 108 sets a virtual boundary line around the camera 210 and the camera 220. As a specific example, when authenticating a person passing through a gate, the identification unit 108 sets a virtual boundary line in front of the gate. Then, based on the tracking result by the tracking unit 106, when a person passes through the boundary line, the identification unit 108 identifies the person as a target for which authentication processing is to be executed.

[0058] Furthermore, the identification unit 108 identifies a person to be subjected to authentication processing, for example, based on the distance to a person within the imaging range of a fisheye camera based on the fisheye camera or the distance measuring sensor 230, etc., acquired by the data acquisition unit 104. For example, the identification unit 108 identifies a person who is closest among the people included in the imaging range of the fisheye camera as a person to be subjected to authentication processing. The identification unit 108 may identify a person whose distance is shorter than a preset distance threshold among multiple people as a person to be subjected to authentication processing.

[0059] The determination unit 110 determines whether to perform face recognition by SCNN or face recognition by CNN for the person identified by the identification unit 108. That is, the determination unit 110 may determine whether to perform face recognition using a CNN specialized for a sphere or face recognition using a CNN not specialized for a sphere for the person identified by the identification unit 108. Specialized for a sphere may mean that it supports distortion and rotation, and not specialized for a sphere may mean that it does not support distortion and rotation. The face recognition by CNN here may be face recognition using a CNN not specialized for a sphere, for example, face recognition using a CNN other than SCNN.

[0060] The authentication execution unit 112 executes face authentication using SCNN on the spherical data acquired by the data acquisition unit 104. Since SCNN is specialized for spherical surfaces, it is possible to execute face authentication with high accuracy on people included in the spherical data acquired by the data acquisition unit 104.

[0061] The authentication execution unit 112 may execute face authentication using SCNN for the person identified by the identification unit 108. By executing SCNN on spherical data including a person identified by the identification unit 108 among the people tracked by the tracking unit 106, it is possible to authenticate, with high accuracy, a person approaching a gate, for example, among a plurality of people included in a wide range captured by the fisheye camera.

[0062] When the determination unit 110 determines that face authentication using SCNN is to be performed, the authentication execution unit 112 may perform face authentication using SCNN on the spherical data. When the determination unit 110 determines that face authentication using CNN is to be performed, the authentication execution unit 112 may perform preprocessing on the spherical data. For example, the authentication execution unit 112 extracts a partial image of a part of a person's face identified by the identification unit 108 from the spherical data, adjusts the orientation of the person's face in the partial image, and performs distortion correction processing on the partial image to correct distortion caused by the fisheye camera. The authentication execution unit 112 may perform face authentication using CNN on the partial image after the preprocessing.

[0063] For example, when the security level required for the face authentication of the authentication system 10 is a first level, the determination unit 110 determines to execute face authentication by SCNN, and when the security level required for the face authentication of the authentication system 10 is a second level higher than the first level, the determination unit 110 determines to execute face authentication by CNN. Between face authentication by SCNN and CNN to which preprocessing by the authentication execution unit 112 is added, the former has a lower processing load than the latter, while the latter can achieve higher authentication accuracy than the former. When a higher security level is required, high authentication accuracy is required even if the processing load is high, so the determination unit 110 can make a determination appropriate to such a situation.

[0064] The determination unit 110 may determine whether to perform face authentication by SCNN or by CNN based on the moving speed of the person to be authenticated, which is determined by the tracking unit 106. For example, when the moving speed of the person to be authenticated is faster than a predetermined threshold, the determination unit 110 determines to perform face authentication by SCNN, and when the moving speed is slower than the threshold, determines to perform face authentication by CNN. In this way, when the moving speed of the person to be authenticated is fast, by performing face authentication by SCNN, which has a lower processing load than face authentication by CNN and can output a result faster, it is possible to reduce the possibility of occurrence of a situation in which the moving speed of the person to be authenticated is too fast to be authenticated in time, resulting in an error, and when the moving speed of the person to be authenticated is slow, it is possible to realize high authentication accuracy.

[0065] The determination unit 110 determines whether to perform face authentication by SCNN or by CNN, for example, based on a value related to the number of people within the imaging range of the fisheye camera. For example, when the value related to the number of people within the imaging range of the fisheye camera is greater than a predetermined threshold, the determination unit 110 determines to perform face authentication by SCNN, and when the value is less than the threshold, determines to perform face authentication by CNN. In this way, when the number of target people is large, by performing face authentication by SCNN, which has a lower processing load than face authentication by CNN, it is possible to reduce the possibility of a situation occurring in which authentication is not completed in time for any person, resulting in an error, and when the number of target people is small, it is possible to achieve high authentication accuracy.

[0066] When the authentication system 10 includes the camera 210 and the camera 220, the data acquisition unit 104 acquires first spherical data including an image captured by the camera 210 and second spherical data including an image captured by the camera 220. When performing face authentication using SCNN, the authentication execution unit 112 executes face authentication on at least one of the first spherical data and the second spherical data.

[0067] For example, the authentication execution unit 112 executes face authentication using both the first spherical data and the second spherical data. As a specific example, the authentication execution unit 112 executes face authentication by SCNN on the first spherical data and executes face authentication by SCNN on the second spherical data, and determines a final authentication result by combining the results. As another specific example, the authentication execution unit 112 executes face authentication by SCNN on the first spherical data and executes face authentication by SCNN on the second spherical data, and adopts the result with higher accuracy as the final authentication result.

[0068] For example, the authentication execution unit 112 selects one of the first spherical data and the second spherical data, and executes face authentication using the selected one. For example, the authentication execution unit 112 selects the first spherical data or the second spherical data depending on how the face of the person to be authenticated is captured from the first spherical data or the second spherical data. As a specific example, the authentication execution unit 112 selects the first spherical data or the second spherical data, which includes a larger range of the face of the person to be authenticated. Also, for example, the authentication execution unit 112 selects the first spherical data or the second spherical data, which includes a larger number of the eyes, nose, and mouth features of the person to be authenticated.

[0069] 5 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the authentication device 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the device according to the present embodiment, and / or cause the computer 1200 to execute a process or steps of the process according to the present embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0070] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0071] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0072] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0073] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0074] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0075] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0076] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0077] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0078] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0079] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0080] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0081] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0082] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0083] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0084] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that the process must be performed in this order.

[0085] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0086] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that the process must be performed in this order. [Explanation of symbols]

[0087] 10 system, 20 network, 40 gate, 100 authentication device, 102 memory unit, 104 data acquisition unit, 106 tracking unit, 108 identification unit, 110 judgment unit, 112 authentication execution unit, 114 authentication result processing unit, 210 camera, 220 camera, 230 distance measurement sensor, 300 spherical data, 310 partial image, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. A data acquisition unit that acquires spherical data including an image of a person captured by a fisheye camera; a tracking unit that tracks a position of a person included in the spherical data continuously acquired by the data acquisition unit; an identification unit that identifies a person who is to undergo authentication processing based on a tracking result by the tracking unit; a determination unit that determines whether to perform face recognition using a spherical convolutional neural network or a convolutional neural network for the person identified by the identification unit; an authentication execution unit that executes face authentication using the spherical convolutional neural network on the spherical data for the person identified by the identification unit when the determination unit determines that face authentication using the spherical convolutional neural network is to be executed; An authentication device comprising:

2. 2. The authentication device according to claim 1, wherein, when the determination unit determines to perform face authentication using the convolutional neural network, the authentication execution unit extracts from the spherical data a partial image of a portion of the face of the person identified by the identification unit, adjusts a direction of the face of the person in the partial image, performs distortion correction processing on the partial image to correct distortion caused by a fisheye camera, and then performs face authentication on the partial image using the convolutional neural network.

3. The authentication device of claim 2 , wherein the convolutional neural network is a non-sphere-specific convolutional neural network.

4. The authentication device according to claim 3 , wherein the convolutional neural network is a convolutional neural network that is not a spherical convolutional neural network.

5. 3. The authentication device according to claim 2, wherein the determination unit determines to perform face authentication using the spherical convolutional neural network when a security level required for the face authentication is a first level, and determines to perform face authentication using the convolutional neural network when the security level required for the face authentication is a second level higher than the first level.

6. 3. The authentication device according to claim 2, wherein the determination unit determines to perform face authentication using the spherical convolutional neural network when a moving speed of the person to be subjected to the authentication process is faster than a predetermined threshold, and determines to perform face authentication using the convolutional neural network when the moving speed is slower than the threshold.

7. 3. The authentication device according to claim 2, wherein the determination unit determines to perform face authentication using the spherical convolutional neural network when a value related to the number of people within an imaging range of the fisheye camera is greater than a predetermined threshold, and determines to perform face authentication using the convolutional neural network when the value related to the number of people within an imaging range of the fisheye camera is less than the threshold.

8. The data acquisition unit acquires first spherical data including an image captured by a first fisheye camera and second spherical data including an image captured by a second fisheye camera disposed at a higher position than the first fisheye camera, The authentication device according to claim 1 , wherein the authentication execution unit executes face authentication using the spherical convolutional neural network on at least one of the first spherical data and the second spherical data.

9. A program for causing a computer to function as the authentication device according to any one of claims 1 to 8.

10. 1. A computer implemented authentication method comprising: A spherical data acquisition step of acquiring spherical data including an image of a person captured by a fisheye camera; a tracking step of tracking a position of a person included in the spherical data continuously acquired in the spherical data acquisition step; an identification step of identifying a person who is to undergo authentication processing based on a tracking result in the tracking step; a determination step of determining whether to perform face recognition using a spherical convolutional neural network or a convolutional neural network for the person identified in the identification step; an authentication execution step of executing face authentication using the spherical convolutional neural network on the spherical data for the person identified in the identification step when it is determined in the determination step that face authentication using the spherical convolutional neural network is to be executed; An authentication method comprising:

Images