Image processing device, image processing system, image processing method and program
The image processing device employs a two-step authentication process to address the challenge of rapid facial recognition at railway gates, achieving efficient and tag-free authentication by combining preliminary one-to-many and one-to-few methods.
Patent Information
- Application Number
- JP2020160447
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-09-25
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-09-25
AI Technical Summary
Existing facial recognition technologies for rapid authentication, such as those used at railway station gates, face challenges in completing one-to-many authentication within the required short time frame without the need for wireless tags, which are restrictive for users.
An image processing device performs a two-step authentication process, including a preliminary one-to-many authentication followed by a faster one-to-few authentication, using face image information from multiple captured images to quickly verify individuals without requiring wireless tags.
This approach allows for rapid and accurate authentication processing, meeting the time constraints and eliminating the need for wireless tags, thereby enhancing user convenience and system efficiency.
Smart Images

Figure 0007576950000001 
Figure 0007576950000002 
Figure 0007576950000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an image processing device and a technique related thereto. [Background technology]
[0002] There is a technology for authenticating a person based on a facial image in a captured image (Patent Document 1, etc.). In detail, there is a technology for analyzing a captured image, extracting a facial image of a person from the captured image, and authenticating the person based on the facial image.
[0003] When identifying one person from among many people, one-to-many authentication (also called one-to-N authentication) is generally performed. One-to-many authentication is a technology in which, for example, one person (more specifically, his / her face image, etc.) extracted from a captured image is used as a matching source, and many people (more specifically, their face images, etc.) registered in advance are used as matching targets, and the one matching source person is identified by matching the one matching source with the many matching targets. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2008-052549 A Summary of the Invention [Problem to be solved by the invention]
[0005] Meanwhile, the above-mentioned face recognition processing technology is expected to be applied to authentication near ticket gates at railway stations, for example.
[0006] However, when authenticating a person near a gate installed at a train station ticket barrier, etc., extremely high-speed processing is required. For example, authentication processing based on an image captured just before a person walking toward a gate reaches the gate must be completed within an extremely short period (several hundred milliseconds) before the person reaches the gate.
[0007] It is not necessarily realistic to complete the above-mentioned one-to-many authentication within such a very short period of time.
[0008] On the other hand, in the technology of Patent Document 1, identification information in a wireless tag carried by a subject is read, and a matching target is identified based on the identification information. Then, a so-called one-to-one authentication process is performed in which a feature amount (matching target feature amount) registered in advance for the matching target is matched with a feature amount (matching source feature amount) acquired from a captured image. According to this technology, since one-to-one authentication process is performed after a matching target is identified based on identification information, it is possible to speed up the process. In detail, faster processing is realized than when one-to-many authentication process is performed based on a captured image taken just before a gate.
[0009] However, when using the technology of Patent Document 1, all users must carry a wireless tag. A technology that requires the possession of a wireless tag is restrictive for users.
[0010] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide an image processing device capable of implementing rapid authentication processing without requiring the user to carry a wireless tag, and to provide techniques related thereto. [Means for solving the problem]
[0011] In order to solve the above problem, an image processing device according to the present invention includes a control unit that acquires at least one first captured image and a second captured image captured after the at least one first captured image, and the control unit executes a first authentication process to authenticate a person by comparing face image information of the person included in the at least one first captured image with face image information of a large number of people registered in advance, and executes a second authentication process to authenticate the person to be authenticated by comparing face image information of the person to be authenticated included in the second captured image with face image information of at least one person authenticated by the first authentication process, and the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process. and a first control unit which is the control unit executes the first authentication process and the second authentication process for a first gate among a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel, thereby executing an authentication process for a person who is about to pass through the first gate, and the first control unit of the image processing device or a second control unit which is a control unit of a device other than the image processing device executes the first authentication process and the second authentication process for a second gate among the plurality of gates, and the second authentication process for the first gate is executed using facial image information regarding a facial image in a captured image of a person who is about to pass through the first gate as a matching source, and using both facial image information regarding the person authenticated in the first authentication process for the first gate and facial image information regarding the person authenticated in the first authentication process for the second gate as a matching target.
[0012] In the first authentication process, a process of authenticating a person by comparing facial image information of the person contained in the at least one first captured image with facial image information of the multiple persons may be repeatedly performed, thereby authenticating the person, multiple persons as the at least one person.
[0013] The at least one first captured image may include a plurality of first captured images, and in the first authentication process, a process of authenticating a person by comparing facial image information of a person included in any of the plurality of first captured images with facial image information of the plurality of persons may be repeatedly performed, thereby authenticating the person, by authenticating a plurality of persons as the at least one person.
[0014] In the first authentication process, face image information about a person included in the at least one first captured image may be collated with face image information about the multiple people, thereby authenticating the person.
[0015] The facial image information to be matched in the second authentication process may be facial image information of the at least one person authenticated in the first authentication process and extracted from the at least one first captured image.
[0016] The facial image information to be compared in the second authentication process may be facial image information corresponding to the at least one person authenticated in the first authentication process among template facial image information composed of facial image information of the large number of persons registered in advance.
[0017] The second authentication process is a process in which a person included in the second captured image is used as a reference person and the at least one person authenticated in the first authentication process is used as a target person, and the control unit authenticates the reference person by comparing facial image information regarding the reference person with facial image information regarding the target person. When a plurality of persons are authenticated in the first authentication process, after authenticating a first person among the plurality of persons by the second authentication process in which the first person is used as the reference person, when executing a new second authentication process in which a second person among the plurality of persons other than the first person is used as the reference person, the control unit may exclude the first person from the target persons in the new second authentication process.
[0018] The second authentication process is a process in which a person included in the second captured image is used as a reference person and the at least one person authenticated in the first authentication process is used as a target person, and the control unit authenticates the reference person by comparing facial image information regarding the reference person with facial image information regarding the target person. In a case in which a plurality of persons are authenticated in the first authentication process, after authenticating a first person among the plurality of persons as the reference person by the second authentication process and after a predetermined period has elapsed since a second person among the plurality of persons was authenticated by the first authentication process, when executing a new second authentication process in which a person other than the first person is used as the reference person, the control unit may exclude the second person from the target persons in the new second authentication process.
[0019] The control unit may register information about the person authenticated in the first authentication process as the matching target information in a database that temporarily stores matching target information for the second authentication process, and in the second authentication process, the person to be authenticated may be authenticated by matching facial image information of the person to be authenticated that is included in the second captured image with the matching target information stored in the database.
[0020] In the first authentication process, if matching is performed with the same number of matching targets, Place The authentication process may be a process that requires a longer processing time than the second authentication process and that achieves authentication with a higher accuracy than the second authentication process.
[0021] The amount of data of the feature amount extracted from the face image for matching in the first authentication process may be larger than the amount of data of the feature amount extracted from the face image for matching in the second authentication process.
[0025] In order to solve the above problem, an image processing system according to the present invention includes an image capture unit that captures at least one first captured image and a second captured image captured after the at least one first captured image, and a control unit that executes a first authentication process to authenticate a person by comparing face image information of the person included in the at least one first captured image with face image information of a large number of people registered in advance, and executes a second authentication process to authenticate the person to be authenticated by comparing face image information of the person to be authenticated included in the second captured image with face image information of at least one person authenticated by the first authentication process, wherein the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process. and the first authentication process and the second authentication process are executed for each of a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel, thereby executing an authentication process for each person attempting to pass through each gate, and the second authentication process for a first gate of the plurality of gates is executed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information of the person authenticated in the first authentication process for the first gate and facial image information of the person authenticated in the first authentication process for a second gate of the plurality of gates as a comparison target.
[0026] The photographing section may separately include a camera that photographs the at least one first photographed image and a camera that photographs the second photographed image.
[0030] In order to solve the above problems, the image processing method according to the present invention comprises the steps of: The computera step of executing a first authentication process for authenticating a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; and b) The computer and executing a second authentication process for authenticating the person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in a second captured image captured after the at least one first captured image, with face image information of at least one person authenticated by the first authentication process, wherein the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process. and the step a) includes: a-1) a step in which a first computer executes the first authentication process for a first gate among a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel; a-2) a step in which a second computer executes the first authentication process for a second gate among the plurality of gates; a-3) a step in which the first computer registers face image information of at least one person authenticated by the first authentication process for the first gate in a first database for the first gate; and a-4) a step in which the second computer registers face image information of at least one person authenticated by the first authentication process for the second gate in a first database for the first gate. Facial image information of at least one person authenticated by the first authentication process, registering the second database for the second gate and the first database for the first gate; Equipped with The step b) includes: b-1) a step in which the first computer executes the second authentication process for the first gate among the plurality of gates based on face image information registered in the first database; and b-2) a step in which the second computer executes the second authentication process for the second gate among the plurality of gates based on face image information registered in the second database. Equipped with The second authentication process for the first gate in step b-1) is performed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information relating to the person authenticated in the first authentication process for the first gate and facial image information relating to the person authenticated in the first authentication process for the second gate as a comparison target. In order to solve the above problems, the image processing method according to the present invention comprises the steps of: The computer a step of executing a first authentication process for authenticating a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; and b) The computer, and executing a second authentication process for authenticating the person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in a second captured image captured after the at least one first captured image, with face image information of at least one person authenticated by the first authentication process, wherein the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process. and the step a) includes: a-1) a step in which the computer executes the first authentication process for a first gate among a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel; a-2) a step in which the computer executes the first authentication process for a second gate among the plurality of gates; a-3) a step in which the computer registers face image information of at least one person authenticated by the first authentication process for the first gate in a first database for the first gate; and a-4) a step in which the computer registers face image information of at least one person authenticated by the first authentication process for the second gate in a first database for the first gate. Facial image information of at least one person authenticated by the first authentication process, registering the second database for the second gate and the first database for the first gate; Equipped with The step b) includes: b-1) a step in which the computer executes the second authentication process for the first gate among the plurality of gates based on face image information registered in the first database; and b-2) a step in which the computer executes the second authentication process for the second gate among the plurality of gates based on face image information registered in the second database. Equipped with The second authentication process for the first gate in step b-1) is performed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information relating to the person authenticated in the first authentication process for the first gate and facial image information relating to the person authenticated in the first authentication process for the second gate as a comparison target.
[0031] In order to solve the above-mentioned problems, the present invention provides a program for causing a computer to execute the steps of: a) executing a first authentication process for authenticating a person by comparing facial image information of the person included in at least one first captured image with facial image information of a large number of people registered in advance; and b) executing a second authentication process for authenticating the person to be authenticated by comparing facial image information of the person to be authenticated included in a second captured image captured after the at least one first captured image with facial image information of at least one person authenticated by the first authentication process, wherein the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process. and the step a) includes: a-1) a step of performing the first authentication process for a first gate among a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel; and a-2) a step of performing the first authentication process for a second gate among the plurality of gates. Equipped with The step b) includes: b-1) executing the second authentication process for the first gate of the plurality of gates; and b-2) executing the second authentication process for the second gate of the plurality of gates. Equipped with The second authentication process for the first gate in the step b-1) is performed using facial image information on a facial image in a photographed image of a person who is about to pass through the first gate as a comparison source, and using both facial image information on a person authenticated in the first authentication process for the first gate and facial image information on a person authenticated in the first authentication process for the second gate as a comparison target. . In order to solve the above problem, a program according to the present invention includes the steps of: a) executing a first authentication process for authenticating a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; and b) executing a second authentication process for authenticating the person to be authenticated by comparing face image information of the person to be authenticated included in a second captured image captured after the at least one first captured image with face image information of at least one person authenticated by the first authentication process. First and second A program to be executed by a computer, the program comprising: a number of persons to be matched in the second authentication process being smaller than a number of persons to be matched in the first authentication process; and the step a) includes: a-1) a step in which the first computer executes the first authentication process for a first gate among a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel; and a-2) a step in which the second computer executes the first authentication process for a second gate among the plurality of gates. Equipped with The step b) includes: b-1) a step in which the first computer executes the second authentication process for the first gate of the plurality of gates; and b-2) a step in which the second computer executes the second authentication process for the second gate of the plurality of gates. Equipped with The second authentication process for the first gate in the step b-1) is performed using facial image information on a facial image in a photographed image of a person who is about to pass through the first gate as a comparison source, and using both facial image information on a person authenticated in the first authentication process for the first gate and facial image information on a person authenticated in the first authentication process for the second gate as a comparison target. . Effect of the Invention
[0032] According to the present invention, it is possible to realize rapid authentication processing without relying on the possession of a wireless tag. [Brief description of the drawings]
[0033] [Figure 1] 1 is a block diagram showing an image processing system according to a first embodiment. [Diagram 2]FIG. 2 is a diagram showing each gate device of the entrance subsystem and the exit subsystem. [Diagram 3] FIG. 2 is a block diagram showing a configuration of an authentication device. [Figure 4] 11A to 11C are diagrams showing how a captured image used in the first authentication process is captured, etc. FIG. [Diagram 5] 11A to 11C are diagrams showing how a captured image used in the second authentication process is captured, etc. FIG. [Figure 6] 11A to 11C are diagrams showing how a captured image used in the first authentication process is captured, etc. FIG. [Figure 7] 11A to 11C are diagrams showing how a captured image used in the second authentication process is captured, etc. FIG. [Figure 8] FIG. 13 is an enlarged view of a preliminary captured image. [Figure 9] FIG. 2 is a diagram showing a person in an actually photographed image. [Figure 10] 13 is a flowchart showing a first authentication process (preliminary authentication process). [Figure 11] 13 is a flowchart showing a second authentication process (main authentication process). [Figure 12] FIG. 13 is a diagram showing how a person is authenticated in the first authentication process. [Figure 13] FIG. 13 is a diagram showing how another person is authenticated in the first authentication process. [Figure 14] FIG. 13 is a diagram showing how a plurality of persons are authenticated in the first authentication process. [Figure 15] FIG. 13 is a diagram showing how a person is authenticated in the second authentication process. [Figure 16] FIG. 13 is a diagram showing how another person is authenticated in another second authentication process. [Figure 17] FIG. 11 is a diagram showing a second authentication process according to a modified example. [Figure 18] 6 is a timing chart showing execution timings of the first authentication process and the second authentication process. [Figure 19] 10 is a timing chart showing another execution timing of the first authentication process and the second authentication process. [Figure 20]FIG. 11 is a block diagram showing an image processing system according to a second embodiment. [Figure 21] FIG. 11 is a top view of the vicinity of multiple entrance gates provided in the entrance subsystem according to the second embodiment. [Figure 22] FIG. 11 is a block diagram showing an image processing system according to a third embodiment. [Figure 23] FIG. 11 is a block diagram showing an authentication device according to a third embodiment. [Figure 24] FIG. 11 is a top view of the vicinity of multiple entrance gates provided in the entrance subsystem according to the third embodiment. [Diagram 25] FIG. 13 is a block diagram showing an image processing system according to a fourth embodiment. [Figure 26] FIG. 13 is a diagram showing authentication timing etc. in an image processing system according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0034] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0035] <1. First embodiment> <1-1. System Overview> Fig. 1 is a block diagram showing an image processing system 1 (also referred to as an authentication system). As shown in Fig. 1, the image processing system 1 (also referred to as 1A) according to the first embodiment includes an entrance subsystem 2 (also referred to as 2A) that controls an entrance gate device and the like, and an exit subsystem 3 (also referred to as 3A) that controls an exit gate device and the like. The entrance subsystem 2 and the exit subsystem 3 are provided in, for example, each of a plurality of stations (railroad stations, etc.).
[0036] 2 is a diagram showing a gate device 91 (also referred to as an entrance gate device 92) of the entrance subsystem 2 and a gate device 91 (also referred to as an exit gate device 93) of the exit subsystem 3. Each gate device 91 (92, 93) is a passage opening / closing type gate device provided, for example, in a ticket gate passage, and controls the passage of people passing through the gate device based on the authentication result by an authentication device described later. Note that hereinafter, the gate device will also be simply referred to as a gate.
[0037] The entrance gate 92 in Fig. 2 is a gate provided at the entrance ticket gate of the boarding station, and is a gate for entering a predetermined area such as the station premises. The exit gate 93 in Fig. 2 is a gate provided at the exit ticket gate of the disembarking station, and is a gate for exiting a predetermined area such as the station premises. As shown in Fig. 2, a user (railway user, etc.) enters through the entrance gate 92 at the entrance ticket gate of the boarding station, and exits through the exit gate 93 at the exit ticket gate of the disembarking station.
[0038] The entrance subsystem 2 and the exit subsystem 3 have similar configurations. In the following, the entrance subsystem 2 will be mainly described, and a description of the exit subsystem 3 will be omitted.
[0039] The entrance subsystem 2 has an authentication device 10 and a gate 91 (92). In addition, cameras 21, 22 (see FIG. 2, etc.) are arranged near the gate 91 (92). The authentication device 10 executes an authentication process for a person who is going to pass through the gate 91 based on images captured by the cameras 21, 22, and determines whether the person is allowed to pass based on the result of the authentication process. The gate 91 controls the passage of the person based on the result of the authentication process by the authentication device 10 and the determination of whether the person is allowed to pass. Specifically, when the person is allowed to pass, the gate 91 opens the door of the gate 91 to allow the person to pass. Conversely, when the person is not allowed to pass, the gate 91 closes the door of the gate 91 to prevent the person from passing.
[0040] 3 is a block diagram showing the configuration of the authentication device 10. As shown in FIG.
[0041] The photographing unit 20 has a first camera 21 and a second camera 22. Each of the cameras 21 and 22 is equipped with an imaging sensor or the like that captures a two-dimensional image. For example, each of the cameras 21 and 22 is equipped with an RGB image sensor or the like that captures a visible light image (such as a color image), and is capable of photographing a color image (or a grayscale image). The second camera 22 is capable of photographing a person (a passing target person) who is about to pass through the gate 91 just before the gate 91 (see FIG. 5, etc.). On the other hand, the first camera 21 is capable of photographing a person who is present at a position further forward (a position farther from the gate 91) than the position just before the gate 91 (see FIG. 4, etc.).
[0042] The image processing device 30 is a device that performs image processing and the like on each of the images captured by the cameras 21 and 22. In addition, the image processing device 30 performs authentication processing on a person attempting to pass through the gate 91 by performing a two-stage authentication processing (first authentication processing and second authentication processing) described later.
[0043] The image processing device 30 includes a controller 31 , a storage unit 32 , a communication unit 34 , and an operation unit 35 .
[0044] The controller 31 (also referred to as a control unit) is a control device that is built into the image processing device 30 and controls the operation of the image processing device 30.
[0045] The controller 31 is configured as a computer system including one or more hardware processors (e.g., a central processing unit (CPU) and a graphics processing unit (GPU)). The controller 31 executes, in the CPU or the like, a predetermined software program (hereinafter also simply referred to as a program) stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk) 32, thereby realizing various processes. The program (more specifically, a group of program modules) may be recorded in a portable recording medium such as a USB memory, read out from the recording medium, and installed in the image processing device 30. Alternatively, the program may be downloaded via a communication network or the like and installed in the image processing device 30.
[0046] The controller 31 acquires the first captured image 110 and the second captured image 210 from the photographing section 20. The first captured image 110 is an image captured at a timing (for example, one to several tens of seconds before) the timing of capturing the second captured image. Specifically, the controller 31 acquires the image captured by the first camera 21 as the first captured image 110, and acquires the image captured by the second camera 22 as the second captured image 210. These captured images 110, 210 are images captured at approximately the same location (or nearby locations). The controller 31 executes authentication processing of a person attempting to pass through the gate 91 based on these captured images 110, 210.
[0047] The storage unit 32 is configured with a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage unit 32 stores a user registration database 32A and a first authentication success database 32B.
[0048] The user registration database 32A is a database in which information (authentic information) on users of the system 1 is registered in advance. In the user registration database 32A, face image information on the face image of each user (the face image itself and feature amount data on the face image, etc.) is stored in advance in association with the registrant ID (identification information for identifying the registrant) of each user. For example, when purchasing a commuter pass, the face image of each user is captured by a camera or the like installed near the commuter pass purchasing device (automatic ticket vending machine, etc.), and information (authentic information) on the face image is registered in the user registration database 32A. The face image information registered in advance on the face image of each user is also referred to as template face image information. In other words, the template face image information is information configured by including face image information of a large number of people registered in advance.
[0049] The first authentication success database 32B is a database that temporarily stores (registers) information (such as facial image information) of a person authenticated by the first authentication process E1 (described later). In other words, the first authentication success database 32B is a database that temporarily stores matching target information of the main authentication process E2. In this embodiment, the information registered in the first authentication success database 32B is frequently (every moment) updated as needed.
[0050] The communication unit 34 is capable of performing network communication via a network. In this network communication, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) are used. By using the network communication, the image processing device 30 can transmit and receive various data to and from a desired destination (for example, the photographing unit 20, another image processing device 30, etc.).
[0051] The operation unit 35 includes an operation input unit 35a that receives operation input to the image processing device 30, and a display unit 35b that displays and outputs various information. A mouse, a keyboard, or the like is used as the operation input unit 35a, and a display (such as a liquid crystal display) is used as the display unit 35b. A touch panel that functions as both a part of the operation input unit 35a and a part of the display unit 35b may also be provided.
[0052] <1-2. Basic concepts etc.> Before describing the basic concept of this embodiment, a technique according to a comparative example will first be described.
[0053] In Figure 5, an image (captured image) 210 of a person U1 is captured when the person U1 walks toward a gate 91 (entrance gate 92, etc.) from a distance (the left side of Figure 5) and arrives just before the gate 91.
[0054] In the technology according to the comparative example, one-to-many (1-to-N) authentication processing (specifically, face authentication processing) is performed based only on the captured image 210. In this one-to-many (1-to-N) authentication processing, a face image of a person (e.g., U1) included in the captured image 210 is matched with face images of a large number (N people) of people (e.g., N=hundreds to millions (people)) registered in advance, thereby authenticating the person.
[0055] Then, depending on the result of the authentication process, it is determined whether or not the person U1 is permitted to pass through the gate. If the person U1 is permitted to pass through the gate, the door (or bar) of the gate 91 is opened to allow the person U1 to pass through. Conversely, if the person U1 is not permitted to pass through the gate, the door (or bar) of the gate 91 is closed to prevent the person U1 from passing through.
[0056] In this way, depending on the processing result of the authentication processing based on the captured image 210, the door of the gate 91 is opened and closed. Therefore, it is preferable that the processing time of the authentication processing (the authentication processing time after the captured image 210 is captured) is short. In particular, if a person other than the person U1 (person to be authenticated) of the captured image 210 cuts in line from the side of the queue and enters the gate 91 after the captured image 210 is captured, there is a possibility that an authentication error and / or an authentication failure may occur. In order to prevent such an abnormality caused by cutting in just before the gate, it is preferable that the timing of capturing the captured image 210 is as close as possible to the timing of the person U1 entering the gate 91. In order to meet such a demand, it is preferable that the processing time of the authentication processing from the time the captured image 210 is captured is within several hundred milliseconds (for example, 500 milliseconds).
[0057] However, when one-to-many (1-to-N) authentication processing is performed from the time of capturing the captured image 210, it may take a relatively long time (for example, 1 second) from the time of capturing the captured image 210. In that case, it is difficult to satisfy the above-mentioned constraint on authentication processing time.
[0058] Therefore, in this embodiment, a preliminary authentication process (preliminary authentication process) based on a preliminary captured image 110 (see FIG. 4) is executed prior to an authentication process (also referred to as a main authentication process) based on the captured image 210. In other words, the preliminary authentication process is executed as a first authentication process E1, and the main authentication process is executed as a second authentication process E2.
[0059] Next, the basic concept of this embodiment will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a diagram showing how a captured image 110 (also referred to as a preliminary captured image) used in a first authentication process (preliminary authentication process) E1 is captured by a camera 21. Meanwhile, Fig. 5 is a diagram showing how a captured image 210 (also referred to as a main captured image) used in a second authentication process (main authentication process) E2 is captured by a camera 22.
[0060] In the preliminary authentication process (first authentication process) E1, one-to-many (1-to-N) (for example, N=several hundred to several million (people)) authentication process is executed based on a preliminary captured image (first captured image) 110 (see FIG. 4) captured at a timing before the capture timing of the actual captured image 210. In detail, one-to-many (1-to-N) authentication process is executed using face image information related to the face image of the person U1 in the preliminary captured image 110 as a matching source and face image information related to face images of a large number of people registered in advance as a matching target. This authenticates the person U1 (preliminarily authenticated or provisionally authenticated). Note that the preliminary captured image 110 is captured at a certain point in time (for example, 2 seconds before) the actual captured image 210 so that the preliminary authentication process E1 based on the preliminary captured image 110 is completed before the start of the actual authentication process E2 based on the actual captured image 210.
[0061] Then, after the preliminary authentication process E1, a main authentication process E2 (second authentication process) is performed (see FIG. 5). In the main authentication process E2, a person U1 to be authenticated in the main authentication process (person U1 in the main captured image 210) is authenticated using one or more people authenticated in the preliminary authentication process E1 as a match target. The main authentication process E2 is executed using face image information on the face image of the person U1 to be authenticated included in the main captured image (second captured image) 210 as a match source and face image information on the face images of at least one person (M people) authenticated by the preliminary authentication process E1 as a match target. That is, the main authentication process E2 is a one-to-few (1-to-M) authentication process. The at least one person (M people) authenticated by the preliminary authentication process E1 is also called a preliminary authentication person or a primary authentication person.
[0062] In this way, the main authentication process E2 is an authentication process in which the person included in the main captured image 210 is the reference person and the M people authenticated in the pre-captured image 110 are the target persons. In detail, the main authentication process E2 is a process for authenticating the reference person by matching face image information of the reference person with face image information of the target person. Through such a process, the person U1 to be authenticated in the main authentication process E2 is authenticated (main authentication).
[0063] The face image information of a person is, for example, a feature amount (data indicating facial features) extracted from the face image of the person. The feature amount may be extracted by performing a specific image processing on the face image. Alternatively, the feature amount may be extracted from the face image using a learning device (trained model) trained by machine learning (deep learning, etc.).
[0064] However, without being limited thereto, the face image information of a person may be the face image itself of the person, or may be both the face image itself and feature amounts extracted from the face image, etc.
[0065] 4, for example, when the preliminary captured image 110 includes only a single person U1, the preliminary authentication process E1 executes a one-to-many (1-to-N) authentication process in which the person U1 in the preliminary captured image 110 is used as a matching source and a large number of people registered in advance are used as matching targets. As a result, the preliminary authentication process E1 authenticates the single person U1.
[0066] Then, as shown in FIG. 5, in the main authentication process E2 in which the person U1 in the main captured image 210 is the person to be authenticated (matching source), a matching process (one-to-one authentication process) is executed with the single person U1 authenticated in the preliminary authentication process E1 as the matching target.
[0067] Since the number of matching targets (number of people (number of face images)) (1 person) in one-to-one authentication processing is smaller than the number of matching targets N in one-to-many authentication processing, one-to-one authentication processing is executed faster than one-to-many authentication processing. Therefore, it is possible to satisfy the above-mentioned authentication processing time constraint (the processing time constraint from the time of capturing the captured image 210).
[0068] Also, as shown in FIG. 6 (and FIG. 8) etc., when the preliminary captured image 110 includes a plurality of persons (five persons U1 to U5 in FIG. 6), a one-to-many (one-to-N) authentication process is repeatedly executed in the preliminary authentication process E1. In detail, a one-to-many (one-to-N) authentication process is repeatedly executed with each person (each of the persons U1 to U5) in the preliminary captured image 110 as a matching source and a large number of persons registered in advance as matching targets. As a result, a plurality (M persons) of persons U1 to U5 (fewer than N persons) are authenticated by the preliminary authentication process E1. Note that FIG. 6 is a diagram showing how the preliminary captured image 110 including a plurality of persons is captured, and FIG. 8 is a diagram showing an enlarged view of the preliminary captured image 110.
[0069] Specifically, multiple (M) persons U1 to U5 are authenticated by repeatedly executing a process of authenticating each person by comparing each of the facial image information relating to the facial images of the multiple persons U1 to U5 in the preliminary captured image 110 with facial image information relating to a large number of facial images.
[0070] More specifically, one-to-many (1-to-N) authentication processing is performed using person U1 in the preliminary captured image 110 as the source of comparison, and person U1 is authenticated. Also, one-to-many (1-to-N) authentication processing is performed using person U2 in the preliminary captured image 110 as the source of comparison, and person U2 is authenticated. Furthermore, one-to-many (1-to-N) authentication processing is performed using person U3 in the preliminary captured image 110 as the source of comparison, and person U3 is authenticated. Similarly, one-to-many (1-to-N) authentication processing is performed using persons U4 and U5 in the preliminary captured image 110 as the sources of comparison, and persons U4 and U5 are authenticated.
[0071] Thereafter, in the main authentication process E2 (see FIG. 7) in which person U1 (see FIG. 9) in the main captured image 210 is the person to be authenticated (matching source), a matching process is executed with a small number (M people) of people U1 to U5 authenticated in the preliminary authentication process E1 as the matching targets. That is, a 1-to-M (1-to-small number) (here, M=5) authentication process is executed. In this way, in the main authentication process E2, authentication process is executed with M people (relatively small number of people) narrowed down from N people (relatively large number of people) as the matching targets.
[0072] Since the number M of matching targets (the number of people (the number of face images)) in the 1:M authentication process is smaller than the number N of matching targets in the 1:many authentication process, the 1:M authentication process is executed faster than the 1:N authentication process, and therefore it is possible to satisfy the above-mentioned constraints on the authentication process time.
[0073] In this manner, in this embodiment, the preliminary authentication process E1 and the main authentication process E2 (second authentication process) are performed.
[0074] Thereafter, whether or not the person U1 is permitted to pass through the gate is determined according to the processing result of this authentication process E2 (second authentication process). If the person U1 is permitted to pass through the gate, the door (or bar) of the gate 91 is opened to allow the person U1 to pass through. Conversely, if the person U1 is not permitted to pass through the gate, the door (or bar) of the gate 91 is closed to prevent the person U1 from passing through.
[0075] In this embodiment, it is possible to determine whether a person is permitted to pass through based on the result of the authentication process based on the face image. In other words, even if the person does not have a medium such as a commuter pass or a transportation card, it is possible to determine whether the person is permitted to pass through. This allows a person to pass through gate 91 (by foot) without having a medium such as a commuter pass (so-called walk-through).
[0076] Here, in order to ensure accuracy in 1:N authentication, the matching process in the preliminary authentication process E1 is preferably a process that is heavier than the matching process in the main authentication process E2. In other words, the preliminary authentication process E1 is preferably a process that realizes authentication with a higher accuracy than the main authentication process E2. Specifically, the preliminary authentication process E1 is preferably a process that requires a longer processing time than the main authentication process E2 if matching is performed with the same number of matching targets. In detail, the amount of data (number of parameters, parameter size, etc.) of features extracted from a face image for matching in the preliminary authentication process E1 is preferably larger than the amount of data of features extracted from a face image for matching in the main authentication process E2. The larger (larger) the amount of data of the features to be matched is, the more accurate authentication can be realized. In the preliminary authentication process E1, for example, a relatively large amount of features extracted using a trained model (first trained model) capable of authentication with a predetermined level or higher of accuracy may be used.
[0077] On the other hand, since the authentication process E2 is a process in which a person authenticated in the preliminary authentication process E1 (authenticated face image) is used as a matching target, it is preferable that the matching process in the authentication process E2 prioritizes high speed and is lighter than the matching process in the preliminary authentication process E1. In detail, it is preferable that the amount of data of the feature amount extracted from the face image for matching in the authentication process E2 is smaller than the amount of data of the feature amount extracted from the face image for matching in the preliminary authentication process E1. In the authentication process E2, for example, a relatively small amount of feature amount extracted using a trained model (second trained model) different from the above-mentioned first trained model may be used. The second trained model is a trained model that has a lower authentication accuracy than the first trained model, but is capable of extracting the feature amount faster than the first trained model.
[0078] In this embodiment, the preliminary authentication process E1 involves relatively heavy processing (matching processing based on a relatively large amount of data) to perform accurate authentication, whereas the main authentication process E2 involves relatively light processing (matching processing based on a relatively small amount of data) to perform high-speed processing (quick processing).
[0079] <1-3. Processing details> Next, the processing according to this embodiment will be described in more detail.
[0080] Figures 10 and 11 are flowcharts showing the processing of the controller 31. Figure 10 is a flowchart showing a preliminary authentication processing E1, and Figure 11 is a flowchart showing a main authentication processing E2.
[0081] In this embodiment, a plurality of preliminary captured images 110 captured at different times are sequentially acquired. In detail, the plurality of preliminary captured images 110 are sequentially acquired at a predetermined time interval (e.g., 250 millisecond intervals). Then, the process of the flowchart in Fig. 10 is sequentially executed for each of the plurality of preliminary captured images 110 (see also Fig. 18).
[0082] The flowchart in Fig. 10 shows the process executed for each preliminary captured image 110. The process of the flowchart in Fig. 10 takes, for example, about one second. The process of Fig. 10 for each of the multiple preliminary captured images 110 is executed in parallel with each other (see also Fig. 18). Furthermore, information (such as facial image information) about the person authenticated by the process of Fig. 10 for each preliminary captured image 110 (the person who succeeded in the first authentication) is stored in the first authentication success database 32B as needed.
[0083] 11 is executed when a person to be authenticated arrives at a position just before gate 91. The process of the flowchart in Fig. 11 is completed within, for example, several milliseconds to several hundred milliseconds (for example, at most 400 milliseconds). Fig. 18 is a timing chart showing the execution timing of preliminary authentication process E1 for a plurality of preliminary captured images 110 and main authentication process E2 for a plurality of main captured images 210.
[0084] In other words, in this embodiment, the process of Fig. 10 for each preliminary captured image 110 (such as one-to-many authentication process using each person in the preliminary captured image 110 as a matching source) is executed at a predetermined time interval as needed. Information on the person authenticated (provisionally authenticated) in the one-to-many authentication process is additionally registered in the first authentication success database 32B as needed. Furthermore, the main authentication process E2 is executed every time a new person arrives at the gate 91. Each main authentication process E2 is executed using face image information registered in the first authentication success database 32B (face image information related to the face images of M people authenticated in the preliminary authentication process E1 based on the multiple preliminary captured images 110) as a matching target.
[0085] In this case, at least one person authenticated by one-to-many authentication processing using a person included in any one of the multiple preliminary captured images 110 as a matching source is adopted as a matching target of the main authentication processing E2. This makes it possible to increase the possibility that the person has been authenticated in advance in the preliminary authentication processing E1 even when the main captured image 210 is captured at the moment when a person who has temporarily left the field of view of the camera 21 moves (returns) to a position just before the gate 91 and the person is set as a person to be authenticated. However, without being limited to this, at least one person authenticated by one-to-many authentication processing regarding the (latest) preliminary captured image 110 captured just before (as close as possible to) the main captured image 210 among the multiple preliminary captured images 110 may be adopted as a matching target of the main authentication processing E2. In other words, at least one person authenticated by the preliminary authentication processing E1 (the preliminary authentication processing E1 based on one preliminary captured image 110) that has been completed at the start of the main authentication processing E2 may be adopted as a matching target of the main authentication processing E2.
[0086] First, the preliminary authentication process E1 will be described with reference to FIG.
[0087] As shown in FIG. 10, in step S11, the controller 31 acquires a pre-captured image 110 captured by the camera 21 (see FIG. 4 or FIG. 6, etc.).
[0088] Camera 21 is capable of capturing an image of a person walking toward gate 91 from a distance away from gate 91 when the person reaches a position a little away from gate 91. The installation position and viewpoint of camera 21 are adjusted so that camera 21 can capture an image of a face of a person who is located a certain distance in front of a position immediately in front of gate 91. For example, camera 21 is installed so that its viewpoint faces a person approaching gate 91. Camera 21 is also installed at a certain height or more (for example, 3 meters above the floor) so that the faces of multiple people approaching gate 91 can be captured from relatively high up while avoiding overlapping. Camera 21 preferably has a wider viewing angle than camera 22 so that it can capture as wide an area as possible.
[0089] In this way, camera 21 is a camera for capturing an image in advance of a person who is likely to enter gate 91. In other words, pre-captured image 110 captured by camera 21 is an image captured in advance of a person who will subsequently enter gate 91. Note that pre-captured image 110 may be a still image or a moving image (more specifically, a frame image within a moving image).
[0090] In the next step S12, the controller 31 extracts face images of people from the pre-captured image 110. For example, if the pre-captured image 110 contains (shows) face images of a small number of people (for example, M people), the controller 31 extracts the face images of the small number of people (M people) from the pre-captured image 110. Here, the value M is an integer equal to or greater than 1 (M=5 in FIG. 6 and the like). The left side of FIG. 12 shows how the face image of person U1 is extracted from the pre-captured image 110, and the left side of FIG. 13 shows how the face image of person U2 is extracted from the pre-captured image 110. Similarly, face images of each of the other people (U3 to U5 and the like) are extracted. It is not necessarily required that all of the face images in the pre-captured image 110 (face images of all people in the pre-captured image 110) are extracted in step S12. For example, only face images of a certain size or larger (and therefore only some of the people corresponding to face images of a certain size or larger (such as people approaching the gate more than a certain extent)) may be extracted. In other words, only some of the people (one or more people) among the people included in the preliminary captured image 110 may be used as the source of matching for each preliminary authentication process E1 (each 1:N authentication) in the next step S13.
[0091] In step S13, the controller 31 executes a preliminary authentication process E1. The preliminary authentication process E1 is a process for authenticating at least one person by repeating a process of matching face image information on a face image of a person (one person or each of a plurality of people) included in the preliminary captured image 110 with face image information on a number of face images registered in advance. More specifically, one-to-N authentication (one-to-many authentication) is executed in which face image information on each of the face images of M people included in the preliminary captured image 110 is used as a matching source, and face image information on the face images of N people registered in advance in the user registration database 32A is used as a matching destination. As a result, a small number of people (for example, M people) are authenticated.
[0092] For example, if multiple persons U1 to U5 are present in the preliminary captured image 110 (see FIG. 6), a process of authenticating each person by comparing each of the facial image information of the multiple persons U1 to U5 with facial image information of multiple facial images (one-to-many authentication) is repeatedly executed. In this way, multiple (M) persons U1 to U5 are authenticated.
[0093] In detail, one-to-many (1-to-N) authentication processing is executed using the person U1 in the preliminary captured image 110 as a matching source, and the person U1 is authenticated. Specifically, as shown in FIG. 12, an authentication processing is executed in which face image information on a face image 120 (face image 121 of the person U1) extracted from the preliminary captured image 110 is matched with face image information on face images 320 (N face images 320) of N people preregistered in the user registration database 32A. More specifically, for example, face feature amount data extracted from the face image 121 is matched with face feature amount data on face images 320 of N people Ri preregistered in the user registration database 32A. As a result, the person U1 is authenticated (provisionally authenticated). In detail, for example, when it is determined that the face image of the person Rj matches the face image of the person U1, the person U1 is determined to be the registered person Rj.
[0094] Furthermore, one-to-many (1-to-N) authentication processing is executed using person U2 in the preliminary captured image 110 as a matching source, and person U2 is authenticated. Specifically, as shown in Fig. 13, authentication processing is executed in which face image information on face image 120 (face image 122 of person U2) extracted from the preliminary captured image 110 is matched with face image information on face images of N persons pre-registered in the user registration database 32A. As a result, person U2 is authenticated (provisionally authenticated).
[0095] Similarly, the authentication process is executed for each of the other persons (U3 to U5, etc.), and each of the persons is authenticated (provisionally authenticated).
[0096] As a result of such preliminary authentication process E1, a plurality of persons U1 to U5 in the preliminary captured image 110 are authenticated (provisionally authenticated) as shown in Fig. 14. In other words, persons to be matched in the main authentication process E2 are narrowed down to M persons out of N registered persons.
[0097] In step S14, the controller 31 registers (stores) information (such as face image information) about the M people authenticated in step S13 in the first authentication success database 32B.
[0098] Specifically, as information regarding the M persons authenticated in step S13, facial image information regarding the facial images contained in the preliminary captured image 110 (the facial image (facial image data) itself extracted from the preliminary captured image 110, and / or feature data extracted from the facial image, etc.) is registered in the first authentication success database 32B.
[0099] Here, duplicate registrations are eliminated using a registrant ID (identification information for identifying a registrant) defined in the user registration database 32A. Specifically, in addition to the facial image information of the provisionally authenticated person, the registrant ID of the person is also registered in the first authentication success database 32B. Then, in order to prevent duplicate registrations in the first authentication success database 32B, a person other than the person already registered in the first authentication success database 32B is additionally registered. More specifically, on condition that a registrant ID identical to the registrant ID of the person authenticated in step S13 has not yet been registered, facial image information of the person is registered in the first authentication success database 32B together with the registrant ID.
[0100] Next, the authentication process E2 will be described with reference to FIG.
[0101] In step S21, the controller 31 acquires an actual captured image 210 (see FIG. 5 or FIG. 7, etc.) captured by the camera 22. The installation position, viewpoint, etc. of the camera 22 are adjusted so that the face of a person present at a predetermined position just before the gate 91 can be captured in a large size (above a certain size). The actual captured image 210 may be a still image or a moving image (more specifically, a frame image within a moving image).
[0102] Specifically, the controller 31 detects face images in images (moving images here) captured by the camera 22 at very short time intervals (for example, every 1 / 30 seconds). Then, when the face image becomes larger than a predetermined size (predetermined dimension), the controller 31 determines that the person has reached a predetermined position just before the gate 91, and obtains the captured image (such as a frame image in the moving image) captured by the camera 22 at that time as the actual captured image 210.
[0103] In the next step S22, the controller 31 extracts a person's face image 220 from the actual captured image 210 (see the left side of FIG. 15), and identifies the person to be authenticated (authentication target face image) in this authentication process E2. Specifically, a face image having a size equal to or larger than a predetermined size is identified as the authentication target face image, and a person corresponding to the authentication target face image is identified as an authentication-compatible person. Note that, if multiple face images each having a size equal to or larger than a predetermined size are included in the actual captured image 210, the largest face image (or the face image closest to the front) of the multiple face images may be identified as the authentication target face image.
[0104] In the next step S23, the controller 31 executes a main authentication process E2 (one-to-M authentication process) to match the person to be authenticated with a small number (M people) of people (e.g., people U1 to U5) previously authenticated in the preliminary authentication process (see FIG. 15). The main authentication process E2 is a process for authenticating (main authentication) the person to be authenticated by matching face image information on the face image of the person to be authenticated included in the main captured image 210 with face image information on the face images of the M people authenticated in the preliminary authentication process E1. As described above, in the main authentication process E2, the authentication process is executed with M people (relatively few people) narrowed down from N people (relatively many people) as the people to be matched.
[0105] Specifically, the controller 31 compares facial image information on the facial image 220 of the person to be authenticated with facial image information (matching target information) of persons (five persons U1 to U5 in FIG. 15) registered in the first authentication success database 32B, and authenticates the person to be authenticated.
[0106] For example, if facial image information on the facial image 220 (221) (see FIG. 15) of the person U1 to be authenticated matches the facial image information of person U1 among the five persons U1 to U5 registered in the first authentication success database 32B, the person U1 to be authenticated is authenticated.
[0107] Furthermore, if the facial image information on the facial image 220 (222) (see FIG. 16) of the person U2 to be authenticated matches the facial image information of the person U2 among the five persons U1 to U5 registered in the first authentication success database 32B, the person U2 to be authenticated is authenticated.
[0108] Steps S24 and S25 will be described later.
[0109] By performing the above operations, the number M of face images to be matched in the main authentication process E2 is smaller than the number N of face images to be matched in the preliminary authentication process E1 (the number N of faces to be matched in the one-to-many authentication process). Therefore, the main authentication process E2 based on the main captured image 210 captured just before the gate 91 can be executed at high speed. Therefore, it is possible to realize a rapid authentication process without relying on the possession of a wireless tag.
[0110] In the above embodiment, face image information extracted from the preliminary captured image 110 is registered in the first authentication success database 32B as information on the M persons authenticated in the preliminary authentication process E1 (step S13). More specifically, face image information on the face images 121-125 extracted from the preliminary captured image 110 (image data of the face images 121-125 and feature amounts extracted from the face image data, etc.) is registered (see the right side of FIG. 15, etc.). Then, in the main authentication process E2, matching is performed based on the face image information. In other words, the face image information to be matched in the main authentication process E2 is the face images of the M persons authenticated in the preliminary authentication process E1 and includes face image information on the face images in the preliminary captured image 110.
[0111] However, without being limited thereto, the facial image information to be matched in the present authentication process E2 may be template facial image information (facial image information related to a large number of template facial images) in which facial image information related to a large number of facial images (facial images of a large number of people) is registered in advance. As described above, the template facial image information is information stored in advance in the user registration database 32A.
[0112] More specifically, the facial image information to be matched in the main authentication process E2 may be facial image information corresponding to one or more persons authenticated in the preliminary authentication process E1, among the template facial image information.
[0113] For example, among the template face image information, face image information corresponding to one or more persons authenticated by the preliminary authentication process E1 may be copied from the user registration database 32A to the first authentication success database 32B and stored therein. Then, the main authentication process E2 may be executed using the face image information (part of the template face image information) copied and stored in the first authentication success database 32B as the target for matching.
[0114] Alternatively, only M registrant IDs corresponding to M persons (U1 to U5, etc.) authenticated in the preliminary authentication process E1 may be registered in the first authentication success database 32B (see FIG. 17). Then, in the main authentication process E2, face image information of M persons corresponding to the registrant IDs registered in the first authentication success database 32B may be extracted from the user registration database 32A and used as matching target information. In short, face image information in the user registration database 32A may be used in matching in the main authentication process E2. Specifically, as shown in FIG. 17, the main authentication process E2 may be executed using face image information on the template face image 320 corresponding to the registrant IDs (registrant IDs registered in the first authentication success database 32B) of the persons U1 to U5 authenticated in the preliminary authentication process E1 as the matching target. This makes it possible to reduce the capacity of the first authentication success database 32B.
[0115] However, from the viewpoint of suppressing the influence of changes over time (including facial aging and whether or not a mask is worn) of a person's facial image from the time of initial registration, it is preferable to use information on the facial image extracted from the preliminary captured image 110 (rather than facial image information on the template facial image in the user registration database 32A). In detail, it is preferable to use facial image information on the facial images of M persons authenticated in the preliminary authentication process E1 and included in the preliminary captured image 110 for matching with facial image information on the authentication target facial image in the main captured image 210 in the main authentication process E2. This makes it possible to suppress the influence of changes over time in a person's facial image compared to the case where a template facial image registered in advance for the person is used for matching with a matching source image.
[0116] For example, as shown in FIG. 16, assume that the face image 220 (222) of person U2 in the actual captured image 210 is an image of a face wearing a mask. In this case, the face image (face image of person U2) 120 (122) extracted from the preliminary captured image 110 captured immediately before the actual captured image 210 (and thus the face image to be matched in the actual authentication process E2) is often a face image wearing a mask. Therefore, in the matching process for matching the two face images 120, 220 (specifically, their face image information), the influence of changes over time is suppressed. Therefore, even when a relatively simple matching is performed with priority given to high speed in the actual authentication process E2, the influence of changes over time in the face image of the person (including whether or not a mask is worn) is suppressed, and sufficient authentication accuracy can be obtained.
[0117] In this case, in the preliminary authentication process E1, face image information on a face image with a mask (face image 120 extracted from the preliminary captured image 110) is compared with face image information on a face image without a mask (pre-registered face image 320). However, as described above, the preliminary authentication process E1 performs a relatively high-precision matching process (authentication process). Therefore, even if matching is performed on two face images that differ in whether or not a mask is worn, accurate matching (authentication) is possible.
[0118] <1-4. Matching destination adjustment process in this authentication process E2> As described above, in this embodiment, the preliminary authentication process E1 is executed at predetermined time intervals as needed. Information about a person who has been provisionally authenticated (preliminarily authenticated) in the preliminary authentication process E1 is additionally registered in the first authentication success database 32B as needed. Moreover, the main authentication process E2 is executed as needed every time a new person arrives at the gate 91.
[0119] This authentication process E2 is executed by using face image information registered in the first authentication success database 32B (face image information related to the face image of the person authenticated in the preliminary authentication process E1) as a match target. Here, it is preferable that the number of people registered in the first authentication success database 32B is minimized. In other words, it is preferable that the match targets in this authentication process E2 are minimized.
[0120] Therefore, it is preferable that a person who has already been authenticated in a certain main authentication process E2 is excluded from being matched in the next main authentication process E2.
[0121] For example, assume that multiple persons U1-U5 are authenticated in the preliminary authentication process E1, and then a person U1 is authenticated in a main authentication process E2 in which the person U1 among the multiple persons U1-U5 is the person to be matched. When a new second authentication process is then performed in which another person U2 is the person to be matched, it is preferable to exclude the person U1 from the people to be matched in the new second authentication process.
[0122] Therefore, when the controller 31 authenticates the person U1 (main authentication) in a certain main authentication process E2 (step S23 (FIG. 11)), it deletes information (face image information, etc.) of the main authenticated person U1 from the first authentication success database 32B (step S24 (FIG. 11)). Specifically, the face image 120 (or 320) of the person U1, the feature amount data of the face image, and the registered user ID of the person U1 are deleted from the first authentication success database 32B. This excludes the person U1 already authenticated in the previous second authentication process (main authentication process E2 using the person U1 as the reference person) from the matched persons used in the new main authentication process E2 (main authentication process E2 using a person U2 different from the person U1 as the matched person). Therefore, the matched persons in the main authentication process E2 are appropriately narrowed down, and the processing time in the main authentication process E2 can be reduced.
[0123] Furthermore, among the face image information registered in the first authentication success database 32B, face image information for which a predetermined period of time has passed since its registration is preferably excluded from the objects of matching in this authentication process E2.
[0124] For example, assume a situation in which a plurality of persons U1 to U5 are authenticated in the preliminary authentication process E1, the person U3 is authenticated in the main authentication process E2 using the person U3 as the reference person, and then a new main authentication process E2 is executed using another person U4 as the reference person. In this situation, if a predetermined period (e.g., several tens of seconds) has passed since the person U5 was authenticated in the preliminary authentication process E1, it is highly likely that the person U5 has moved to another location after being detected (after provisional authentication) in the preliminary authentication process E1. Therefore, it is preferable to exclude the person U5 from the persons to be matched in the new second authentication process. In other words, it is preferable to exclude the person U5 from the persons to be matched in the new second authentication process when a predetermined period (e.g., several tens of seconds) has passed since the person U5 authenticated in the first authentication process is registered in the first authentication success database 32B.
[0125] Therefore, the controller 31 deletes information of person U5, who was authenticated in the preliminary authentication process E1, from the first authentication success database 32B for whom a predetermined period of time has passed since registration in the first authentication success database 32B (more specifically, facial image information relating to the facial image of person U5, etc.) (step S25).
[0126] According to this, person U5 who remains without being officially authenticated in main authentication process E2 even after a predetermined period of time has passed since the time of detection in preliminary authentication process E1 (time of provisional authentication) because he / she moves to another place after detection in preliminary authentication process E1 is excluded from the persons to be matched in main authentication process E2. Therefore, the persons to be matched in main authentication process E2 are appropriately narrowed down, and the processing time can be reduced.
[0127] <1-5. Modifications, etc.> In the above embodiment, before a certain main authentication process E2, the process of FIG. 10 (one-to-many authentication process, etc.) is executed in parallel for a plurality of pre-captured images 110 (see FIG. 18). Then, the certain main authentication process E2 is executed based on the processing results of the plurality of one-to-many authentications (face image information related to persons provisionally authenticated in the plurality of one-to-many authentications). For example, the main authentication process E2 for person U2 is executed using face image information of a person authenticated in the preliminary authentication process E1 based on the pre-captured images 110 at each of times T11 to T16 as a collation target. This makes it possible to more reliably execute the preliminary authentication process E1 for the person to be authenticated in the main authentication process E2 before the main authentication process E2.
[0128] However, the present invention is not limited to this. For example, as shown in FIG. 19, the process of FIG. 10 (one-to-many authentication process, etc.) for the preliminary captured image 110 may be executed sequentially (not in parallel). Also, only the processing result of the process of FIG. 10 for a single preliminary captured image 110 may be used for a certain main authentication process E2 (for example, the main authentication process E2 for a person U1). In other words, the main authentication process E2 may be executed using a person (more specifically, face image information related to the face image of the person) authenticated in the preliminary authentication process E1 based on a single preliminary captured image 110 as a collation target. Also, the processing result of the process of FIG. 10 for a certain preliminary captured image 110 may be shared for a plurality of main authentication processes E2. For example, the processing result of the preliminary authentication process E1 based on the preliminary captured image 110 captured at the time T1 may be shared for both the main authentication process E2 for the person U1 and the main authentication process E2 for the next person U2.
[0129] In the above embodiment, the multiple preliminary captured images 110 are captured at different times, but the present invention is not limited thereto. For example, the multiple preliminary captured images 110 may be acquired at the same time. In the above embodiment, the multiple preliminary captured images 110 are captured at the same location, but the present invention is not limited thereto. For example, the multiple preliminary captured images 110 may be captured at locations adjacent to each other. More specifically, for example, the multiple preliminary captured images 110 may be captured by multiple cameras sharing a range equivalent to the field of view of the camera 21 (or a wider field of view). Then, the preliminary authentication process E1 may be executed based on the multiple preliminary captured images 110.
[0130] Also, in the above embodiment and the like, the entry subsystem 2 has been mainly described, but the same is true for the exit subsystem 3. Specifically, the exit subsystem 3 also executes a preliminary authentication process E1 and a main authentication process E2, similar to the entry subsystem 2. In detail, an authentication process is executed for a person who is about to pass through the gate 91 (exit gate 93).
[0131] For example, the controller 31 of the image processing device 30 in the exit subsystem 3 executes the main authentication process E2 of the exit subsystem 3 using a facial image of at least one person authenticated by the preliminary authentication process E1 of the exit subsystem 3 as the target for comparison.
[0132] More specifically, the exit subsystem 3 has its own first authentication success database 32B (a database separate from the first authentication success database 32B for the entrance subsystem 2). The controller 31 of the exit subsystem 3 registers the authentication process result of the preliminary authentication process E1 of the exit subsystem 3 in the first authentication success database 32B of the exit subsystem 3. In other words, the matching target information related to the main authentication process E2 of the entrance subsystem 2 and the matching target information related to the main authentication process E2 of the exit subsystem 3 are registered in different databases.
[0133] Then, in the exit subsystem 3, the main authentication process E2 is executed by comparing the facial image information registered in its own first authentication success database 32B (facial image information relating to the facial image of the person authenticated in the preliminary authentication process E1 of the exit subsystem 3).
[0134] According to this, the main authentication process E2 of each subsystem 2, 3 is executed based on unique matching target information (face image information) registered in a different database. Therefore, compared to the case where both subsystems 2, 3 share the first authentication success database 32B, it is possible to reduce the number of matching targets (number of people (number of face images)) in the main authentication process E2 of each subsystem.
[0135] In the above embodiment, the concept of the present invention is applied to both the entrance subsystem 2 and the exit subsystem 3, but is not limited to this. For example, the concept of the present invention may be applied to only one of the two subsystems 2, 3 (for example, the entrance subsystem 2). In a system that allows free exit, the concept may be applied only to the entrance subsystem 2.
[0136] In the above embodiment and the like, a case where a plurality of persons are authenticated (provisionally authenticated) in the first authentication process and one of the plurality of persons is authenticated (mainly authenticated) in the second authentication process is mainly exemplified (see FIGS. 6 and 7). However, the present invention is not limited to this. For example, a single person may (always) be authenticated (provisionally authenticated) in the first authentication process and the same single person may be authenticated (mainly authenticated) in the second authentication process (see FIGS. 4 and 5).
[0137] In the above embodiment, the camera 21 for capturing the first captured image 110 and the camera 22 for capturing the second captured image 210 are provided separately, but this is not limiting. Both the first captured image 110 and the second captured image 210 may be captured by one camera.
[0138] <2. Second embodiment> In the first embodiment, a single gate 91 is provided in each of the subsystems 2 and 3, but this is not limiting. For example, a plurality of gates 91 may be provided in the entrance subsystem 2. More specifically, the entrance subsystem 2 may be provided with a plurality of gates 91 (a plurality of entrance gates 92a, 92b, 92c, etc.) through which a plurality of people proceeding in the same direction (toward the station premises) can pass in parallel (see FIG. 21). Then, the above-mentioned preliminary authentication process E1 and main authentication process E2 may be performed for each of the plurality of gates 91, thereby performing authentication processes for people who are about to pass through each gate 91 (92a, 92b, 92c). The same applies to the exit subsystem 3.
[0139] In the second embodiment, such an aspect will be described.
[0140] Fig. 20 is a block diagram showing an image processing system 1 (1B) according to the second embodiment. Fig. 21 is a diagram showing the vicinity of multiple entrance gates 92 provided in an entrance subsystem 2 (2B) according to the second embodiment as viewed from above. In Fig. 21, the field of view of each camera 21 is indicated by an arc-shaped curve with a double-headed arrow.
[0141] The entrance subsystem 2 (2B) according to the second embodiment includes a plurality of authentication devices 10 (specifically, three authentication devices). More specifically, an authentication device 10a for a first gate 92a, an authentication device 10b for a second gate 92b, and an authentication device 10c for a third gate 92c are provided. Each authentication device 10 has a similar configuration to the authentication device 10 of the first embodiment (see FIG. 3).
[0142] Each authentication device 10 (and image processing device 30, etc.) may execute the same process as in the first embodiment. However, in this second embodiment, a certain authentication device 10 (for example, 10a) executes the main authentication process E2 by using face image information on a person authenticated in the preliminary authentication process E1 in another authentication device 10 (10b).
[0143] Specifically, the authentication device 10a utilizes not only the processing results of the preliminary authentication process E1 based on the preliminary captured image 110 taken by the camera 21 for the entrance gate 92a, but also the processing results of the preliminary authentication process E1 based on the preliminary captured image 110 taken by the camera 21 for the adjacent entrance gate 92b.
[0144] In detail, the authentication device 10a (specifically, its image processing device 30 (30a)) executes a preliminary authentication process E1 based on a preliminary captured image 110 captured by a camera 21 for the entrance gate 92a. For example, in a situation as shown in FIG. 21, the authentication device 10a executes the preliminary authentication process E1 based on the preliminary captured image 110 including the persons U3 and U4, and authenticates (provisionally authenticates) the persons U3 and U4. Then, the authentication device 10a registers face image information on the provisionally authenticated persons U3 and U4 in the first authentication success database 32B for the authentication device 10a itself. Furthermore, the authentication device 10a also registers face image information on the provisionally authenticated persons U3 and U4 in the first authentication success database 32B for the adjacent authentication device 10b.
[0145] Similarly, the authentication device 10c (specifically, its image processing device 30 (30c)) executes a preliminary authentication process E1 based on a preliminary captured image 110 captured by a camera 21 for the entrance gate 92c. For example, in a situation as shown in FIG. 21, the authentication device 10c executes the preliminary authentication process E1 based on the preliminary captured image 110 including the persons U5 and U6, and authenticates (provisionally authenticates) the persons U5 and U6. Then, the authentication device 10c registers face image information on the provisionally authenticated persons U5 and U6 in the first authentication success database 32B for the authentication device 10c itself. Furthermore, the authentication device 10c also registers face image information on the provisionally authenticated persons U5 and U6 in the first authentication success database 32B for the adjacent authentication device 10b.
[0146] Moreover, the authentication device 10b (specifically, its image processing device 30 (30b)) executes a preliminary authentication process E1 based on a preliminary captured image 110 captured by a camera 21 for the entrance gate 92b. For example, in a situation as shown in FIG. 21, the authentication device 10b executes the preliminary authentication process E1 based on the preliminary captured image 110 including the persons U1 and U2, and authenticates (provisionally authenticates) the persons U1 and U2. Then, the authentication device 10b registers face image information on the provisionally authenticated persons U1 and U2 in the first authentication success database 32B for its own device 10b. Furthermore, the authentication device 10b also registers face image information on the provisionally authenticated persons U1 and U2 in the first authentication success database 32B for the adjacent authentication device 10a and the first authentication success database 32B for another adjacent authentication device 10c.
[0147] As a result, face information on persons U1 to U4 is registered in the first authentication success database 32B for the authentication device 10a, and face information on persons U1 to U6 is registered in the first authentication success database 32B for the authentication device 10b. Moreover, face information on persons U1, U2, U5, and U6 is registered in the first authentication success database 32B for the authentication device 10c.
[0148] Thereafter, the authentication device 10a executes the main authentication process E2 based on the main captured image 210 captured by the camera 22 for the entrance gate 92a. More specifically, the authentication device 10a executes the matching process using face image information registered in the first authentication success database 32B of the authentication device 10a as the target for matching. Here, as described above, the first authentication success database 32B of the authentication device 10a registers not only face image information of persons authenticated in the preliminary authentication process E1 of the authentication device 10a itself, but also face image information of persons authenticated in the preliminary authentication process E1 of the adjacent authentication device 10b.
[0149] For example, in a situation as shown in FIG. 21, the authentication device 10a executes the main authentication process E2 based on the main image 210 including the person U3, and authenticates (mainly authenticates) the person U3. Thereafter, while waiting for the timing of capturing the next main image 210, it is considered that the person U2 waiting in line at the gate 92b suddenly changes course and moves toward the gate 92a. In this case, the main image 210 including the person U2 is captured by the camera 22 at the entrance gate 92a, and the authentication process in the main authentication process E2 is executed with the person U2 as the matching source. At this time, the people to be matched in the main authentication process E2 also include the people U1 and U2 who were provisionally authenticated in the preliminary authentication process E1 of the adjacent authentication device 10b. As a result, the person U2 can be properly authenticated in the main authentication process E2.
[0150] In this way, the main authentication process E2 based on the actual photographed image 210 of the person U2 attempting to pass through the entrance gate 92a is executed using facial image information on the facial image of the person U2 in the actual photographed image 210 as a comparison source. In particular, the main authentication process E2 is executed using facial image information on the facial images of the persons U3 and U4 authenticated in the preliminary authentication process E1 for the entrance gate 92a as well as facial image information on the facial images of the persons U1 and U2 authenticated in the preliminary authentication process E1 for the adjacent entrance gate 92b as a comparison target. In other words, the main authentication process E2 is executed using facial image information of the persons (U3, U4, U1, U2) authenticated in both of the two preliminary authentication processes E1 for both gates 92a and 92b as a comparison target.
[0151] If each authentication device 10 independently executes the main authentication process E2 using only the preliminary authentication process E1 of its own device, a person U2 who suddenly changes course may not be provisionally authenticated in the preliminary authentication process E1 for the entrance gate 92a, and thus may not be authenticated in the main authentication process E2.
[0152] In contrast, according to the second embodiment, facial image information related to the processing result of the preliminary authentication process E1 in the adjacent authentication device 10b (specifically, facial image information of the person U2 who was waiting in the queue at the adjacent entrance gate 92b) is also included in the comparison targets of the main authentication process E2 for the gate 92a. Therefore, it is possible to avoid a situation where the person U2 cannot be authenticated in the main authentication process E2 for the gate 92a.
[0153] The same applies when a person U6 who was waiting in the queue at the gate 92c suddenly changes course and moves toward the gate 92b, and the main authentication process E2 based on the main captured image 210 including the person U6 is executed by the authentication device 10b. In this case, face image information related to the processing result of the preliminary authentication process E1 in the adjacent authentication device 10c (specifically, face image information of the person U6 who was waiting in the queue at the entrance gate 92c) is also included in the comparison targets. Therefore, it is possible to avoid a situation where the person U6 cannot be authenticated in the main authentication process E2 for the gate 92b.
[0154] In this embodiment, each authentication device 10 is provided for each gate 91, but the present invention is not limited to this. For example, the preliminary authentication process E1 and the main authentication process E2 for each of the multiple gates 91 may be executed by a single authentication device 10. In this case, similarly to the above, an individual first authentication success database 32B may be provided for each gate, and the main authentication process E2 for a certain gate may be executed by adding face image information of a person authenticated in the preliminary authentication process E1 for an adjacent gate to the matching target.
[0155] Also, while the entrance subsystem 2 (2B) has been mainly described here, the same is true for the exit subsystem 3 (3B).
[0156] <3. Third embodiment> The third embodiment is a modification of the second embodiment. The following description will focus on the differences from the second embodiment. As in the second embodiment, the description will be focused on the entrance subsystem 2.
[0157] In the third embodiment as well, a plurality of gates 91 (a plurality of entrance gates 92a, 92b, 92c, etc.) are provided through which a plurality of people proceeding in the same direction can pass in parallel (almost simultaneously) (see FIG. 24).
[0158] However, in the third embodiment, instead of the multiple cameras 21 that are provided individually for the multiple gates 91 in the second embodiment, a common camera 23 is provided for the multiple gates 91 (see FIG. 24). Also, the preliminary authentication process E1 for each gate 91 is not performed for each gate, but is executed commonly for the multiple gates 91.
[0159] FIG. 22 is a block diagram showing an image processing system 1 (1C) according to the third embodiment. FIG. 23 is a block diagram showing an authentication device 10 according to the third embodiment. As shown in FIG. 23, each authentication device 10 has a camera 22 that captures a main captured image 210, but does not have a camera 21 that captures a preliminary captured image 110. As shown in FIG. 22, a camera 23 that captures the preliminary captured image 110 is provided in common for the multiple authentication devices 10 (10a, 10b, 10c) of the entrance subsystem 2 (2C). Then, a common preliminary authentication process E1 based on the preliminary captured image 110 captured by the camera 23 is executed by one authentication device 10a of the multiple authentication devices 10 (10a, 10b, 10c).
[0160] Furthermore, each authentication device 10 does not have an individual first authentication success database 32B (see FIG. 23). Facial image information of a person authenticated by the common preliminary authentication process E1 is registered (stored) in a common first authentication success database 38 (see FIG. 22) in the entrance subsystem 2 (2C). The common first authentication success database 38 (see FIG. 22) may be provided in a server device or the like. However, it is not limited to this and may be provided in one authentication device 10a.
[0161] In the third embodiment as well, a preliminary authentication process E1 and a main authentication process E2 are executed for each of a plurality of gates 91 through which a plurality of persons proceeding in the same direction can pass in parallel. This executes authentication processes for persons attempting to pass through each gate 91. However, the preliminary authentication process E1 for each gate 91 uses as a matching source face images in a common preliminary captured image 110 capturing a plurality of persons heading toward the plurality of gates 91, and is executed as a process common to the plurality of gates.
[0162] For example, in the situation shown in FIG. 24, the following processing is executed.
[0163] First, a common preliminary captured image 110 is acquired by the camera 23. The camera 23 is configured to include, for example, a wide-angle lens and has a wide field of view. When a plurality of persons U1 to U6 are captured by the common preliminary captured image 110, a common preliminary authentication process E1 based on the common preliminary captured image 110 is executed. The common preliminary authentication process E1 can also be expressed as a preliminary authentication process E1 for each of the plurality of gates 91.
[0164] Then, when a plurality of persons U1 to U6 are provisionally authenticated by the common preliminary authentication process E1, face image information relating to the persons U1 to U6 is registered in a common first authentication success database .
[0165] Thereafter, the main authentication process E2 for each gate 91 is executed using as a comparison source face image information for each face image in each actual captured image 210 captured at a timing after the capture timing of the preliminary captured image 110 of each person attempting to pass through that gate. Each main authentication process E2 is executed as a process unique to each gate 91. More specifically, the main authentication process E2 for each gate 91 is executed by the authentication device 10 corresponding to that gate 91. In addition, the main authentication process E2 for each gate 91 is executed using as a comparison target face image information for the face images of multiple persons U1 to U6 authenticated in the preliminary authentication process E1 common to the multiple gates 91.
[0166] In the third embodiment, a common preliminary authentication process E1 is executed for a plurality of gates 91. Therefore, it is possible to avoid duplicated processing compared to the case where individual preliminary authentication processes E1 are executed for a plurality of gates 91. Therefore, it is possible to improve the efficiency of processing in the entire system (at least the entire subsystem).
[0167] Also, for example, after the main authentication process E2 of a person U3 who is about to pass through the entrance gate 92a is completed, a person U2 who is waiting in a queue at the gate 92b suddenly changes course and moves toward the gate 92a, and the main authentication process E2 based on the main captured image 210 including the person U2 is executed by the authentication device 10a. Here, all of the multiple persons U1 to U6 who may enter the multiple gates 91 are captured in the preliminary captured image 110 by the common camera 23 having a wide viewing angle. Then, the multiple persons U1 to U6 are authenticated by the preliminary authentication process E1 based on the preliminary captured image 110, and the facial image information of the multiple persons U1 to U6 is registered in the first authentication success database 38. Furthermore, the main authentication process E2 for each gate is executed by checking the facial image information of the multiple persons U1 to U6 registered in the first authentication success database 38. Therefore, even in this authentication process E2 in which a person U2 who suddenly changes course from one of the multiple gates 91 to another gate is the person to be authenticated, it is possible to properly authenticate the person U2.
[0168] In the above description, the common preliminary authentication process E1 based on the preliminary captured image 110 captured by the camera 23 is executed by one authentication device 10a among the multiple authentication devices 10, but this is not limiting. For example, the common preliminary authentication process E1 may be executed by a device provided separately from the multiple authentication devices 10 (for example, a dedicated device that executes only the common preliminary authentication process E1, etc.).
[0169] In addition, although each authentication device 10 is provided for each gate 91 here, this is not limited to the above. For example, each main authentication process E2 for each of the multiple gates 91 may be executed by a single authentication device 10 (e.g., 10a).
[0170] Also, while the entrance subsystem 2 (2C) has been mainly described here, the same is true for the exit subsystem 3 (3C).
[0171] <4. Fourth embodiment> In each of the above-mentioned embodiments, the concept of the present invention is applied to the entrance subsystem 2, etc., but is not limited thereto. For example, the concept of the present invention may be applied to an absence management system that manages the absence of users of personal computers (PCs), etc. In the fourth embodiment, such an aspect will be described.
[0172] In this absence management system, it is possible to detect when a user leaves the desk, and to confirm the identity of the user. Specifically, for example, it is possible to periodically check whether the user has been replaced (whether another user has used the PC while the real user was away, etc.). The idea of the present invention is applied to the detection of the identity of the user, etc.
[0173] In the fourth embodiment, a login authentication process is performed as the first authentication process E1, and a periodic check authentication process is performed as the second authentication process E2.
[0174] 25 is a diagram showing an image processing system 1 (specifically, an absence management system 1D) according to the fourth embodiment. The absence management system 1D includes an authentication device 10D.
[0175] The authentication device 10D has a similar configuration to the authentication device 10 of the first embodiment, except that the authentication device 10D does not have a camera.
[0176] The authentication device 10D is capable of performing network communication with a remote personal computer 60. The authentication device 10D is capable of accepting a login request (a login request via a network) from the personal computer 60 to a certain system (another system on the network) and performing authentication processing (authentication processing using a face image) for the login request.
[0177] The personal computer 60 includes a controller 61 having a CPU and the like, an imaging unit 62 having a camera 63, and a communication unit 64 for performing network communication with other devices.
[0178] For example, the authentication device 10D is realized as a server device, and the personal computer 60 is realized as a client device.
[0179] The authentication device 10D executes the first authentication process E1 in response to a login request (login request via a network) from the personal computer 60 (see the leftmost part of FIG. 26). Specifically, at the time of the login request, a photographed image 110 of a person (user) U1 present in front of the personal computer 60 is taken by the camera 63 of the personal computer 60. Then, the photographed image 110 is transmitted from the personal computer 60 to the authentication device 10D, and the authentication device 10D executes the first authentication process E1 based on the photographed image 110. Specifically, the person U1 is authenticated by an authentication process (1:N authentication process) in which face information of the person U1 extracted from the photographed image 110 is compared with face image information of a large number (N people) of people (a large number of registered users of the absence management system) registered in advance. Then, the person U1 is permitted to log in. Furthermore, face image information of the person U1 authenticated in the first authentication process E1 is registered in the first authentication success database 32B.
[0180] Thereafter, the second authentication process E2 and the like are executed periodically (for example, every several seconds to several tens of seconds) (see the central part of FIG. 26). Specifically, the camera 63 of the personal computer 60 captures a photographed image 210 of the person U1 who is present in front of the personal computer 60 at each time point. Then, the photographed image 210 is transmitted from the personal computer 60 to the authentication device 10D, and the authentication device 10D executes the second authentication process E2 based on the photographed image 210. Specifically, the person U1 is authenticated by an authentication process (one-to-one authentication process) in which face information of the person U1 extracted from the photographed image 210 is collated with face image information of the person U1 authenticated in the first authentication process E1 (face image information registered in the first authentication success database 32B). In this embodiment, the first photographed image 110 and the second photographed image 210 are photographed by the same camera 63.
[0181] According to such a process, since the second authentication process E2 performs one-to-one authentication instead of one-to-many authentication, the authentication process time in the second authentication process E2 can be very short, i.e., the processing load on the authentication device 10D can be reduced.
[0182] In addition, in the periodic second authentication process E2, if a matching process is performed using a person (such as U2) other than the person U1 as a matching source based on a captured image 210 capturing the person U2, the authentication process of the person U1 fails. Specifically, it is determined that a person U2 other than the original person U1 is present in front of the personal computer 60 (see the right part of FIG. 26). In this case, the authentication device 10D executes a forced logout process for the personal computer 60.
[0183] In this way, the initial authentication process (login authentication process) is performed as the first authentication process, and then the confirmation authentication process (periodic confirmation authentication process) is performed as the second authentication process. This aspect also makes it possible to achieve rapid processing in the second authentication process.
[0184] In the fourth embodiment, a mode involving logging in to another system (an external system of the personal computer 60) via a network is exemplified, but the present invention is not limited to this.
[0185] For example, the above idea may be applied to the login process (initial authentication process) to the personal computer 60 itself and the subsequent periodic confirmation process (user identity confirmation process). Specifically, the various processes executed by the authentication device 10D in the fourth embodiment may be executed by the personal computer 60.
[0186] Alternatively, the above idea may be applied to other systems. For example, in an AI diagnostic system equipped with a patient-side subsystem, the idea of the present invention may be applied to the authentication process of a patient. In particular, the same process as in the fourth embodiment may be performed between an authentication device 10 similar to the authentication device 10D according to the fourth embodiment and a personal computer 60 used by the patient. More specifically, an initial authentication process (initial authentication process of the patient) may be performed as the first authentication process E1, and then a confirmation authentication process (periodic identity authentication process of the patient) may be performed as the second authentication process E2. This makes it possible to realize a rapid process in the second authentication process.
[0187] <5.Other> Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described contents.
[0188] For example, the gate in each of the above embodiments is not limited to a passage-type (passage opening / closing type) gate device installed in a station, airport, theater, etc., but may be a door opening / closing type (locking / unlocking type) gate device that opens and closes a door by locking and / or unlocking a door installed at the entrance / exit of a room, etc. In other words, the entrance gate may be a door opening / closing type gate device for entering a room, etc. (predetermined area), and the exit gate may be a door opening / closing type gate device for exiting a room, etc. (predetermined area). [Explanation of symbols]
[0189] 1. Image Processing System 2. Entry Subsystem 3 Exit Subsystem 10 Authentication Device 21,22,23 Camera 30 Image processing device 32A User Registration Database 32B,38 First authentication success database 60 Personal Computers 63 Camera 91 Gate device 92 Entrance gate device 93 Exit Gate Device 110 First captured image (preliminary captured image) 120,121~125,220 Face images 210 Second image (main image) 320 template face images E1 First authentication process (pre-authentication process) E2 Second authentication process (main authentication process)
Claims
1. An image processing device, comprising: a control unit that acquires at least one first captured image and a second captured image captured after the at least one first captured image; Equipped with The control unit is executing a first authentication process for authenticating a person by comparing face image information of the person included in the at least one first photographed image with face image information of a large number of people registered in advance; executing a second authentication process for authenticating the person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in the second captured image with face image information of at least one person authenticated by the first authentication process; the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process, a first control unit which is the control unit, performs the first authentication process and the second authentication process for a first gate among a plurality of gates through which a plurality of persons proceeding in the same direction can pass in parallel, thereby performing an authentication process for a person who is about to pass through the first gate; the first control unit of the image processing device or a second control unit which is a control unit of a device other than the image processing device, executes the first authentication process and the second authentication process for a second gate of the plurality of gates; an image processing device, characterized in that the second authentication process for the first gate is executed using facial image information regarding a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information regarding the person authenticated in the first authentication process for the first gate and facial image information regarding the person authenticated in the first authentication process for the second gate as a comparison target.
2. 2. The image processing device according to claim 1, characterized in that in the first authentication process, a process of authenticating a person by comparing facial image information of a person included in the at least one first captured image with facial image information of the multiple persons is repeatedly executed, thereby authenticating a plurality of persons as the at least one person.
3. The at least one first captured image includes a plurality of first captured images, 3. The image processing device according to claim 1, wherein in the first authentication process, a process of authenticating a person by comparing facial image information of a person included in any one of the plurality of first captured images with facial image information of the plurality of persons is repeatedly executed, thereby authenticating a plurality of persons as the at least one person.
4. 2. The image processing device according to claim 1, wherein in the first authentication process, the one person is authenticated by comparing facial image information of the one person included in the at least one first captured image with facial image information of the multiple persons.
5. 5. The image processing device according to claim 1, wherein the facial image information to be compared in the second authentication process is facial image information of the at least one person authenticated in the first authentication process and is facial image information extracted from the at least one first captured image.
6. 5. The image processing device according to claim 1, wherein the facial image information to be compared in the second authentication process is facial image information corresponding to the at least one person authenticated in the first authentication process among template facial image information constituted by facial image information of the plurality of persons registered in advance.
7. the second authentication process is a process for authenticating the reference person by determining a person included in the second captured image as a reference person and determining the at least one person authenticated in the first authentication process as a target person, and matching face image information regarding the reference person with face image information regarding the target person, 7. The image processing device according to claim 1, wherein, when a plurality of persons are authenticated in the first authentication process, after authenticating a first person among the plurality of persons by the second authentication process in which the first person is the reference person, when executing a new second authentication process in which a second person among the plurality of persons other than the first person is the reference person, the control unit excludes the first person from the reference persons in the new second authentication process.
8. the second authentication process is a process for authenticating the reference person by determining a person included in the second captured image as a reference person and determining the at least one person authenticated in the first authentication process as a target person, and matching face image information regarding the reference person with face image information regarding the target person, 7. The image processing device according to claim 1, wherein, in a case where a plurality of persons are authenticated in the first authentication process, after authenticating a first person among the plurality of persons as the reference person in the second authentication process and after a predetermined period has elapsed since a second person among the plurality of persons was authenticated in the first authentication process, when executing a new second authentication process in which a person other than the first person is the reference person, the control unit excludes the second person from the reference persons in the new second authentication process.
9. the control unit registers information on the person authenticated in the first authentication process as the matching target information in a database that temporarily stores matching target information of the second authentication process, as the matching target information, at any time; 9. The image processing device according to claim 1, wherein in the second authentication process, the person to be authenticated is authenticated by matching facial image information of the person to be authenticated that is included in the second captured image with the matching target information stored in the database.
10. 10. The image processing device according to claim 1, wherein the first authentication process is a process that requires a longer processing time than the second authentication process if matching is performed with the same number of matching targets, and that achieves authentication with a higher accuracy than the second authentication process.
11. 11. The image processing device according to claim 1, wherein an amount of data of the feature vectors extracted from the face image for matching in the first authentication process is greater than an amount of data of the feature vectors extracted from the face image for matching in the second authentication process.
12. an image capturing unit configured to capture at least one first captured image and a second captured image captured after the at least one first captured image; a control unit that executes a first authentication process for authenticating a person by comparing face image information of the person included in the at least one first captured image with face image information of a large number of people registered in advance, and executes a second authentication process for authenticating the person to be authenticated by comparing face image information of the person to be authenticated included in the second captured image with face image information of at least one person authenticated by the first authentication process; Equipped with the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process, the first authentication process and the second authentication process are executed for each of a plurality of gates through which a plurality of persons traveling in the same direction can pass in parallel, thereby executing authentication processes for persons attempting to pass through each of the gates; an image processing system characterized in that the second authentication process for a first gate of the plurality of gates is executed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information of the person authenticated in the first authentication process for the first gate and facial image information of the person authenticated in the first authentication process for a second gate of the plurality of gates as a comparison target.
13. 13. The image processing system according to claim 12, wherein the photographing section comprises a camera that photographs the at least one first photographed image and a camera that photographs the second photographed image, the camera being separate from the first photographed image.
14. a) executing a first authentication process by a computer to authenticate a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; b) executing a second authentication process by the computer to authenticate the person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in a second captured image captured after the at least one first captured image, with face image information of at least one person authenticated by the first authentication process; Equipped with the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process, The step a) comprises: a-1) a step in which a first computer executes the first authentication process for a first gate among a plurality of gates through which a plurality of persons traveling in the same direction can pass in parallel; a-2) a second computer executing the first authentication process for a second gate of the plurality of gates; a-3) the first computer registers face image information of at least one person authenticated by the first authentication process related to the first gate in a first database for the first gate; a-4) the second computer registers face image information of at least one person authenticated by the first authentication process related to the second gate in both a second database for the second gate and the first database for the first gate; Equipped with The step b) comprises: b-1) the first computer executes the second authentication process for the first gate among the plurality of gates based on face image information registered in the first database; b-2) the second computer executes the second authentication process for the second gate among the plurality of gates based on face image information registered in the second database; Equipped with The image processing method is characterized in that the second authentication processing for the first gate in step b-1) is executed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a matching source, and using both facial image information relating to the person authenticated in the first authentication processing for the first gate and facial image information relating to the person authenticated in the first authentication processing for the second gate as a matching target.
15. a) executing a first authentication process by a computer to authenticate a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; b) executing a second authentication process by the computer to authenticate the person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in a second captured image captured after the at least one first captured image, with face image information of at least one person authenticated by the first authentication process; Equipped with the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process, The step a) comprises: a-1) a step in which the computer executes the first authentication process for a first gate among a plurality of gates through which a plurality of persons traveling in the same direction can pass in parallel; a-2) the computer executes the first authentication process with respect to a second gate among the plurality of gates; a-3) a step in which the computer registers face image information of at least one person authenticated by the first authentication process related to the first gate in a first database for the first gate; a-4) a step in which the computer registers face image information of at least one person authenticated by the first authentication process related to the second gate in both a second database for the second gate and the first database for the first gate; Equipped with The step b) comprises: b-1) the computer executes the second authentication process for the first gate among the plurality of gates based on face image information registered in the first database; b-2) the computer executes the second authentication process for the second gate among the plurality of gates based on face image information registered in the second database; Equipped with The image processing method is characterized in that the second authentication processing for the first gate in step b-1) is executed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a matching source, and using both facial image information relating to the person authenticated in the first authentication processing for the first gate and facial image information relating to the person authenticated in the first authentication processing for the second gate as a matching target.
16. a) executing a first authentication process for authenticating a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; b) executing a second authentication process for authenticating a person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in a second captured image captured after the at least one first captured image, with face image information of at least one person authenticated by the first authentication process; A program for causing a computer to execute the above, the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process, The step a) comprises: a-1) executing the first authentication process for a first gate among a plurality of gates through which a plurality of persons traveling in the same direction can pass in parallel; a-2) executing the first authentication process for a second gate among the plurality of gates; Equipped with The step b) comprises: b-1) performing the second authentication process for the first gate of the plurality of gates; b-2) executing the second authentication process for the second gate of the plurality of gates; Equipped with The second authentication process for the first gate in step b-1) is executed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information relating to the person authenticated in the first authentication process for the first gate and facial image information relating to the person authenticated in the first authentication process for the second gate as a comparison target.
17. a) executing a first authentication process for authenticating a person by comparing face image information of the person included in at least one first captured image with face image information of a large number of people registered in advance; b) executing a second authentication process for authenticating a person to be authenticated by comparing face image information of the person to be authenticated, the face image information being included in a second captured image captured after the at least one first captured image, with face image information of at least one person authenticated by the first authentication process; A program for causing a first and a second computer to execute the above, the number of people to be matched in the second authentication process is smaller than the number of people to be matched in the first authentication process, The step a) comprises: a-1) a step in which the first computer executes the first authentication process for a first gate among a plurality of gates through which a plurality of persons traveling in the same direction can pass in parallel; a-2) the second computer executing the first authentication process with respect to a second gate among the plurality of gates; Equipped with The step b) comprises: b-1) the first computer executing the second authentication process for the first gate of the plurality of gates; b-2) the second computer executes the second authentication process for the second gate of the plurality of gates; Equipped with The second authentication process for the first gate in step b-1) is executed using facial image information relating to a facial image in a captured image of a person attempting to pass through the first gate as a comparison source, and using both facial image information relating to the person authenticated in the first authentication process for the first gate and facial image information relating to the person authenticated in the first authentication process for the second gate as a comparison target.
Citation Information
Patent Citations
Image processing system
JP2008052549A
Checking system
JP2019057284A
Gate device, and arrangement structure of gate device
JP2019071126A
Information processing device, information processing system, information processing method, and information processing program
JP2020057191A
Facial verification system, facial verification method, and recording medium
WO2017146160A1
