Verification device, verification system, and verification method

Abstract

This invention contributes to providing a matching device, matching system, and matching method that can improve the processing speed of matching using facial images of people passing through a specific area. [Solution] The matching device is a matching device for regulating the flow of people, and in a path where there is a flow of people from a first area to a second area located upstream from a position where the flow of people should be regulated, the device includes a processing unit that performs a second face image matching using a first candidate face image narrowed down from a plurality of face images in a face image database and a face image included in a second image taken of the second area, based on the result of a first face image matching using a face image included in a first image taken of the first area and a plurality of face images included in a face image database, and a communication unit that outputs the result of the second face image matching.

Description

Technical Field

[0001] The present disclosure relates to a collation device, a collation system, and a collation method.

Background Art

[0002] There is known a technique for managing the entry and exit of people passing through gates installed at stations, airports, etc. by face authentication. Patent Document 1 discloses a technique for realizing smooth passage of people through a gate. The technique of Patent Document 1 extracts the feature amounts of objects in a captured image obtained by capturing the area in front of the gate, and performs a collation determination based on the pre-registered collation information (information regarding the feature amounts of people, etc.) and the estimated distance from a person approaching the gate to the gate. According to the technique of Patent Document 1, face authentication is performed after confirming whether the estimated distance is a distance suitable for collation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Since the time it takes for a person to pass through a gate is about several seconds, when collating (or authenticating) a person passing through a gate by a face image, processing in a short time is expected.

[0005] Non-limiting embodiments of the present disclosure contribute to providing a collation device, a collation system, and a collation method that can improve the processing speed of collation (hereinafter sometimes abbreviated as "face image collation" or "face image authentication") using a face image of a person passing through a specific area such as a gate.

Means for Solving the Problems

[0006] A matching device according to one embodiment of the present disclosure is a matching device for regulating the flow of people, and in a path where there is a flow of people from a first area toward a second area located upstream of a position where the flow of people should be regulated, the device includes a processing unit that performs a second face image matching using a first face image matching result using a face image included in a first image taken of the first area and a plurality of face images included in a face image database, and a face image included in a second image taken of the second area, based on the result of a first face image matching using a face image included in a first image taken of the first area and a plurality of face images included in a face image database, and a communication unit that outputs the result of the second face image matching.

[0007] A matching system according to one embodiment of the present disclosure is a matching system for regulating the flow of people, comprising: a first camera that photographs a first region in the flow of people moving from a first region to a second region located upstream of a position where the flow of people should be regulated; a second camera that photographs the second region; a first matching device that performs a first face image matching using a face image included in a first image captured by the first camera and a plurality of face images included in a face image database; and a second matching device that performs a second face image matching using a first candidate face image narrowed down from a plurality of face images included in the face image database based on the result of the first face image matching and a face image included in a second image captured by the second camera.

[0008] A matching method according to one embodiment of the present disclosure is a matching method for regulating the flow of people, and in a path where there is a flow of people from a first area toward a second area located upstream from a position where the flow of people should be regulated, a second face image matching is performed using a first candidate face image narrowed down from the multiple face images in the face image database and a face image in a second image taken toward the second area, based on the result of a first face image matching using a face image included in a first image taken toward the first area and a plurality of face images included in a face image database, and the result of the second face image matching is output.

[0009] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0010] According to one embodiment of this disclosure, the processing speed of matching facial images of people passing through a specific area can be improved.

[0011] Further advantages and effects of one embodiment of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by several embodiments and features described in the specification and drawings, but not all of them are necessarily provided in order to obtain one or more identical features. [Brief explanation of the drawing]

[0012] [Figure 1] A diagram showing an example configuration of the facial recognition system related to this disclosure. [Figure 2] Diagram showing an example hardware configuration of a facial recognition server and matching device. [Figure 3] Diagram showing an example of the functional configuration of a facial recognition server and matching device. [Figure 4] Diagram showing an example of installing multiple cameras at a gate. [Figure 5] A diagram illustrating the operation of the facial recognition system. [Figure 6] A flowchart illustrating an example of how a facial recognition system works. [Figure 7] A flowchart illustrating the behavior when matching cannot be performed using the medium-range refinement search process. [Figure 8] A flowchart illustrating an example of what happens when close-range facial recognition fails. [Figure 9] A diagram illustrating how people are photographed by long-range cameras. [Figure 10] A diagram illustrating how a person is photographed by a mid-range camera. [Figure 11] A diagram showing a person being photographed by a close - range camera [Figure 12] A diagram for explaining the relationship between the walking speed of a person passing through a gate and face authentication processing [Figure 13] A diagram for explaining a modified example of a gate

Embodiments for Carrying Out the Invention

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same function are denoted by the same reference numerals, and redundant description is omitted.

[0014] (Embodiment) FIG. 1 is a diagram showing a configuration example of a face authentication system according to the present disclosure. The face authentication system 100 according to the present embodiment is, for example, a system that controls a gate (such as an entrance gate or a ticket gate) installed at an entrance / exit of a facility such as an airport, a station, or an event venue. In the face authentication system 100 according to the present embodiment, illustratively, the management of the entry and exit of users who use the facility is executed by face authentication. For example, when a user passes through the gate and enters the facility, it is determined by face authentication whether the user is a person permitted to enter the facility. Also, when a user passes through the gate and exits the facility, it is determined by face authentication whether the user is a person permitted to exit the facility. Note that "face authentication" may be regarded as a concept included in "verification using a face image".

[0015] The face authentication system 100 includes a gate control device 20 and a face authentication server 200. Further, the face authentication system 100 includes a plurality of cameras 1 for face photographing, a QR code (registered trademark) reader 2, a passage management photo - electric sensor 3, an opening / closing door mechanism 4, an entrance guidance indicator 5, a passage guidance LED (Light Emitting Diode) 6, and a guidance display 7. Also, the face authentication system 100 includes a speaker 8, an interface board 9, an interface driver 10, a network hub 30, etc.

[0016] The gate control device 20 is connected to the network hub 30 and can communicate with the server 200 via the network hub 30 and the network 300. The server 200 performs processing related to face authentication. Therefore, the server 200 may be referred to as the face authentication server 200. The gate control device 20 is, for example, a device that controls gates (such as entrance gates, ticket gates, etc.) installed in facilities such as airports, railway stations, and event venues. The gate control device 20 controls the opening and closing door mechanism 4 of the gate. For example, for a person permitted by face authentication, the gate is opened. On the other hand, for a person who fails face authentication, the gate is closed.

[0017] In face authentication, for example, information on face images of hundreds of thousands to tens of millions of individuals is used. This information is, for example, recorded in the face authentication server 200. Hereinafter, the information used for face authentication may be referred to as "authentication information" or "matching information". For example, the authentication information may be registered in the face authentication server 200 in advance through the user procedures of users who use the face authentication service.

[0018] The matching device 21 is communicably connected to the face authentication server 200 via the network 300. The matching device 21 matches the face image of a person passing through the gate with the face image of the population included in the registered authentication information to authenticate the person passing through the gate.

[0019] Matching means determining whether the pre-registered face image and the face image of the person passing through the gate match, or whether the pre-registered face image and the face image of the person passing through the gate are face images of the same person, by comparing the registered face image with the face image of the person passing through the gate.

[0020] On the other hand, authentication means proving to the outside (for example, the gate) that a person with a face image that matches the pre-registered face image is the person himself / herself (in other words, a person who may be permitted to pass through the gate).

[0021] However, in this disclosure, the terms "verification" and "authentication" may be used interchangeably.

[0022] For example, the matching process compares the feature points of each individual's face image, which have been registered in advance, with the feature points extracted from the face image detected by the face detection process, to identify whose face is in the image data. The gate control device 20 controls the gate (for example, the opening and closing operation of the opening and closing door mechanism 4) according to the result of this authentication. The matching device 21 only needs to be positioned to communicate with the face authentication server 200, and may be incorporated inside the gate control device 20 or provided outside the gate control device 20.

[0023] QR code reader 2 reads a QR code containing information that identifies the person passing through the gate. For example, among the people passing through the gate, those who do not use facial recognition for entry and exit management are authenticated by having QR code reader 2 read their QR code.

[0024] The passage management photoelectric sensor 3 detects whether a person has entered the gate and whether a person authorized to pass through the gate has finished passing through. For example, the passage management photoelectric sensor 3 may be installed at multiple locations, including locations for detecting whether a person has entered the gate and locations for detecting whether a person has finished passing through the gate. The passage management photoelectric sensor 3 is connected to the gate control device 20, for example, via an interface board 9. Note that the method for detecting the entry and passage of a person is not limited to using a photoelectric sensor; it can also be implemented by other methods, such as monitoring the movement of a person captured by a camera installed on the ceiling. In other words, a photoelectric sensor is just one example of a sensor for passage management, and other sensors may be used.

[0025] The opening and closing door mechanism 4 is connected to the gate control device 20, for example, via the interface board 9.

[0026] The entry guidance indicator 5 indicates whether or not passage to gate 400 is permitted. The entry guidance indicator 5 is connected to the gate control device 20, for example, via the interface driver 10.

[0027] The passing guidance LED 6 illuminates in a color corresponding to the status of gate 400, for example, to indicate whether gate 400 is passable or not.

[0028] The information display 7 displays, for example, information regarding whether or not passage is permitted.

[0029] Speaker 8, for example, generates a sound indicating whether passage is permitted or not.

[0030] Next, the hardware configuration of the facial recognition server 200 and the matching device 21 will be described with reference to Figure 2. Figure 2 is a diagram showing an example of the hardware configuration of the facial recognition server 200 and the matching device 21.

[0031] The facial recognition server 200 comprises a processor 601, memory 602, and an input / output interface 603 used for transmitting various types of information. The processor 601 is a computing device such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit). The memory 602 is a storage device implemented using RAM (Random Access Memory), ROM (Read Only Memory), etc. The processor 601, memory 602, and input / output interface 603 are connected to a bus 604, and various types of information are exchanged via the bus 604. The processor 601 realizes the functions of the facial recognition server 200 by, for example, reading programs and data stored in ROM onto RAM and executing processing.

[0032] The matching device 21 comprises a processor 701, a memory 702, and an input / output interface 703 used for transmitting various types of information. The processor 701 is a computing device such as a CPU or GPU. The memory 702 is a storage device implemented using RAM, ROM, etc. The processor 701, memory 702, and input / output interface 703 are connected to a bus 704, and various types of information are exchanged via the bus 704. The processor 701 realizes the functions of the matching device 21 by, for example, reading programs, data, etc., stored in ROM onto RAM and executing processing.

[0033] Next, the functions of the facial recognition server 200 and the matching device 21 will be explained with reference to Figure 3, and an example of camera 1 placement will be explained with reference to Figure 4. Figure 3 is a diagram showing an example of the functional configuration of the facial recognition server 200 and the matching device 21. Figure 4 is a diagram showing an example of the installation of multiple cameras 1 at a gate.

[0034] Gate 400 includes, for example, three cameras 1 (camera 1-1, camera 1-2, and camera 1-3).

[0035] Each of the three cameras 1 photographs a person moving towards gate 400 in the direction of arrow X in Figure 4. In Figure 4, arrow X indicates a path of people moving from area A1 (first area) through area A2 (third area) to area A3 (second area), with at least a portion of gate 400 located in area A3.

[0036] Camera 1-1 captures the faces of people located in area A1, which is a certain distance away from gate 400. Area A1 is located upstream of gate 400 in the direction of travel X. Area A1 is, for example, the area from 1.5m away from the support T that supports cameras 1-1 and 1-2 of gate 400 to 3.0m away from the support T. Camera 1-1 captures people located in area A1, which is relatively far from gate 400. Hereafter, camera 1-1 may be referred to as the "long-range camera". The image captured by camera 1-1 is input to processing unit 102.

[0037] Cameras 1-3 are second cameras that capture the faces of people located in region A3, which is closer to gate 400 than region A1. Region A3 is, for example, the area from support T to 50 cm in the direction opposite to the direction of movement X of the person. Hereafter, cameras 1-3 may be referred to as "close-range cameras".

[0038] Camera 1-2 is a third camera that captures the faces of people located in region A2, which is between region A1 and region A3. Region A2 is, for example, the area from 1.5m to 50cm from the support T. Hereafter, camera 1-2 may be referred to as the "mid-range camera".

[0039] The shooting ranges of these cameras 1 are not limited to the examples above. For example, at least a portion of the shooting ranges of each camera 1 may overlap. For instance, the area captured by cameras 1-3, which are close-range cameras, is not limited to the range of area A3, but may include the entire or a portion of area A2 within the range of area A3. Also, although Figure 4 shows an example where the shooting ranges of each camera 1 are adjacent in the direction of travel (arrow X), there may be gaps between the shooting ranges of each camera 1.

[0040] However, since area A3 is the area for authenticating a person attempting to pass through gate 400, it may be defined as the area downstream from the point where it is determined that a person has entered gate 400. For example, if gate 400 determines whether or not a person has entered using the passage management photoelectric sensor 3, then area A3 would be defined as the area downstream from the point where the passage management photoelectric sensor 3 detects the person's entry.

[0041] Furthermore, the shooting range of camera 1 illustrated in Figure 4 is merely a conceptual representation of the range in which each camera 1 can acquire an image clear enough to perform face matching, based on its focal length and field of view, and is not intended to exclude images corresponding to parts outside that range.

[0042] Furthermore, the installation location of camera 1 is not limited to the above examples. For example, the long-range camera (camera 1-1) may be installed at a location away from gate 400 instead of being attached to gate 400, and may photograph area A1. Also, for example, the medium-range camera (camera 1-2) may be installed at a location away from gate 400 instead of being attached to gate 400, and may photograph area A2. Also, for example, the short-range camera (camera 1-3) may be installed at a location away from gate 400 instead of being attached to gate 400, and may photograph area A3.

[0043] Furthermore, instead of the camera 1 attached to the gate 400, the facial recognition server 200 and the matching device 21 may use images captured by cameras for other purposes, such as surveillance cameras.

[0044] Furthermore, the shooting frame rate, number of shots (number of face images recorded), and maximum number of faces detected by these cameras are set according to the type of gate 400, the camera placement, and other factors.

[0045] The matching device 21 includes a communication unit 101 that communicates with the face recognition server 200 via the network 300, a buffer 103 that temporarily records various information, and a processing unit 102. The processing unit 102 performs processing such as face recognition and face matching for people who can pass through the gate 400.

[0046] The facial recognition server 200 includes a communication unit 202 that communicates with the matching device 21 via the network 300, a facial registration database (DB) 203 that manages authentication information, and a processing unit 201. The authentication information includes, for example, information on the facial images of hundreds of thousands to tens of millions of users.

[0047] Next, we will explain the overview and details of the operation of the facial recognition server 200 and the matching device 21.

[0048] Figure 5 is a diagram illustrating the operation overview of the facial recognition system 100. The facial recognition server 200 detects the region of a human face (face image) from the image captured by the long-range camera and compares it with the face images contained in the face registration DB 203, thereby narrowing down the matching candidates from the face images contained in the face registration DB 203. Hereafter, the process of narrowing down the matching candidates using the image captured by the long-range camera may be referred to as "long-range refinement search". In Figure 5, the long-range refinement search corresponds to the first refinement search (primary refinement search).

[0049] For example, the facial recognition server 200 calculates a score indicating the similarity between two facial images and narrows down the matching candidates based on the calculated score. The similarity between two facial images indicates the likelihood that the two facial images belong to the same person.

[0050] For example, the face recognition server 200 calculates a score for each face image in the face registration DB 203 between the face image detected from the image captured by the long-range camera and the face images in the face registration DB 203. The face recognition server 200 then buffers N1 face images (where N1 is an integer greater than or equal to 1) with the highest scores into the matching candidate list ML. For example, Figure 5 shows an example where N1=6. Note that the process of detecting a person's face image from the image captured by the long-range camera may be performed by the matching device 21.

[0051] As a result of the long-range refinement search, the matching candidate list ML includes, for example, face images (candidate face images) narrowed down from the face images in the face registration DB 203. For example, in Figure 5, for each of the six face images taken by the long-range camera, there are six candidate face images. The face authentication server 200 then transmits the matching candidate list ML to the matching device 21. The matching candidate list ML is transmitted to the matching device 21. The matching candidate list ML is an example of matching candidates narrowed down using face images taken by the long-range camera.

[0052] The matching device 21 detects human face images from images captured by the medium-range camera and compares the detected face images with the face images included in the matching candidate list ML, thereby narrowing down the matching candidates from the face images included in the matching candidate list ML. Hereinafter, the process of narrowing down the matching candidates using images captured by the medium-range camera may be referred to as medium-range refinement search. In Figure 5, the medium-range refinement search corresponds to the second refinement search (secondary refinement search).

[0053] For example, the matching device 21 calculates a score for each of the candidate face images in the matching candidate list ML between the face image detected from the image captured by the medium-range camera and the face images included in the matching candidate list ML. Then, the matching device 21 buffers the N2 candidate face images with the highest scores (N2 is an integer greater than or equal to 1) into the matching candidate list SL. For example, Figure 5 shows an example where N2=2. Note that N2 may be less than N1.

[0054] Based on the results of the medium-range refinement search, the matching device 21 retrieves N2 candidate face images from the matching candidate list ML and buffers them in the matching candidate list SL. In the example in Figure 5, the matching candidate list SL contains, for example, three candidate face images for each of the two face images captured by the medium-range camera. If the refinement search is not possible, the matching device 21 may request the face recognition server 200 to perform the refinement search.

[0055] Next, the matching device 21 performs face authentication by comparing the face image corresponding to the face captured by the close-range camera with the matching candidate list SL. If face authentication fails, the matching device 21 requests the face authentication server 200 to perform a refined search.

[0056] In this way, the facial recognition system 100 narrows down the matching candidates before a person enters gate 400. By narrowing down the matching candidates before a person enters gate 400, the facial recognition process performed when a person enters gate 400 is faster because the number of matching candidates is reduced.

[0057] In Figure 5, the matching candidate list ML may include the results of multiple long-range refinement searches. For example, the results of long-range refinement searches performed on each of the images captured by the long-range camera at multiple points in time may be included in the matching candidate list ML.

[0058] Furthermore, in Figure 5, the matching candidate list SL may include the results of multiple intermediate-range refinement searches. For example, the results of long-range refinement searches performed on each of the images captured by the intermediate-range camera at multiple time points may be included in the matching candidate list SL.

[0059] Furthermore, information included in the matching candidate list ML and matching candidate list SL (for example, facial images) may be deleted after a predetermined time has elapsed since the information was added to the list. Other conditions for the deletion of such information from the list may include when the passage management photoelectric sensor 3 detects that the person corresponding to the information has finished passing through the gate. Also, if the gate 400 manages entry and exit to a closed space (such as a building or public transportation), the information may be deleted when it is detected that the person corresponding to the information has left that closed space.

[0060] Figure 6 is a flowchart illustrating an example of the operation of the facial recognition system 100. The matching device 21 detects faces from images captured by the long-range camera (step S1). The matching device 21 sends a request for long-range face refinement search to the facial recognition server 200 (step S2). In the request for long-range face refinement search, for example, the matching device 21 sends the face image detected from the image captured by the long-range camera to the facial recognition server 200. The matching device 21 may send the face image data, or it may extract data related to feature points used for the search from the face image data and send the extracted data. When sending face image data, the facial recognition server 200 can process the face image data, so face matching can be performed using any matching method. On the other hand, when extracting and sending data related to feature points, the amount of data to be sent can be reduced.

[0061] In step S3, the processing unit 201 of the face recognition server 200, which is waiting to receive a search request, receives the request (step S4) and performs a long-distance refinement search (step S5). The processing unit 201 buffers the results into a matching candidate list ML (step S7). The processing unit 201 sends the matching candidate list ML to the matching device 21 (step S8).

[0062] The matching device 21 detects face images from images captured by the medium-range camera (step S9). The matching device 21 performs a medium-range refined search by matching the face images detected from the images captured by the medium-range camera with candidate face images in the matching candidate list ML (step S10). The matching device 21 buffers the candidate face images narrowed down from the matching candidate list ML into the matching candidate list SL (step S11).

[0063] The matching device 21 detects a face image from the image captured by the close-range camera (step S12). The matching device 21 performs face recognition processing by comparing the face image detected from the image captured by the close-range camera with the candidate face images in the matching candidate list SL (step S13).

[0064] Specifically, the matching device 21 compares the face image detected from the image captured by the close-range camera with the candidate face images in the matching candidate list SL. If the matching result shows that the face image detected from the image captured by the close-range camera corresponds to any one of the face images in the matching candidate list SL, the matching device 21 determines that the person captured by the close-range camera is allowed to pass through gate 400.

[0065] If, as a result of the matching process, the face image detected from the image captured by the close-range camera does not correspond to any of the face images in the matching candidate list SL, the matching device 21 determines that the person captured by the close-range camera will not be permitted to pass through gate 400.

[0066] For example, if one of the scores calculated from a face image detected from an image taken with a close-range camera and a face image in the matching candidate list SL is above a threshold, the matching device 21 determines that the face image in the matching candidate list SL corresponding to the score above the threshold corresponds to the face image detected from the image taken with the close-range camera.

[0067] Furthermore, if the score calculated from the face image detected from the image captured by the close-range camera and the face images in the matching candidate list SL is below a threshold, the matching device 21 may determine that the face image detected from the image captured by the close-range camera does not correspond to any of the face images in the matching candidate list SL. Also, if there are multiple face images that show scores above the threshold, the matching device 21 may determine that the face image detected from the image captured by the close-range camera does not correspond to any of the face images in the matching candidate list SL. In this case, if it is not possible to narrow it down to one person, it is determined that face matching has not been successful, thus providing a strict determination result.

[0068] Conversely, if there are multiple face images that show scores above the threshold, it is acceptable to determine that the face image detected from the image taken with the close-range camera corresponds to one of the face images in the matching candidate list SL. In this case, even if face matching occurs for a person who is difficult to narrow down to one, such as twins, it is possible to prevent the flow of people from being stopped. Even in this case, however, a score above the threshold has been obtained, so a certain level of reliability can be guaranteed, and face matching will not succeed for people who are clearly not matching candidates (for example, people whose face images are not registered in the face registration DB203).

[0069] If facial recognition is successful in S13 and the person photographed by the close-range camera is able to pass through gate 400 (step S14, Yes), the matching device 21 generates result information R indicating that passage is permitted. The gate control device 20 then issues a gate opening instruction based on this result information R (step S15), and in step S16, the gate remains open until the person has finished passing through gate 400. After the person has passed through gate 400, a gate closing instruction is issued (step S17), and the process from step S12 onward is repeated.

[0070] If a person photographed by the close-range camera is unable to pass through gate 400 (step S14, No), the verification device 21 generates result information R indicating that passage is not possible. Then, the gate control device 20 performs the process in step S17 based on this result information R.

[0071] Figure 6 shows an example in which the matching device 21 can create a matching candidate list SL by medium-range refinement search. However, there are cases in which the matching device 21 cannot create the matching candidate list SL if the results of the medium-range refinement search process are inappropriate. Cases where the results of the medium-range refinement search process are inappropriate include, for example, when the score between the face image detected from the image captured by the medium-range camera and the candidate face image in the matching candidate list ML is below a threshold. For example, a case where the results of the medium-range refinement search process are inappropriate occurs when a person who is not included in the image from the long-range camera is included in the image from the medium-range camera.

[0072] Figure 7 is a flowchart illustrating an example of operation when the results of the medium-range refinement search process are not appropriate. In the following, the same processes as those shown in Figure 6 will be omitted from the explanation, and different processes will be described.

[0073] If the result of the intermediate-range refinement search process is not appropriate (step S100, No), the matching device 21 sends an intermediate-range refinement search request to the face recognition server 200 (step S101). Here, the matching device 21 may send the face image detected from the image captured by the intermediate-range camera to the face recognition server 200.

[0074] When the face recognition server 200 receives a request for a medium-range refined search (step S102), it performs a medium-range refined search process (step S103) and transmits the result of the medium-range refined search process to the matching device 21 (step S104). In the medium-range refined search process in step S103, the face recognition server 200 compares the face image detected from the image captured by the medium-range camera with candidate face images included in the face registration DB 203.

[0075] When the matching device 21 receives the result of the medium-range narrowing search process (step S105), it buffers this result in the matching candidate list SL (step S106).

[0076] If the medium-range refinement search process is successful (step S100, Yes), the process in step S106 is performed.

[0077] As shown in the flowchart in Figure 7, even if the matching device 21 cannot create a matching candidate list SL because the result of the medium-range narrowing search process is not appropriate, it can still perform face recognition processing (see Figure 6) by requesting the face recognition server 200 to perform a re-search.

[0078] Furthermore, if the matching device 21 fails the face recognition process and, in the pass-through determination in S14 of Figure 6, determines that the person is not eligible to pass, it may request the face recognition server 200 to perform a re-search. For example, if the score between the face image detected from the image captured by the close-range camera and the face image in the matching candidate list SL is below a threshold, the matching device 21 may request the face recognition server 200 to perform a re-search. For example, if a person who is not included in the image from the medium-range camera is included in the image from the close-range camera, the matching device 21 may request the face recognition server 200 to perform a re-search. However, if the matching device 21 requests the face recognition server 200 to perform a re-search, the matching process in the matching device 21 will be delayed by the time required for communication to obtain the matching candidate list again from the face recognition server 200. Nevertheless, since the possibility of a re-search request occurring is low, even if such processing is performed, the processing speed of the matching device 21 will improve compared to the case where no filtering is performed at all.

[0079] Figure 8 is a flowchart illustrating an example of what happens when the facial recognition process fails. In the following sections, we will omit explanations of processes that are identical to those shown in Figure 6, and instead explain different processes.

[0080] If the person photographed by the close-range camera is not eligible to pass through gate 400 (step S14, No), the matching device 21 sends a close-range search request to the face recognition server 200 (step S201). The matching device 21 may send the face image detected from the image taken by the close-range camera to the face recognition server 200.

[0081] The face recognition server 200 receives a proximity search request (step S202) and performs proximity search processing (step S203). For example, the face recognition server 200 identifies a face image detected from the image captured by the proximity camera and, for example, one person in the face registration DB 203, and transmits this processing result to the matching device 21 (step S204). In the proximity face refinement search processing in step S203, the face recognition server 200 compares the face image detected from the image captured by the proximity camera with the face images included in the face registration DB 203.

[0082] When the matching device 21 receives the result of the close-range face narrowing search process (step S205), it performs face recognition processing in the same manner as in step S13 by comparing this result with the face image detected from the image captured by the close-range camera, and determines whether or not to allow passage in the same manner as in step S14 (step S206).

[0083] As a result, if the person photographed by the close-range camera is able to pass through gate 400 (step S206, Yes), the verification device 21 generates result information R indicating that passage is permitted. This then allows the processing from step S15 onward to proceed.

[0084] If the verification results indicate that the person photographed by the close-range camera cannot pass through gate 400 (step S206, No), the verification device 21 generates result information R indicating that passage is not permitted. As a result, the gate door does not open (step S17).

[0085] In Figure 8, an example is shown in which the matching device 21 sends a close-range search request to the face recognition server 200 when a person captured by the close-range camera is not eligible to pass through gate 400 (step S14, No). However, this disclosure is not limited to this example. For instance, instead of sending a close-range search request to the face recognition server 200, the matching device 21 may compare the face image detected from the image captured by the close-range camera with the candidate face images in the matching candidate list ML. Since the matching candidate list ML is buffered by the matching device 21, matching within the range of candidate face images included in the matching candidate list ML can be performed without communication with the face recognition server 200. Therefore, if matching within the range of candidate face images included in the matching candidate list ML is successful, the matching process can be sped up by omitting communication. However, since the matching candidate list ML is a relatively large list, depending on the communication speed of the matching device 21 and the network 300, it may be faster to send a close-range search request to the face recognition server 200. Therefore, if the size of the matching candidate list ML is variable, you may switch between sending a nearby search request or performing a match against the candidate face images in the matching candidate list ML, depending on its size.

[0086] Furthermore, in Figure 8, the information that the face recognition server 200 returns in response to a short-range search request is the result of the short-range face refinement search process, and the matching device 21 performs a matching process with that result (steps S205, S206). However, if the face recognition server 200 has sufficient processing capacity, the face recognition server 200 may determine whether the face matching was successful or not and transmit that result directly. In this case, the matching device 21 determines whether passage is permitted or not according to the received result of the face matching and performs the gate opening and closing process.

[0087] Figure 9 illustrates how a person is photographed by a long-range camera. The long-range camera (camera 1-1) can capture the faces of people located in area A1 at a certain distance from gate 400. Therefore, by using the facial images of multiple people who may enter gate 400, it is possible to roughly narrow down the number of people who may pass through gate 400 from a large registered population of matching information. In this way, by using facial images captured by the long-range camera to perform preliminary filtering, the facial recognition process performed when a person enters gate 400 can be sped up.

[0088] Figure 10 shows how people are photographed by the mid-range cameras. The mid-range cameras (cameras 1-2) can photograph the faces of people in area A2, which is closer to gate 400 than area A1. For example, just before a person enters gate 400, it is possible to narrow down the list of matching candidates ML to one or more people who are more likely to pass through gate 400. This narrowing down speeds up the facial recognition process performed when a person enters gate 400. Furthermore, even if there are people who cannot be captured by the long-range cameras, matching using the mid-range cameras is possible by executing the process shown in Figure 7. People who cannot be captured by the long-range cameras include, for example, people who cut in line to enter gate 400, or people who approach the back of the passenger in front of them to enter gate 400.

[0089] Figure 11 illustrates how a person is captured by a close-range camera. The close-range cameras (cameras 1-3) can clearly capture facial images of people passing through gate 400, enabling accurate facial recognition processing. Furthermore, because the target of matching is pre-defined, matching can be performed quickly, potentially even matching people with relatively fast walking speeds. Additionally, the burden of facial recognition processing is reduced compared to processing a large population, allowing the use of less powerful and inexpensive CPUs. The reduced burden of facial recognition processing also allows for higher resolution of the close-range cameras, further improving authentication accuracy.

[0090] Figure 12 illustrates the relationship between the walking speed of a person passing through gate G and the facial recognition process. In the case of a gate where passage is determined by facial recognition, the length of the gate is determined by the relationship between the facial recognition processing time and the time (performance) from the moment the door opening command is issued by the door opening mechanism Dr to the moment the door is opened. This relationship will be explained below.

[0091] In Figure 12, position SP indicates the position where facial recognition processing for a person passing through gate G begins. To prevent the gate from being opened by a face of someone who has not yet entered gate G, it is desirable that this position be after a person has entered gate G. For example, in the case of gate 400 described above, this corresponds to the position where entry into gate 400 is detected by the passage management photoelectric sensor 3. Position LP indicates the limit position for issuing a command to open or close gate G. In the case of gate G equipped with a physical opening / closing door mechanism Dr, a certain amount of time is required for this opening / closing operation to be completed. Significant reduction of this time (door opening processing time) is difficult from a technical performance and safety standpoint. Therefore, a length L2 reflecting a standard human walking speed becomes the shortest length of gate G, assuming zero time for determining whether to open or close. On the other hand, in a configuration like this embodiment, where the decision to open or close the opening / closing door mechanism 4 is made based on the results of facial recognition processing, facial recognition processing must be completed by the time the person reaches position LP at the latest. Therefore, the length from the opening / closing door mechanism Dr of gate G to the position SP where facial recognition processing for a person passing through gate G begins is the sum of length L1, which reflects the time required for facial recognition processing (facial recognition processing time), and length L2.

[0092] As a specific example, if the walking speed is 3.6 km / h, the facial recognition processing time is 200 msec, and the door opening processing time is 600 msec, the distance from the gate end (gate entrance) to the opening / closing door mechanism Dr may be set to 800 mm or more. As mentioned above, it is difficult to shorten the length L2, so in order to shorten the gate length, it is necessary to shorten the length L1, that is, to shorten the time required for facial recognition processing.

[0093] Furthermore, delays in facial recognition may occur when the number of registered users reaches 1 million to 10 million. For example, with a False Acceptance Rate (FAR) of 0.001%, there is a possibility that one in 100,000 people will be identified as a different person. Even with an FAR of 0.0001%, there is a possibility that one in 1 million people will be identified as a different person. Lowering the FAR requires more complex facial recognition processing, including evaluation from various perspectives, which tends to increase the processing time for facial recognition.

[0094] Furthermore, in the case of gate 400, which has a narrow passage that prevents more than two people from passing through at the same time, like a ticket gate, if there is a queue behind, it becomes difficult for a person who is denied passage to move backward. Therefore, it is ideal for authentication to be completed the moment a person enters gate 400.

[0095] Thus, whether the goal is to reduce gate size, lower FAR (Forward Access Rate), or ensure convenience in narrow passages, faster facial recognition is required.

[0096] According to the matching device 21 described herein, a long-range camera is used to capture a facial image of a person approaching gate 400, and before the person enters gate 400, that facial image is used to perform a narrowed-down search from a large pool of matching information. This speeds up facial recognition processing using a close-range camera.

[0097] Figure 13 is a diagram illustrating a modified gate. The arch-shaped gate 400 shown in Figure 13 has a reader 500 that reads codes such as QR codes containing information to identify people passing through the gate 400, and a detection camera 501 that detects tailgating and the like. The arch-shaped gate 400 shown in Figure 13 also has two cameras each for long-range, medium-range, and short-range photography. Cameras 1-1A and 1-1B are two long-range cameras, positioned on the left and right sides of the pillars of the gate 400 housing. Cameras 1-2A and 1-2B are two medium-range cameras, positioned on the left and right sides of the pillars of the gate 400 housing. Cameras 1-3A and 1-3B are two short-range cameras, positioned on the left and right sides of the pillars of the gate 400 housing. By positioning one camera on each side to capture images at each distance, even if a person attempting to pass through the gate is facing either left or right, one of the cameras can capture a frontal image suitable for matching.

[0098] In Figure 13, long-range, medium-range, and short-range cameras are all positioned one on each side, but it is not always necessary to have one camera on each side for every distance. For example, in environments where some bias in the direction people face is expected, the camera facing the opposite direction will only be used for auxiliary purposes, so it is not necessary to have long-range, medium-range, and short-range cameras. Other examples of omitting a camera for one of the distances on either the left or right side include situations where it is difficult to mount a camera on either side due to the gate's shape, internal structure, or design constraints. Furthermore, the greater the distance, the less the difference in face orientation affects the shooting results, so if one camera is to be omitted, it is better to omit the camera for the longer distance to minimize the impact.

[0099] By arranging multiple cameras for long-range, medium-range, and short-range photography in this manner, it becomes easier to obtain a face image that captures the front of a person's face, even if their face is facing a direction different from their direction of travel as they pass through the gate 400. Note that the shape of the gate 400, the camera positions, and the number of cameras are not limited to the embodiment described above.

[0100] In the above embodiment, an example of performing two refinement search processes was described, but it is also possible to configure the system to perform only one refinement search process (long-range refinement search or medium-range refinement search).

[0101] According to this configuration example, since the refinement search process is performed only once, the processing time for refinement can be shortened, and the number of cameras used to capture images for the refinement search process can be reduced. Therefore, the system configuration is simplified, and facial recognition processing can be accelerated while reducing the cost associated with system construction.

[0102] Furthermore, with this configuration example, since the facial recognition server 200 performs the refined search process, it can handle a vast amount of matching information that cannot be stored in the matching device 21. In addition, since the facial recognition server 200 can be shared by gates 400 set up at multiple locations, it is possible to speed up facial recognition processing and make efficient use of resources.

[0103] In this example configuration, the refinement search is performed once in the face recognition server 200. However, it is also possible to configure the system to perform a second refinement search in the matching device 21 after the refinement search in the face recognition server 200. In this case, the processing unit 102 of the matching device 21 uses a third face image captured by a medium-range camera that captures the faces of people in area A2 between area A1 and area A3 to further narrow down the face images of people who can pass through gate 400 from the face images acquired from the face recognition server 200. The processing unit 102 of the matching device 21 then matches the second face image with the narrowed-down face image. For example, if the size of the matching candidate list ML is large, it may take time to transmit it from the face recognition server 200 to the matching device 21, or the performance of the matching device 21 may be insufficient to perform the refinement search using the matching candidate list ML. In such cases, the above configuration makes it possible to further speed up the face recognition process. Furthermore, if the size of the matching candidate list ML is sufficiently small, processing may be faster if the face recognition server 200 sends the matching candidate list ML to the matching device 21 at an early stage, as shown in the configuration example above, and the matching device 21 performs a second refinement. Based on this, depending on the size of the matching candidate list ML, it may be possible to switch whether the second refinement search process is performed by the face recognition server 200 or the matching device 21.

[0104] The following describes variations of the refined search process and facial recognition process in the facial recognition server 200 and the matching device 21.

[0105] In the long-distance refinement search (first-order refinement search) shown in Figure 5 above, an example was described in which the face recognition server 200 buffers a predetermined number (e.g., N1) of face images with the highest scores calculated for each face image included in the face registration DB 203 into the matching candidate list ML. However, this disclosure is not limited to this. For example, the face recognition server 200 may buffer into the matching candidate list ML face images corresponding to scores exceeding a first threshold among the scores calculated for each face image included in the face registration DB 203.

[0106] Furthermore, in the long-distance refinement search (primary refinement search) shown in Figure 5 above, the matching device 21 buffers a predetermined number (e.g., N2) of face images into the matching candidate list SL, starting with the one with the highest score calculated for each of the face images included in the matching candidate list ML.

[0107] This configuration example allows you to define an upper limit on the number of face images to be buffered in the matching candidate list SL, thus preventing buffer overflow even if a large number of high-scoring face images are found.

[0108] However, this disclosure is not limited thereto. For example, the matching device 21 may buffer in the matching candidate list SL any face images in which the score calculated for each face image included in the matching candidate list ML exceeds a second threshold that is higher than the first threshold.

[0109] According to this configuration example, since filtering is performed based on a threshold, face images corresponding to scores above the threshold are not excluded from the matching candidates, and face images corresponding to scores below the threshold are excluded from the matching candidates. This enables faster face recognition processing and improved accuracy.

[0110] Note that this example configuration includes a secondary refinement search, but it is also possible to omit the secondary refinement search. An example of this case will be explained below.

[0111] If the results of the primary refinement search indicate that a secondary refinement search is not necessary, the processing unit 102 may perform the face recognition process without performing the secondary refinement. Cases where a secondary refinement search is not necessary may, for example, correspond to a number of face images corresponding to scores exceeding the first threshold in the results of the primary refinement search that does not require a secondary refinement search (e.g., 1). Alternatively, cases where a secondary refinement search is not necessary may correspond to a case where the difference between the highest score and the Nth (where N is an integer greater than or equal to 2) highest score in the results of the primary refinement search is greater than or equal to a predetermined difference.

[0112] In this example, since the secondary refinement search can be omitted, the processing time for refinement can be reduced, and facial recognition processing can be sped up.

[0113] Furthermore, if the aforementioned primary and secondary refinement searches yield sufficiently high matching results, the facial recognition process may be omitted. An example of this case will be explained below.

[0114] A sufficiently high matching result can be obtained through the primary and secondary refinement searches, for example, when the difference between the highest score and the second highest score in the secondary refinement search is greater than or equal to a predetermined difference. In this case, the processing unit 102 may not perform the facial recognition process and may generate the result of the secondary refinement search as result information R indicating that authentication has been performed.

[0115] This example shows that facial recognition processing can sometimes be omitted, thus reducing processing time.

[0116] If the scores for any of the face candidates included in the secondary filtering results are too low, the face images captured by the mid-range camera may be sent to the face recognition server 200 to retrieve new candidates for buffering. An example configuration in this case is described below.

[0117] If, as a result of the secondary filtering, a face image corresponding to a score exceeding the second threshold can be matched with the third face image, the processing unit 102 of the matching device 21 performs face authentication processing without sending multiple third face images to the face authentication server 200.

[0118] If, as a result of the secondary filtering, a face image corresponding to a score exceeding the second threshold cannot be matched with the third face image, the processing unit 102 of the matching device 21 sends multiple third face images to the face recognition server 200 and instructs the face recognition server 200 to filter the face images from among the multiple third face images. The processing unit 102 of the matching device 21 retrieves the filtered face images from the face recognition server 200 and further filters the retrieved face images to identify faces of people who can pass through the gate 400.

[0119] In this configuration example, facial images captured by the mid-range camera can be sent to the facial recognition server 200, and buffered candidates can be retrieved again. Therefore, even if the scores of any of the facial candidates included in the secondary filtering results are too low, the filtering accuracy can be maintained at a certain level or higher, and facial recognition processing can be sped up while suppressing a decrease in the accuracy of the facial recognition process.

[0120] The processing unit 102 may be configured to acquire feature quantities indicating facial characteristics contained in face images captured by a long-range camera, a medium-range camera, or a short-range camera, and to use these feature quantities to narrow down the face images of people who can pass through gate 400 from among multiple matching information. Alternatively, the processing unit 102 may be configured to acquire feature quantities indicating facial characteristics contained in face images narrowed down by the face recognition server 200, and to use these feature quantities to narrow down the face images of people who can pass through gate 400 from among multiple matching information. Here, examples of feature quantities include the distribution of face color, shape, and brightness. They may also be feature quantities generated by more complex processing used in the field of machine learning. By using feature quantities, the size of information exchanged between the face recognition server 200 and the matching device 21 can be reduced. In addition, depending on the feature quantities used, the influence of parameters that tend to change in the real environment can be suppressed, thus enabling robust face recognition.

[0121] In the above embodiment, an example was described in which authentication information is recorded in the facial recognition server 200, but this disclosure is not limited thereto. For example, the authentication information may be recorded in the matching device 21 or the gate control device 20. For example, if the matching device 21 has a large recording capacity that can record a large amount of facial image information (authentication information) and processing capacity that can perform a primary refinement search process, the processing unit 102 of the matching device 21 may use the first facial image to narrow down the facial image from among multiple matching information.

[0122] Furthermore, in the facial recognition system 100, the number of cameras 1 may be four or more. Also, the number of filtering steps may be, for example, three or more. Specifically, the configuration involves capturing each of the four or more regions with four or more cameras. Increasing the number of filtering steps allows the system to handle a larger number of facial images (for example, when the number of stored faces in the face registration DB 203 is large). However, increasing the number of filtering steps may increase the time required for re-searching in the event of a failed determination, so the threshold score for determination may be lowered as the number of steps increases.

[0123] Furthermore, the facial recognition server 200 may perform a second refinement search. In particular, if the size of the matching candidate list ML is large, depending on the performance of the network 300 and the matching device 21, it may take longer to send the matching candidate list ML to the matching device 21 and have it perform the second refinement search locally. In addition, the facial recognition server 200 may be able to switch whether or not to perform the second refinement search based on the size of the matching candidate list ML, the communication speed of the network 300, the buffer size of the matching device 21, or the processing capacity of the matching device 21. When adding candidates with scores above a threshold to the matching candidate list ML, the size of the matching candidate list ML is variable, so such a switch is beneficial.

[0124] Furthermore, in cases where more than three refined search processes are performed, the decision of how many refined search processes should be performed by the facial recognition server 200 can be made from a similar perspective.

[0125] Furthermore, instead of providing short-range cameras, medium-range cameras, and long-range cameras at each of the multiple gates 400, for example, a medium-range camera and a long-range camera may be shared among the multiple gates 400.

[0126] Furthermore, in the above embodiment, the types of information used for each filtering process and the information used to obtain the authentication result may be different. For example, features of the facial contour could be used for filtering, and features of facial parts could be used to obtain the authentication result. Since the size of the facial images captured by the long-range camera and the close-range camera are different, the accuracy of the judgment can be further improved by using information appropriate for each distance. Also, simply in the sense of comprehensively evaluating from multiple perspectives, it can be expected that accuracy will be improved by using different information for filtering and obtaining the authentication result.

[0127] On the other hand, the same type of information may be used for each filtering step and for obtaining the authentication result. In this way, the same evaluation criteria are used for the previous filtering and the current filtering or facial recognition process, which helps to suppress discrepancies in the judgment results. As a result, the frequency of requests to the facial recognition server 200 due to facial recognition failures can be reduced, and thus, faster facial recognition processing can be expected.

[0128] Furthermore, in the above embodiment, the face images included in the matching candidate list obtained as a result of the narrowing down process may not be the images themselves, but rather their feature quantities (this information is also referred to as "candidate face images"). In particular, for the matching candidate list transmitted from the face recognition server 200 to the matching device 21, a list consisting of feature quantities can reduce the amount of communication. However, since it is generally difficult to perform matching with other feature quantities using information extracted from feature quantities, if matching with different feature quantities is performed at each stage, it is preferable to use the face images themselves as the matching candidate list, even if the size of the matching candidate list becomes large. Also, in situations where the size of the matching candidate list does not significantly affect the amount of communication, such as when the face recognition server 200 performs narrowing down processes multiple times, the face images themselves may be used as the matching candidate list, and feature quantities may be extracted when creating the matching candidate list to be transmitted to the matching device 21.

[0129] Furthermore, in the above embodiment, the gate 400 was equipped with an opening and closing door mechanism 4, but the means (restriction unit) for restricting a person's movement when face recognition fails is not limited to this. For example, a psychological restriction mechanism such as a siren and / or alarm may be used. Alternatively, a mechanism may be used to indirectly restrict movement by notifying a nearby security guard and / or robot without notifying the person attempting to pass through the gate. The time from face recognition failure to restriction will differ depending on the type of restriction unit used, but regardless of the means used, it is equally useful to speed up face recognition to obtain the face recognition result before the person reaches the restriction unit.

[0130] In other words, the means (restriction mechanism) for restricting a person's movement when facial recognition fails is not limited to examples that physically restrict (block) a person's movement, such as the opening and closing door mechanism 4 provided in the middle of a person's movement path at gate 400. For example, a specific point (or a specific range) may be set at gate 400, and gate 400 may restrict the movement of people from upstream of the specific point to downstream of the specific point in the direction of a person's movement. In this case, the means of restriction may be a siren and / or alarm, as described above, or it may be a notification to a security guard and / or robot. In this case, the shooting range of each camera may be located upstream of the specific point. For example, the shooting range of a short-range camera (e.g., area A3 in Figure 4), the shooting range of a medium-range camera (e.g., area A2 in Figure 4), and the shooting range of a long-range camera (e.g., area A1 in Figure 4) may be set in order from the one closest to the specific location.

[0131] Furthermore, although the matching device 21 was described in the above embodiment as a device used in a gate that restricts the movement of people, it is not limited to this. It can be applied to any system that performs facial recognition of people approaching from a distance. In that case, the definition of the area A3 (see Figure 4) where facial recognition processing is performed in the above embodiment will differ depending on the requirements of the system. For example, when applied to a surveillance system that uses a surveillance camera to record people passing through a specific monitoring point, area A3 may be defined as the area around the monitoring point (for example, the area upstream from the monitoring point).

[0132] As described above, the matching device 21 compares the matching candidates narrowed down using the first face image captured by the first camera that photographs area A1 with the second face image captured by the second camera that photographs area A3, to which a person may move from area A1.

[0133] This configuration allows for capturing facial images of people approaching gate 400, and then, before the person enters gate 400, using those facial images to perform a refined search from a large pool of matching information. This enables faster facial recognition processing using medium-range or short-range cameras.

[0134] This disclosure can be implemented using software, hardware, or software integrated with hardware.

[0135] Each functional block used in the description of the above embodiments may be implemented partially or entirely as an integrated circuit (LSI), and each process described in the above embodiments may be controlled partially or entirely by a single LSI or a combination of LSIs. An LSI may consist of individual chips, or it may consist of a single chip that includes some or all of the functional blocks. An LSI may have data inputs and outputs. Depending on the degree of integration, LSIs may be referred to as ICs, system LSIs, super LSIs, or ultra LSIs.

[0136] The method of integration is not limited to LSIs; it may also be implemented using dedicated circuits, general-purpose processors, or dedicated processors. Furthermore, FPGAs (Field Programmable Gate Arrays) that can be programmed after LSI manufacturing, or reconfigurable processors that allow for the reconfiguration of the connections and settings of circuit cells within the LSI, may also be used. This disclosure may be implemented as digital or analog processing.

[0137] Furthermore, if advancements in semiconductor technology or other derived technologies lead to the emergence of integrated circuit technologies that replace LSIs, then naturally, it would be possible to use those technologies to integrate functional blocks. The application of biotechnology, for example, is a possibility.

[0138] This disclosure is applicable to all types of devices, systems, and equipment having communication capabilities (collectively referred to as communication equipment). Communication equipment may include a radio transceiver and a processing / control circuit. A radio transceiver may include a receiver and a transmitter, or both as functions. A radio transceiver (transmitter, receiver) may include an RF (Radio Frequency) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or similar. Non-exclusive examples of communication devices include telephones (mobile phones, smartphones, etc.), tablets, personal computers (PCs) (laptops, desktops, notebooks, etc.), cameras (digital still / video cameras, etc.), digital players (digital audio / video players, etc.), wearable devices (wearable cameras, smartwatches, tracking devices, etc.), game consoles, digital book readers, telehealth / telemedicine devices, vehicles or mobile transport with communication capabilities (cars, airplanes, ships, etc.), and combinations of the above-mentioned devices.

[0139] Communication devices are not limited to portable or movable devices, but also include all kinds of non-portable or fixed devices, devices, and systems, such as smart home devices (appliances, lighting equipment, smart meters or measuring instruments, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0140] Furthermore, in recent years, Cyber-Physical Systems (CPS), a new concept in IoT (Internet of Things) technology that creates new added value through information linkage between the physical and cyber spaces, has been attracting attention. This CPS concept can also be adopted in the above-described embodiment.

[0141] In other words, as a basic configuration of CPS, for example, edge servers located in physical space and cloud servers located in cyberspace are connected via a network, and processing can be distributed and performed by the processors installed on both servers. Here, it is preferable that each processing data generated on the edge server or cloud server is generated on a standardized platform, and by using such a standardized platform, it is possible to improve efficiency when building systems that include various diverse groups of sensors and IoT application software.

[0142] Communication includes data communication via cellular systems, wireless LAN systems, and communication satellite systems, as well as data communication using combinations of these.

[0143] Furthermore, the communication device also includes devices such as controllers and sensors that are connected to or linked to a communication device that performs the communication functions described in this disclosure. For example, this includes controllers and sensors that generate control signals and data signals used by the communication device that performs the communication functions of the communication device.

[0144] Furthermore, communication equipment includes infrastructure facilities such as base stations, access points, and any other devices, devices, and systems that communicate with or control the aforementioned non-limited types of equipment.

[0145] Although various embodiments have been described above with reference to the drawings, it goes without saying that this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of this disclosure. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the disclosure.

[0146] The specific examples of this disclosure have been described in detail above, but these are merely illustrative and do not limit the scope of the claims. The technologies described in the claims include various modifications and changes to the specific examples described above. [Industrial applicability]

[0147] One embodiment of the present disclosure is suitable for a device or system that performs facial image matching (or authentication). [Explanation of symbols]

[0148] 1, 1-1, 1-2, 1-3 Camera 2 QR code reader 3. Passage control photoelectric sensor 4. Door opening and closing mechanism 5. Entrance guidance indicator 6. Passing guidance LED 7. Information display 8 speakers 9 Interface boards 10 Interface Drivers 20 Gate control device 21 Verification device 30 Network Hubs 100 facial recognition systems 101, 202 Communications Department 102, 201 Processing Unit 103 buffer 200 facial recognition servers 203 Face Registration Database 300 Networks 400 gates 500 Leaders 501 Detection Camera 601, 701 processors 602, 702 memory 603, 703 Input / Output Interfaces Buses 604 and 704

Claims

1. A matching device for regulating the flow of people, In a path where there is a flow of people from a first area to a second area located upstream of a point where the flow of people should be restricted, a processing unit performs a second face image matching using a first face image matching result using a face image included in a first image taken of the first area and a plurality of face images included in a face image database, and a first candidate face image narrowed down from the plurality of face images included in the face image database, and a face image included in a second image taken of the second area. A communication unit that outputs the results of the second facial image matching, A matching device equipped with a matching device.

2. The aforementioned regulation is at least one of a siren, alarm, guard, or robot. The matching device according to claim 1.

3. The processing unit uses the face images included in the third image, which is taken of the third region between the first region and the second region, to narrow down the first candidate face images, and compares the narrowed-down second candidate face images with the face images included in the second image. The matching device according to claim 1.

4. The communication unit acquires the first candidate face image from a server located outside the matching device. The matching device acquires the second candidate face image by narrowing down the first candidate face image within the matching device. The verification device according to claim 3.

5. The processing unit determines a score indicating the similarity between the two face images between each of the N1 (where N1 is an integer of 2 or more) first candidate face images and the third image. The processing unit determines a second candidate face image by narrowing down the first candidate face image to the top N2 candidates (where N2 is an integer between 1 and N1) of the N1 scores. The verification device according to claim 3.

6. The processing unit determines a score indicating the similarity between the two face images between each of the first candidate face images and the third image. The processing unit determines a second candidate face image by narrowing down the first candidate face image to candidates whose score is equal to or greater than the second threshold. The first candidate face image is one in which the score with respect to the face image included in the first image is equal to or greater than the first threshold. The first threshold is lower than the second threshold. The verification device according to claim 3.

7. The processing unit does not determine the second candidate face image if the number of first candidate face images is less than the third threshold. The matching device according to claim 6.

8. The processing unit does not perform a comparison with the face images included in the second image if the number of second candidate face images is less than the fourth threshold. The matching device according to claim 6.

9. If the scores between the first candidate face image and the third image are less than the fifth threshold, the processing unit requests a server located outside the matching device to determine the second candidate face image. The matching device according to claim 6.

10. If the processing unit fails to match the face image contained in the second image, it requests a server located outside the matching device to perform a match between the face image contained in the second image and multiple face images contained in the face image database. The matching device according to claim 1.

11. It is a matching system for regulating the flow of people. A first camera photographs the first region in the flow of people moving from the first region to the second region, which is located upstream of the point where the flow of people should be restricted. A second camera that photographs the second region, A first matching device that performs a first face image matching using a face image contained in a first image captured by the first camera and multiple face images contained in a face image database, A second matching device performs a second face image matching using a first candidate face image narrowed down from a plurality of face images included in the face image database based on the results of the first face image matching, and a face image included in a second image captured by the second camera. A matching system equipped with this system.

12. A matching method for regulating the flow of people, In a path where there is a flow of people from the first region to the second region located upstream of the point where the flow of people should be restricted, a first face image matching is performed using the face images contained in the first image taken of the first region and multiple face images contained in the face image database. Based on the result of the first face image matching, a first candidate face image narrowed down from the multiple face images contained in the face image database and a face image contained in the second image taken of the second region are used to perform a second face image matching. Output the results of the second facial image matching described above. Verification method.

Images