Information processing device and information processing method

Abstract

To contribute to provision of an information processing device and an information processing method for realizing passage management of a target (e.g., a person) through a gate with a simple configuration.SOLUTION: An information processing device includes: an acquisition unit that acquires an image obtained by imaging a person in a direction from an exit toward an entrance of a gate; a first face authentication processing unit that executes, for a person determined to have entered a first area, first face authentication processing based on a first face image imaged in the first area and outputs comparison processing result information including information concerning a compared face; a second face authentication processing unit that when the first face authentication processing succeeds, executes, for a person determined to have entered a second area between the first area and the exit, second face authentication processing based on a second face image imaged in the second area; and an output unit that when the person who succeeded in the first face authentication processing and the person included in the second face image are determined to be the same person, outputs passage of the person through the gate.SELECTED DRAWING: Figure 2A

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] Techniques for managing the entry and exit of persons passing through gates installed at stations, airports, etc. are known. Patent Document 1 describes an apparatus that, when a person obtains permission to pass through a gate using a wireless card and enters the gate from the entrance of the gate, tracks whether the person has passed through the gate (whether the person has not returned to the entrance of the gate) based on changes in the position of the wireless card.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a gate for managing entry and exit, it is desired to manage whether any person has passed through the gate. Hereinafter, this management may also be abbreviated as "passage management" or "tracking management". There is room for examination regarding a method for realizing passage management with a simple configuration.

[0005] Non-limiting embodiments of the present disclosure contribute to providing an information processing apparatus and an information processing method for realizing passage management of a target (e.g., a person) with respect to a gate with a simple configuration.

Means for Solving the Problems

[0006] An information processing apparatus according to an embodiment of the present disclosure includes an acquisition unit that acquires an image including a face image of a person entering a gate, taken from the direction from the exit to the entrance of the gate; a first face authentication processing unit that performs a first face authentication process based on a first face image taken in the first area on a person determined to have entered a first area defined for the gate, and outputs collation processing result information including information on the collated face to a second face authentication processing unit; a second face authentication processing unit that, when the first face authentication process is successful, performs a second face authentication process based on a second face image taken in the second area on a person determined to have entered a second area defined between the first area and the exit of the gate; and an output unit that outputs that the person has passed through the gate when it is determined by the second face authentication process that the person who succeeded in the first face authentication process and the person included in the second face image are the same person.

[0007] An information processing method according to an embodiment of the present disclosure is such that an acquisition unit of an information processing apparatus acquires an image including a face image of a person entering a gate, taken from the direction from the exit to the entrance of the gate; a first face authentication processing unit of the information processing apparatus performs a first face authentication process based on the first face image of the person taken in the first area when it is determined that the person has entered a first area defined for the gate, and outputs collation processing result information including information on the collated face to a second face authentication processing unit; a second face authentication processing unit of the information processing apparatus performs a second face authentication process based on a second face image taken in the second area when it is determined that the person has entered a second area defined between the first area and the exit of the gate when the first face authentication process is successful; and an output unit of the information processing apparatus outputs that the person has passed through the gate when it is determined by the second face authentication process that the person who succeeded in the first face authentication process and the person included in the second face image are the same person.

[0008] Note that these general or specific aspects may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium, or may be implemented in any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0009] Non-limiting examples of the present disclosure can implement passage management for a target (e.g., a person) with respect to a gate in a simple configuration.

[0010] Further advantages and effects in an embodiment of the present disclosure will be clarified from the specification and drawings. Such advantages and / or effects are respectively provided by several embodiments and features described in the specification and drawings, but not all necessarily need to be provided to obtain one or more identical features.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same function are denoted by the same reference numerals, and redundant description is omitted.

[0013] (One embodiment) <Findings leading to the present disclosure> At gates installed in facilities such as stations and airports for managing entry and exit from the facilities, the use of passage management for accurately managing which person has passed through the gate is being considered. If the passage management is insufficient, for example, a person who entered from the entrance of the gate but did not head towards the exit of the gate and returned to the entrance of the gate may be erroneously determined to have passed through, or a person who actually passed through may be erroneously recognized as not having passed through. Such errors can lead to incorrect billing of fees, for example, in services that bill a person who has passed through a gate such as a ticket barrier at a station.

[0014] To implement passage management, for example, an authentication process for authenticating a person attempting to pass through (which may include a process of determining that authentication is impossible) and a tracking process for recording the movement history of that person are implemented. These processes are preferably implemented early, for example, to ensure time for recording a person's passage or for performing processes such as regulating a person's movement like opening and closing a door.

[0015] For example, it is considered that a camera is installed above a person and a gate (for example, on the ceiling), and the camera on the ceiling photographs the person and the gate, and a tracking process of the photographed person is performed by analyzing the photographed image.

[0016] However, when installing a camera on the ceiling, depending on the structure or environment of the installation location, the installation location of the camera may be restricted. Also, even if installation is possible, if large-scale construction is required for installation, the installation cost increases. Therefore, the introduction of a passage management system using a camera installed on the ceiling may be difficult in some cases.

[0017] For example, it is conceivable to provide an arch-shaped or pole-shaped support portion that extends above the gate on the gate and install a camera on the support portion. However, a gate having such a support portion has a larger size in the height direction than a gate without a support portion, so the installation location of the gate may be restricted. Also, in terms of design, it may not be preferable to provide a support portion on the gate.

[0018] In the present embodiment, by performing tracking processing using an image captured by a camera used for face authentication processing of a person attempting to pass through the gate, a passage management process including authentication processing and tracking processing is implemented. By using the camera used for authentication processing also for tracking processing, it is not necessary to separately provide a device for tracking processing (for example, a camera on the ceiling part). Therefore, an increase in the introduction cost of passage management can be suppressed. Also, compared to the case of providing equipment such as a dedicated camera for tracking processing, the restrictions on the installation location are relaxed, so the degree of freedom in the installation location is improved and the introduction of the passage management system becomes easier.

[0019] <Example of gate configuration> FIG. 1 is a diagram showing an example of a gate 10 according to the present embodiment. FIG. 1 is a view of the gate 10 seen from above, and illustrates a state in which a person h enters from the entrance / exit E1 of the gate 10 and exits from the entrance / exit E2 of the gate 10. Note that for the gate 10 shown in FIG. 1, a person may enter from the entrance / exit E2 and exit from the entrance / exit E1. In other words, the gate 10 allows a person to pass in both directions.

[0020] The gate 10 has, for example, side walls V facing each other, and a passage L for guiding a person passing through the gate 10 is formed between the side walls V. On the upper part of one side wall V having a height of about 1 m, cameras 11 are provided, for example, at two positions closer to each of the entrance / exit E1 and E2 than the central part of the side wall V, and a total of four cameras 11 (11-R1, 11-R2, 11-L1, 11-L2) are provided for the two side walls V.

[0021] Cameras 11-R1 and 11-L1 are installed, for example, on side wall V at a position closer to entrance / exit E2 than the central part of gate 10, and are used to photograph a person who enters gate 10 from entrance / exit E1 on the opposite side and passes through to entrance / exit E2.

[0022] On the other hand, cameras 11-R2 and 11-L2 are installed, for example, on side wall V at a position closer to entrance / exit E1 than the central part of gate 10, and are used to photograph a person who enters gate 10 from entrance / exit E2 on the opposite side and passes through to entrance / exit E1.

[0023] Exemplarily, camera 11-R1 is installed at a position where it can photograph a person entering from entrance / exit E1 from the right front of the person. Camera 11-L1 is installed, for example, at a position where it can photograph a person entering from entrance / exit E1 from the left front of the person.

[0024] Camera 11-R2 is installed, for example, at a position where it can photograph a person entering from entrance / exit E2 opposite to entrance / exit E1 from the right front of the person. Camera 11-L2 is installed, for example, at a position where it can photograph a person entering from entrance / exit E2 opposite to entrance / exit E1 from the left front of the person.

[0025] Therefore, a person who enters gate 10 from entrance / exit E1 and passes through to entrance / exit E2 is photographed from two directions (for example, the left-right direction) by two cameras 11-R1 and 11-L1 installed at positions separated from each other with passage L sandwiched between them on the upper parts of two side walls V.

[0026] On the other hand, a person passing through gate 10 in the reverse direction, that is, a person who enters gate 10 from entrance / exit E2 and passes through to entrance / exit E1, is photographed from two directions (for example, the left-right direction) by two cameras 11-R2 and 11-L2 installed at positions separated from each other with passage L sandwiched between them on the upper parts of two side walls V.

[0027] Note that, in FIG. 1, it is a configuration example in which a person can enter from both the entrance E1 and the exit E2 of the gate 10, but the present disclosure is not limited to this. For example, the gate 10 may be configured such that a person can enter from one entrance (for example, entrance E1) and cannot enter from the other entrance (for example, exit E2). In the case where the gate 10 does not allow a person to enter from the exit E2, the cameras 11-R2 and 11-L2 may not be provided. In the case where the gate 10 does not allow a person to enter from the entrance E1, the cameras 11-R1 and 11-L1 may not be provided.

[0028] Also, in FIG. 1, an example is shown in which a person entering from the entrance of the gate 10 is photographed from two directions by two cameras 11, but the present disclosure is not limited to this. For example, a person entering from the entrance of the gate 10 may be photographed from one direction (the direction from the exit to the entrance of the gate 10) by one camera 11. Alternatively, when a person entering from the entrance of the gate 10 is photographed from two directions by two cameras 11, one of the images photographed by the two cameras 11 may be selected, and authentication processing and tracking processing may be executed based on the selected image.

[0029] Hereinafter, by way of example, an example will be described in which, in the gate 10 shown in FIG. 1, passage management of a person entering from the entrance E1 and passing through to the exit E2 is performed using an image photographed by the camera 11-R1 or the camera 11-L1. Note that the cameras 11-R1 and 11-L1 may be collectively referred to as the camera 11 for convenience.

[0030] Also, a person entering the gate 10 corresponds to a person to be processed including face authentication processing. Hereinafter, a person to be processed will be referred to as a "subject".

[0031] Note that the gate 10 in FIG. 1 is an example, and the present disclosure is not limited to this. For example, five or more cameras 11 may be provided in the gate 10, or three or fewer cameras 11 may be provided. By varying the photographing direction and / or angle of the camera 11, a person's face can be photographed in a wider range.

[0032] Note that the plurality of cameras 11 do not have to be identical to each other. For example, the plurality of cameras 11 may be configured to capture images with different resolutions, viewing angles, and image qualities. Further, the installation position and / or the shooting direction of the camera 11 may be fixed or adjustable.

[0033] <System Configuration> FIG. 2A is a diagram showing an example of a conceptual configuration of the passage management system according to the present embodiment. FIG. 2B is a block diagram showing a configuration example of the passage management system according to the present embodiment. The passage management system 1 according to the present embodiment is a system for managing the passage of persons at a gate 10 (for example, an entrance gate, a ticket gate, etc.) installed at an entrance or exit of a facility such as an airport, a station, or an event venue.

[0034] In the passage management system 1 according to the present embodiment, illustratively, the management of the entry and exit of users who use the facility is executed by face authentication. For example, when a user passes through the gate 10 and enters the facility, it is determined by face authentication whether the user is a person permitted to enter the facility. Further, when a user passes through the gate and exits the facility, it is determined by face authentication which user exits the facility. Note that "face authentication" may be regarded as a concept included in "matching using a face image".

[0035] The passage management system 1 includes, for example, the gate 10 illustrated in FIG. 1, the camera 11 (right camera 11 or left camera 11), the face authentication function unit 12, the passage management function unit 13, the face authentication server 14, and the passage history management server 15. Note that in the passage management system 1, the number of gates 10 may be one or plural.

[0036] The gate 10 is installed in a facility such as an airport, a station, or an event venue. A user permitted to use the facility passes through the gate 10 when entering the facility and / or exiting the facility. Further, the gate 10 performs control to prevent the passage of persons not permitted to enter the facility.

[0037] As shown in FIG. 1, the camera 11 is provided, for example, on the side wall V of the gate 10. The camera 11 captures a shooting range including a person passing through the gate 10 and the face of the person when there is a person attempting to pass through the gate 10. For example, the shooting range of the camera 11 is a range capable of capturing the front face of a person.

[0038] The face authentication function unit 12 performs face authentication processing on the image. In the present embodiment, illustratively, the face authentication function unit 12 performs at least two-stage face authentication processing.

[0039] For example, the face authentication function unit 12 includes a camera control unit 121, a first face matching processing unit 122, and a second face matching processing unit 123. The first face matching processing unit 122 performs the first-stage face authentication processing, and the second face matching processing unit 123 performs the face authentication processing from the second stage and later.

[0040] The camera control unit 121 controls, for example, the shooting timing of the camera 11. For example, the camera 11 shoots at a speed of about 5 fps under the control of the camera control unit 121. Also, the right camera 11 and the left camera 11 may shoot simultaneously or with a difference in shooting timing within an allowable range under the control of the camera control unit 121, or the synchronization of their shooting timings may be achieved. Alternatively, the right camera 11 and the left camera 11 may shoot asynchronously and at independent shooting timings under the control of the camera control unit 121.

[0041] The camera control unit 121 detects a face frame from, for example, an image captured by the camera 11. The method for detecting the face frame is not particularly limited. For example, parts (eyes, nose, and mouth) included in the face are detected from the image, and based on the positions and color information of the detected parts, the boundary between the face region and the region outside the face is detected, so that a frame (face frame) surrounding the face region may be detected. When a face frame is detected, for example, the camera control unit 121 outputs information regarding the detected face frame (face frame information) and the captured image to the first face matching processing unit 122 and / or the second face matching processing unit 123. For example, the camera control unit 121 determines the output destination to either the first face matching processing unit 122 or the second face matching processing unit 123 according to the positional relationship between the gate 10 and the person. Hereinafter, the description of "face frame" may refer to a frame surrounding the face region or the face region surrounded by the frame.

[0042] The first face matching processing unit 122, for example, cuts out the face region included in the image based on the face frame information, and notifies the face authentication server 14 of a face authentication request including information of the cut-out face region. The information of the face region may be, for example, an image of the face region or information indicating feature points extracted from the image of the face region.

[0043] Exemplarily, face images of persons permitted to pass through the gate 10 are registered in the face authentication server 14. The face images registered in the face authentication server 14 may be described as registered face images. The registered face images may be associated with information that can uniquely identify or specify a person, such as the ID of the registered person. Further, the registered face images may be, for example, information indicating feature points extracted from the image.

[0044] When the face authentication server 14 receives a face authentication request from, for example, the first face matching processing unit 122, it determines (matches) whether a face of the same person as the face in the face area included in the face authentication request is included in the registered face images. The face authentication server 14 notifies, for example, the first face matching processing unit 122 of a face matching result including the determination result. Note that the face matching result may include, for example, information indicating whether a face of the same person as the face in the face area is included in the registered face images (for example, a flag indicating "OK" or "NG") and, when a face of the same person as the face in the face area is included in the registered face images, information (for example, an ID) of the person associated with the registered face images.

[0045] Matching, for example, is to determine whether a pre-registered registered face image and a face image of a person passing through the gate 10 match, or whether a pre-registered registered face image and a face image of a person passing through the gate 10 are face images of the same person, by comparing the registered face image with the face image of the person passing through the gate 10.

[0046] On the other hand, authentication, for example, is to prove to the outside (for example, the gate 10) that a person with a face image that matches a pre-registered face image is the person himself / herself (in other words, a person who may be permitted to pass through the gate 10).

[0047] However, in the present disclosure, "matching" and "authentication" may be used as mutually interchangeable terms.

[0048] For example, the matching process is a process of comparing feature points of a pre-registered registered face image with feature points extracted from a detected face area to identify who the face in the image data is. For this matching process, for example, a method using machine learning may be used. Further, the matching process may be performed, for example, in the face authentication server 14, but may also be performed in another device such as the gate 10, or may be distributedly processed by a plurality of devices.

[0049] The first face matching processing unit 122 outputs information including, for example, the matching processing result to the second face matching processing unit 123 and the passage management function unit 13. The matching processing result may include, for example, information regarding the registered face image and the matching score. Further, the information output from the first face matching processing unit 122 may include, for example, face frame detection information, the shooting time of the face capture camera image in which the face frame was detected, and information on the face region cropped in the first face matching processing unit 122.

[0050] Note that the output destinations of the first face matching processing unit 122 are not limited to the second face matching processing unit 123 and the passage management function unit 13. For example, the first face matching processing unit 122 outputs information including the matching processing result to a display device that presents image information to a person attempting to pass through the gate 10 and / or an audio output device that presents audio information to the person.

[0051] The second face matching processing unit 123, for example, crops the face region included in the image based on the face frame information, and performs face matching using the information on the cropped face region.

[0052] For example, the second face matching processing unit 123 compares the information on the face region cropped in the second face matching processing unit 123 with the information on the face region cropped in the first face matching processing unit 122, and determines whether the person corresponding to the face region cropped in the second face matching processing unit 123 is the same as the person corresponding to the face region cropped in the first face matching processing unit 122.

[0053] Alternatively, the second face matching processing unit 123 compares the information on the face region cropped in the second face matching processing unit 123 with the information regarding the registered face image included in the matching processing result, and determines whether the person corresponding to the face region cropped in the second face matching processing unit 123 is the same as the person corresponding to the registered face image included in the matching processing result.

[0054] The second face verification processing unit 123 outputs information including the determination result as to whether the persons corresponding to the two face regions are the same to the passage management function unit 13. Note that the output destination of the second face verification processing unit 123 is not limited to the passage management function unit 13. For example, the second face verification processing unit 123 outputs the information including the determination result to a display device that presents image information to the person attempting to pass through the gate 10 and / or a voice output device that presents voice information to the person.

[0055] The passage management function unit 13 manages the passage of persons located around the gate 10, for example, based on the information output from the face authentication function unit 12. Persons located around the gate 10 include, for example, persons passing through the gate 10, persons attempting to pass through, and persons passing by around the gate 10. Here, the person attempting to pass through the gate 10 is not limited to a person permitted to pass through the gate 10 (for example, a person whose face image is registered in the face authentication server 14), and may be, for example, a person whose face image is not registered in the face authentication server 14 but who attempts to pass through. Also, the person passing by around the gate 10 is, for example, a person who is not attempting to pass through the gate 10 but has passed through the shooting range of the camera 11 or has entered the shooting range without attempting to pass through the gate 10. Also, the state of the person may be, for example, a state related to the movement of the person, such as whether the person is moving, stationary, and the moving direction when the person is moving.

[0056] The passage management function unit 13 includes, for example, a passage management state transition processing unit 131, a history management unit 132, and a history database (DB) 133.

[0057] The passage management state transition processing unit 131 transmits control information regarding the control of the gate 10 when a person permitted to pass through the gate 10 passes through, for example, in the person passage management process. Also, the passage management state transition processing unit 131 transmits control information regarding the control of the gate 10 when a person not permitted to pass through the gate 10 attempts to pass through, for example.

[0058] The history management unit 132 stores and manages information (passing history information) indicating the history of the person who passed through the gate 10, for example. Also, the history management unit 132 stores the passing history information in the history DB 133 and transmits the passing history information to the passing history management server 15, for example. In the railway network, for example, the history management unit 132 may manage the local passing history information for each station (or each ticket gate).

[0059] The passing history management server 15 stores and manages information (passing history information) indicating the history of the person who passed through the gate 10, for example. For example, the passing history management server 15 may manage the passing history information of a plurality of gates 10. In a large facility with a plurality of entrances and exits, for example, the passing history information of the gates 10 provided at each of the plurality of entrances and exits may be aggregated and managed by the passing history management server 15. Also, in the railway network, for example, the passing history information of each of the gates 10 installed at the ticket gates of a plurality of stations may be aggregated and managed by the passing history management server 15.

[0060] Note that the passing management function unit 13 may output information related to passing management (passing management information) to a display device (not shown), for example. The passing management information may include information output from the face authentication function unit 12, for example. The display device may display the state of the person (for example, the result of face authentication of the person and the moving direction), for example. For example, the display device may display an image and superimpose the detected face frame on the image. Also, the display device may superimpose and display information related to the person (person ID) obtained by face authentication on the image, for example.

[0061] The above-described face authentication function unit 12 may operate synchronously or asynchronously with the passing management function unit 13, for example. In the case of asynchronous operation, for example, the face authentication function unit 12 may operate when a face frame is detected in the camera control unit 121.

[0062] At least a part of the functions of the above-described face recognition function unit 12 and the passage management function unit 13 may be replaced with each other. For example, a part of the functions of the passage management function unit 13 may be included in the face recognition function unit 12. For example, the face recognition function unit 12 may include an output unit that outputs that a person corresponding to two face regions is determined to be the same and that the person to be determined has passed through the gate 10. The information output from the output unit may include control information regarding the control of the gate 10 when a person permitted to pass through the gate 10 passes, for example, control information regarding the control of the gate 10 when a person not permitted to pass through the gate 10 attempts to pass.

[0063] The configurations of the above-described face recognition function unit 12 and the passage management function unit 13 may each have the form of one information processing device (for example, a server device), or two may be included in one information processing device. For example, the face recognition function unit 12 may have the form of one information processing device, and the passage management function unit 13 may be included in one information processing device. Further, the face recognition function unit 12 and / or the passage management function unit 13 having the form of an information processing device may be included in the gate 10.

[0064] Note that the above-described information processing device may include a processor, a memory, and an input / output interface used for transmitting various information. The processor is an arithmetic device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory is a storage device realized using, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The processor, the memory, and the input / output interface are connected to a bus and transfer various information via the bus. The processor realizes the functions of the configurations included in the information processing device by, for example, reading a program, data, etc. stored in the ROM onto the RAM and executing processing.

[0065] In the above-described face recognition function unit 12, for example, the timing of the face recognition process may be determined according to where a person is present in a zone defined around the gate 10. Hereinafter, an example of the zone defined for the gate 10 will be described below.

[0066] <Gate area management> FIGS. 3A and 3B are diagrams showing examples of zones defined for the gate 10. In FIGS. 3A and 3B, examples of a plurality of zones are shown when the gate 10 is viewed from above. Note that in FIGS. 3A and 3B, an example is shown in which the side wall V of the gate 10 forming the passage L extends along the vertical direction of the paper surface.

[0067] As shown in FIG. 1, among the entrance / exit E1 and E2 of the gate 10, for example, the upstream side along a specific entry direction (for example, the entry direction) corresponds to the entrance, and the downstream side corresponds to the exit.

[0068] FIG. 3A shows an example of a zone defined when a person enters through the entrance / exit E2 in the gate 10 where a person can enter from both the entrance / exit E1 and E2. FIG. 3B shows an example of a zone defined when a person enters through the entrance / exit E1.

[0069] When the gate 10 can be entered in both directions, it is conceivable that whether the moving direction is normal or not differs depending on the entrance / exit through which a person enters. For example, the moving direction of a person from the entrance / exit E1 to the entrance / exit E2 is a normal moving direction for a person entering from the entrance / exit E1, while it is an abnormal moving direction for a person entering from the entrance / exit E2. With respect to such differences in regulations, in the passage management function, for example, the side of the entrance / exit E1 is defined as the "north side", and the side of the entrance / exit E2 is defined as the "south side".

[0070] Note that the expressions "north side" and "south side" are just examples, and the present disclosure is not limited to these expressions. For example, the expressions "north side" and "south side" do not limit the arrangement of gate 10 to an arrangement along the geographical north-south direction. For example, when the passage L of gate 10 is located along a direction different from the north-south direction, or when the passage includes a curve, one side may be defined as the "north side" and the other side as the "south side".

[0071] For example, FIG. 3A shows an example of the zones defined when a person enters from entrance / exit E2. In FIG. 3A, for gate 10, "Zone outside-S" (area outside the south zone), "Zone A", "Zone B", and "Zone C" are defined.

[0072] In contrast, FIG. 3B shows an example of the zones defined when a person enters from entrance / exit E1. In FIG. 3B, for gate 10, "Zone outside-N" (area outside the north zone), "Zone A", "Zone B", and "Zone C" are defined.

[0073] Hereinafter, each zone will be described with reference to the example of FIG. 3A. Note that the example of FIG. 3B is the same as that of FIG. 3A except for the point that a person enters from entrance / exit E1 and the "Zone outside-S" (area outside the south zone) is replaced by the "Zone outside-N" (area outside the north zone).

[0074] The boundary between the area outside the south zone and Zone A may be referred to as, for example, the "face authentication start line".

[0075] The "Face Authentication Start Line" is used, for example, to determine whether to start the face authentication process (face matching process) in the first face matching processing unit 122. For example, when a person crosses the "Face Authentication Start Line" and enters Gate 10, the face authentication process is started. For example, the first face matching processing unit 122 issues a face authentication request from the face frame information, links the matching result (face authentication ID) with the person detection information, and starts tracking the person. Note that the "Face Authentication Start Line" may be referred to as the "A Line".

[0076] The "Face Authentication Start Line" may be provided outside Gate 10 (for example, on the upstream side along the path of Gate 10). Also, the "Face Authentication Start Line" is not limited to a single line segment, and may have a plurality of line segments, for example, in a U-shape. Note that the shape having a plurality of line segments is not limited to a shape corresponding to a part of the sides of a rectangular shape such as a U-shape, and may be a shape corresponding to a part of the sides of other polygonal shapes. Alternatively, the "Face Authentication Start Line" may have an arc or may be a shape in which a straight line and a curve are mixed. For example, since the "Face Authentication Start Line" has a plurality of line segments and / or arcs, the face authentication process can be started not only when a person enters from the front of Gate 10 but also when a person enters from a direction shifted from the front, such as the side.

[0077] The boundary between Zone A and Zone B may be referred to as, for example, the "Closed Door Limit Line".

[0078] The "door closing limit line" indicates, for example, the position where the closing of the exit-side gate door in response to a door closing instruction can catch up with a person before they pass through. The "door closing limit line" is determined, for example, by considering the maximum speed assumed as the speed at which a person passes through gate 10 (for example, 6 km / h. Hereinafter, the "maximum passable speed") and the time required to physically close the gate door (for example, 0.5 seconds). For example, the "door closing limit line" is set in front of the physical position of the gate door (the "gate door position") by a distance corresponding to the maximum passing speed × the time required to physically close the gate door. By doing so, when a person who is not permitted to pass through gate 10 passes through the "door closing limit line" and moves at the maximum passable speed, the exit-side gate door is closed before the person passes through the exit-side gate door.

[0079] Note that the "door closing limit line" may also be referred to as the "illegal intrusion detection line" or the "B line (B LINE)".

[0080] The boundary between zone B and zone C may be referred to as the "exit line".

[0081] The "exit line" indicates, for example, the position where it is determined that the person has exited gate 10. The "exit line" may be provided outside gate 10, for example, in the same manner as the above-mentioned "face authentication start line". Also, the "exit line" is not limited to a single line segment, and may have a plurality of line segments, for example, in a U-shape. Alternatively, the "exit line" may have an arc. Note that the "exit line" may also be referred to as the "Z line (Z LINE)", for example.

[0082] In passage management, the physical position of the gate door (the "gate door position") may be merely a passage point. In this case, the physical position of the gate door may be different from or the same as the logically set "exit line". For example, in actual operation, the "gate door position" and the "exit line" may be set to be the same.

[0083] For example, in the case of the gate 10 that charges a person passing through, the "exit line" may correspond to the "charging line".

[0084] For example, when a person who has entered the gate 10 crosses the charging line (for example, when entering from zone B to zone C), the person is charged. In other words, when the person has not crossed the charging line (for example, when not entering zone C), the person is not charged. By providing this charging line, it is possible to avoid the mistake of charging a person who has entered the gate 10 but turned back before crossing the charging line.

[0085] Note that in the above, an example where the "charging line" corresponds to the "Z line" ("exit line") has been shown. However, for example, the "charging line" may correspond to the "B line".

[0086] Note that in the above example, an example where three zones are defined has been shown excluding the north side out-of-zone area and the south side out-of-zone area. However, the present disclosure is not limited to this. The number, size, position, and shape of the zones may be changed according to the situation to which the present disclosure is applied.

[0087] The face authentication function unit 12 in the present embodiment executes, for example, the first-stage face authentication process when a person enters the above-described zone A, and executes the second-stage face authentication process when the person enters from zone A to zone B. By performing the authentication process in two stages, it is possible to confirm whether a person authenticated in the first stage has passed through the gate 10 by the second-stage face authentication process. Hereinafter, the flow of the two-stage face authentication process will be described.

[0088] FIG. 4 is a flowchart showing an outline of the flow of the two-stage face authentication process. The flow shown in FIG. 4 may be started, for example, every time a passerby approaches.

[0089] The face authentication processing unit 12 detects that a passerby approaches the gate 10 and enters the first authentication area (for example, zone A) (S101). Note that the first authentication area is not limited to zone A shown in FIGS. 3A and 3B. The first authentication area may be an area on the upstream side of the charging line of the gate 10, and in the captured image, for example, an area where a face frame of a size capable of face authentication can be obtained. Whether or not the passerby has entered the first authentication area may be determined by a sensor (for example, a photoelectric sensor) provided at the gate 10.

[0090] The face authentication function unit 12 executes the first face authentication process on the passerby who has entered the first authentication area (S102). For example, the face authentication function unit 12 transmits a face authentication request including information about the face frame in the image captured by the camera 11 to the face authentication server 14. When the face authentication server 14 receives the face authentication request, for example, it determines whether or not there is a passerby among the registered persons based on the information about the face frame. As a result of the determination, if there is a passerby among the registered persons, the face authentication server 14 transmits information indicating the corresponding registered person to the face authentication function unit 12, for example. On the other hand, if there is no passerby among the registered persons, the face authentication server 14 transmits information indicating the result of the determination to the face authentication function unit 12.

[0091] The face authentication function unit 12 determines whether or not the authentication has succeeded as a result of the first face authentication process (S103).

[0092] If the authentication fails (NO in S103), the face authentication function unit 12 executes the process of S102 again. Here, the case where the authentication fails is, for example, the case where the passerby is not included among the registered persons held by the face authentication server 14. The fact that the passerby is not included among the registered persons held by the face authentication server 14 may be regarded as a person who is not permitted to pass through the gate 10. Alternatively, the case where the authentication fails may be the case where, even though the passerby is included among the registered persons, it is erroneously determined that the passerby and the registered person are not the same person in the face authentication process.

[0093] When authentication is successful (YES in S103), wait until the entry of the passerby into the second authentication area (Zone B) is detected (S104). Note that the second authentication area is not limited to Zone B shown in FIGS. 3A and 3B. For example, the second authentication area may be an area beyond the charging line provided at Gate 10.

[0094] The face authentication processing unit 12 detects that the passerby has entered the second authentication area (S105). Whether the passerby has entered the second authentication area may be determined by a sensor such as a photoelectric sensor, for example.

[0095] The face authentication function unit 12 performs a second face authentication process on the passerby who has entered the second authentication area (S106). For example, instead of requesting the face authentication server 14 to perform the second face authentication process, the face authentication function unit 12 may perform the second face authentication process locally at Gate 10 itself (or the face authentication function unit 12 itself).

[0096] The face authentication function unit 12 determines whether authentication has been successful as a result of the second face authentication process (S107).

[0097] When authentication fails (NO in S107), the face authentication function unit 12 determines, for example, that the passerby is not allowed to pass (S109). In this case, a process of closing the door of Gate 10 (door closing process) is executed. Note that when authentication fails (NO in S107), before determining that the passerby is not allowed to pass, the first face authentication process similar to S103 may be performed again (re - authentication or re - verification). In this case, similar to S103, the face authentication function unit 12 transmits a face authentication request to the face authentication server 14, but the information included in the face authentication request may be information regarding the face frame in the image in S107. And the flow shown in FIG. 4 ends.

[0098] If authentication is successful (YES in S107), the face authentication function unit 12 determines that it may permit the passerby to pass through the gate 10 (S108). In response to this determination, the face authentication function unit 12 may, for example, instruct the gate 10 to open the door that restricts passage through the gate 10. Then, the flow shown in FIG. 4 ends.

[0099] Through the two-stage face authentication process as described above, it is possible to confirm whether a person who has succeeded in authentication in the first stage has passed through the gate 10 by the second-stage face authentication process.

[0100] When performing face authentication processing in two stages, there are a plurality of methods with different methods and / or timings for the second-stage face authentication processing. For example, as shown below, there are a multi-local authentication method, a local face authentication tracking method, and a post-batch method.

[0101] In the multi-local authentication method, the information acquired from the face authentication server in the first-stage face authentication process is registered in the local face authentication engine, and in the face authentication process after the second stage, the registered information is used.

[0102] For example, in the multi-local authentication method, when receiving a face matching result from the face authentication server 14, a face matching result including information on the feature amount of the face of the candidate is received. Then, the information on the feature amount of the face of the candidate is registered in the local face authentication engine. In the face authentication process after registering the information in the local face authentication engine, the degree of coincidence indicating whether the passerby and the candidate are the same person is confirmed based on the image of the passerby taken and the registered information. Note that in the multi-local authentication method, the information on the feature amount of the face of the candidate does not necessarily have to be included in the face matching result. The face authentication function unit 12 may receive the information on the feature amount of the face of the candidate as information different from the face matching result, or may receive it at a timing different from the face matching result. That is, in the face authentication server 14, if the face authentication function unit 12 can receive the feature amount of the face of the candidate corresponding to the registrant when the collation between the image information of the first face image area and the registrant's information is successful, the data structure and / or timing of the received information are not limited.

[0103] In the multiple local authentication method, since face authentication processing is performed multiple times, the legitimacy of the passerby being a person permitted to pass through Gate 10 can be improved. Here, in the multiple local authentication method, since face authentication processing from the second stage onward is performed using the feature amounts registered in the face authentication server 14, the face authentication processing from the second stage onward is also carried out with the same accuracy as the face authentication processing using the face authentication server 14. Therefore, it is possible to more reliably confirm that the passerby is a person permitted to pass through Gate 10.

[0104] In the local face authentication tracking method, information regarding the passerby included in the image used in the first face authentication processing is registered in the local face authentication engine, and the registered information is used in the face authentication processing from the second stage onward.

[0105] For example, in the local face authentication tracking method, at the timing of sending a face authentication request to the face authentication server 14, face registration of the passerby is executed for the local face authentication engine. Here, the face registration of the passerby may correspond to extracting the face area of the passerby from the image capturing the passerby and registering the information of the image of the extracted area (that is, the face image), or may correspond to registering the feature amounts obtained from the extracted face image. In the face authentication processing after registering the information in the local face authentication engine, face matching (1:1 face matching) is executed between the image capturing the passerby and the registered face image to confirm that the captured passerby and the registered passerby are the same person. Then, when the size of the face frame and the capturing area in the image capturing the passerby are equal to or larger than the specified size and are within the specified area, and further, when the matching result from the face authentication server 14 is OK, passage is permitted.

[0106] In the local face authentication tracking method, a passerby who has successfully authenticated that they are a person permitted to pass through Gate 10 in the first face authentication process can be tracked for entering and passing through Gate 10 in the subsequent authentication processes from the second stage onwards. Therefore, passage through Gate 10 can be guaranteed. Also, in the local face authentication tracking method, since the feature amounts used in the first face authentication process are used in the authentication processes from the second stage onwards, authentication reflecting the most recent environment (e.g., lighting, sunlight) and the like can be performed in the authentication processes from the second stage onwards. As a result, even when there is a divergence between the face image registered in the face authentication server 14 and the latest face image (e.g., the face region in the image captured by the camera 11) due to the influence of shadows near Gate 10 and aging changes, etc., passage management can be appropriately performed.

[0107] In the post-batch method, similar to the above local face authentication tracking method, information regarding the passerby included in the image used in the first face authentication process is registered in the local face authentication engine, and the registered information is used in the face authentication processes from the second stage onwards. However, different from the local face authentication tracking method, in the post-batch method, the face authentication processes from the second stage onwards are not executed in real time. For example, the face authentication processes from the second stage onwards are executed in a batch at a specific timing. The specific timing may be a time period when there are few people passing through Gate 10 (e.g., at night), or a timing such as the end of the month.

[0108] In the post-batch method, since real-time performance is not pursued, waste of computer resources (e.g., CPU processing power) related to face authentication processing can be suppressed in the control device, server, etc.

[0109] Note that the examples of the two-stage authentication method are not limited to the above examples. Also, in the above examples, an example is shown where the authentication from the second stage onwards is executed by the local face authentication engine instead of sending a face authentication request to the face authentication server 14, but the present disclosure is not limited to this. For example, even in the second stage, a face authentication request may be sent to the face authentication server, and it may be determined in the face authentication server 14 whether there are candidates who are the same person as the passerby.

[0110] In addition, in FIG. 4, in the two-factor authentication method, the timing of performing authentication is shown as an example in which a photosensor detects that a passerby has entered a certain specific area, but the present disclosure is not limited thereto.

[0111] For example, the camera 11 photographs a person entering through the entrance of the gate 10 from the direction from the exit to the entrance of the gate 10. Therefore, the face frame of a person present in the area for performing the first-stage face authentication (for example, zone A in FIG. 3A) is smaller than the face frame of a person present in the area for performing the second-stage face authentication (for example, zone B in FIG. 3A).

[0112] Using the fact that the size of the face frame is different between the area for performing the first-stage face authentication and the area for performing the second-stage face authentication, instead of the photosensor, the timing of performing authentication may be specified based on the size of the face frame. Hereinafter, an example of two-stage face authentication processing based on the size of the face frame will be described.

[0113] FIG. 5 is a flowchart showing a first example of the flow of two-stage face authentication processing. The flow shown in FIG. 5 may be started, for example, every time a passerby approaches.

[0114] The face authentication functional unit 12 acquires an image and detects a face frame (S201).

[0115] The face authentication functional unit 12 determines whether the size of the face frame is equal to or greater than a predetermined first threshold value (S202). Note that the size of the face frame and the first threshold value to be compared with the size of the face frame may be defined by the area in the image region, or may be defined by the vertical length or the horizontal length of the face frame. Further, the first threshold value may be defined, for example, based on the lower limit or the upper limit of the size of the face frame used in the face authentication processing in S203 described later.

[0116] If the size of the face frame is not equal to or greater than the first threshold value (NO in S202), the face authentication functional unit 12 executes the process of S201.

[0117] When the size of the face frame is greater than or equal to the first threshold value (YES in S202), the face authentication function unit 12 executes the first face authentication process for the detected face frame (S203). In the first face authentication process, a face authentication request is sent to the face authentication server 14, and the face authentication server 14 determines whether there is a candidate who is the same person as the passerby among the registered persons.

[0118] As a result of the first face authentication process, the face authentication function unit 12 determines whether the authentication was successful (S204). The determination of failure or success in authentication may be the same as S104 in FIG. 4.

[0119] If the authentication fails (NO in S204), the face authentication function unit 12 executes the process of S201.

[0120] If the authentication is successful (YES in S204), the face authentication function unit 12 acquires an image at the next timing and detects a face frame (S205).

[0121] The face authentication function unit 12 determines whether a timeout has occurred (S206). For example, in the process of S206, a timer is started at the timing when the authentication is successful in the first face authentication process in S203, and if the time indicated by the timer is longer than or equal to a predetermined time, it is determined that a timeout has occurred. Here, the time indicated by the timer is the time from the timing when the authentication is successful in the first face authentication process in S203 to the timing when the determination in S206 is executed. Also, for example, the predetermined time may be the upper limit time allowed for a passerby who has successfully authenticated in the first face authentication process to pass through the gate 10. By this determination of timeout, for example, appropriate processing (such as a warning to the passerby) can be executed for a situation where a passerby who has successfully authenticated in the first stage stays in the passage of the gate 10.

[0122] If a timeout occurs (YES in S206), the flow ends.

[0123] If no timeout occurs (NO in S206), the face authentication function unit 12 determines whether the size of the face frame is equal to or greater than a predetermined second threshold value (S207). For example, the second threshold value is larger than the first threshold value. For example, the second threshold value may be the lower limit or the upper limit of the size of the face frame used in the face authentication process of S208 described later. Alternatively, the second threshold value may have an average face size and may be defined by the size of the face frame of an image of a person who has reached the billing line. Further, the second threshold value may be defined by a value obtained by multiplying the size of the face frame to be processed in S203 by a coefficient greater than 1.

[0124] If the size of the face frame is less than the second threshold value (NO in S207), the face authentication function unit 12 executes the process of S205.

[0125] If the size of the face frame is equal to or greater than the second threshold value (YES in S207), the face authentication function unit 12 performs a one-to-one face authentication process, which is the second face authentication process, on the face frame detected in S205 (S208).

[0126] For example, in the second face authentication process of the multi-local authentication method, the information on the feature amount of the face of the candidate acquired in S203 is compared with the information on the face frame of the passerby detected in S205 to determine whether the candidate and the passerby are the same person.

[0127] Also, for example, in the second face authentication process of the local face authentication tracking method, the information on the face frame on which the first face authentication process was performed in S203 is compared with the information on the face frame of the passerby detected in S205 to determine whether the candidate and the passerby are the same person.

[0128] As a result of the second face authentication process, the face authentication function unit 12 determines whether the authentication was successful (S209).

[0129] If authentication fails (NO in S209), the face authentication function unit 12 treats the passerby as not allowed to pass (S211). In this case, a process of closing the door of the gate 10 (door closing process) is executed. Note that if authentication fails (NO in S209), before treating the passerby as not allowed to pass, the first face authentication process similar to S203 may be performed again (re - authentication or re - verification). In this case, similar to S203, the face authentication function unit 12 transmits a face authentication request to the face authentication server 14, and the face authentication request may include information regarding the face frame in the image in S208. Then, the flow ends.

[0130] Here, the case where authentication fails may be, for example, when the information of the candidate identified by the first face authentication process in S203 does not match the information of the image of the passerby. The case where the information does not match may be, for example, when the score obtained by the face authentication process is lower than the threshold value for determining the same person. Note that the case where the information of the identified candidate does not match the information of the image of the passerby may be the case where the identified candidate and the passerby included in the image are not the same person (for example, when they are swapped on the way). Alternatively, the case where the information of the identified candidate does not match the information of the image of the passerby may be the case where, although the identified candidate and the passerby included in the image are the same person, it is erroneously determined that they are not the same person.

[0131] If authentication is successful (YES in S209), the face authentication function unit 12 may determine to permit the passerby to pass through the gate 10 (S210). In response to this determination, the face authentication function unit 12 may output, for example, an instruction to open the door that restricts the passage of the gate 10 to the gate 10. Then, the flow ends.

[0132] In addition, in FIG. 5, an example in which the second-stage face authentication process is executed locally is shown, but the present disclosure is not limited to this. For example, as shown below with reference to FIG. 6, in the second-stage face authentication process, the face authentication may be requested from the face authentication server 14. By doing so, the function of performing face authentication from the gate 10 can be omitted, and thus the configuration of the gate 10 can be simplified.

[0133] FIG. 6 is a flowchart showing a second example of the flow of the two-stage face authentication process. The flow shown in FIG. 6 may be started, for example, every time a passerby approaches. In FIG. 6, for the processes similar to those shown in FIG. 5, the same reference numerals are given and the description is omitted. In FIG. 6, S208 in FIG. 5 is replaced with S301.

[0134] When the size of the face frame is equal to or greater than the second size (YES in S207), the face authentication function unit 12 executes a second face authentication process on the face frame detected in S205 (S301). Here, in S301, the face authentication function unit 12 transmits a face authentication request to the face authentication server 14. The face authentication request includes, for example, the information of the face frame detected in S205. Further, in the face authentication server 14, it may be determined whether there is a candidate who is the same person as the passerby among the registered persons. Alternatively, the face authentication server 14 may simplify the process using the collation result executed based on the face authentication request in S203. For example, the information of the registered persons narrowed down by the collation result executed based on the face authentication request in S203 may be collated with the information of the face frame included in the newly received face authentication request.

[0135] As described above, in the passage management system 1 of the present embodiment, based on the face frame in the image of the person present in the first authentication area, the first face authentication process is executed, and for the person who has succeeded in the first face authentication process, based on the face frame in the image of the person present in the second authentication area between the first authentication area and the exit of the gate, the second face authentication process is executed. By the two-stage face authentication process executed by this configuration, the passage management of the target (for example, a person) for the gate can be realized with a simple configuration.

[0136] In the above-described embodiment, an example of performing two-stage face authentication processing has been shown. However, the present disclosure is not limited to this, and three-stage or more face authentication processing may be performed. For example, for a person who has succeeded in the first-stage face authentication processing, for each image captured after the first-stage face authentication processing, face frame detection and face authentication processing may be executed, and the person may be tracked. In this case, the face authentication processing after the second stage may be executed locally. With this configuration, the movement of the person can be sequentially followed, and it is possible to accurately determine whether the person has passed through Gate 10. For example, in a configuration where the person is tracked, when a sensor provided at the exit of Gate 10 (or the charging line of Gate 10) detects the passage of the person, the face authentication processing may be stopped and the tracking may be terminated.

[0137] In the first-stage face authentication processing, since the accuracy of face authentication (face matching) is ensured, the processing in the second-stage face authentication processing may be simplified. For example, the number of face feature amounts used in the second-stage face authentication processing may be reduced compared to the number of face feature amounts used in the first-stage face authentication processing. For example, when the respective feature amounts of the face parts are used in the first-stage face authentication processing, in the second-stage face authentication processing, only the feature amounts of the eyes of the face may be used, and the feature amounts of the parts other than the eyes may not be used. Thereby, the processing load related to the comparison of feature amounts can be reduced, and the processing speed in the second stage is improved.

[0138] Also, for example, in face authentication processing, when a score calculated by comparing the feature amounts of two faces (for example, the face of a registered person and the face of a passerby) is compared with a threshold value, and when it is determined that the two faces are the faces of the same person if the score is equal to or higher than the threshold value, the threshold value may be adjusted according to the stage of the face authentication processing. The threshold value used in the second-stage face authentication processing may be smaller than the threshold value used in the first-stage face authentication processing. By adjusting the threshold value, the processing load related to the comparison of scores can be reduced, and the processing speed in the second stage is improved.

[0139] In the above-described embodiment, an example was shown in which, in the second-stage face authentication process, when authentication fails, a door closing process is performed. However, the present disclosure is not limited to this. For example, in the second-stage face authentication process, when authentication fails, depending on the installation status and / or congestion status of gate 10, it may not be necessary to perform the door closing process. For example, in the case of a gate installed at a railway station, since the number of people passing through the gate is large, congestion will occur if the flow of people is stopped by the door closing process. In such a case, rather than detecting a person who passes through the gate illegally, it may not be necessary to perform the door closing process so as not to stop the flow of people.

[0140] Also, in the above-described embodiment, it was determined whether a person entered the first area and / or the second area based on whether the size of the face frame was equal to or greater than the first threshold value and / or the second threshold value. However, the present disclosure is not limited to this. The rate of change of the size of the face frame obtained based on the size of the face frame may be used to determine whether a person entered the first area and / or the second area. The rate of change of the size of the face frame represents the ratio of the change in the size of the face frame in an image that changes over time (or the movement of a person). For example, the rate of change of the size of the face frame at a certain time point t is calculated by comparing the size of the face frame obtained at time point t with the size of the face frame of the same person as the face frame at time point t, which was obtained one time point before (or a plurality of time points before) time point t. Generally, since the rate of change of the size of the face frame increases as the face approaches the camera, the rate of change of the size of the face frame changes according to the distance between the person and the camera. Also, since the rate of change of the size of the face frame indicates the ratio (or amount of change), it is less affected by differences in the size of the face for each person. Therefore, by determining whether the rate of change of the size of the face frame is equal to or greater than a predetermined threshold value, it is possible to determine the approach of the face (for example, which area the person entered) while absorbing the difference in the size of the face for each person.

[0141] In addition, in the above-described embodiment, it has been determined whether a person has entered the first area and / or the second area using the information of the face frame, but the present invention is not limited to this. Based on the distance between facial parts, it may be determined whether a person has entered the first area and / or the second area. Generally, as the face approaches the camera, the distance between facial parts in the captured image increases. Therefore, it is possible to determine the approach of the face by determining whether the distance between facial parts, such as the distance between eyes and the distance between eyes and mouth, is equal to or greater than a predetermined threshold. Examples of facial parts include facial organs such as eyes, nose, mouth, and ears, but features different from general organs such as moles, freckles, and wrinkles may also be used as facial parts. Further, instead of the distance between parts, the size of facial parts in the captured image may be used. Similar to the size of the face frame, the size of facial parts also increases as the face approaches the camera. Further, instead of the distance between parts, the rate of change of the size of facial parts may be used. Similar to the rate of change of the size of the face frame, the rate of change of the size of facial parts also increases as the face approaches the camera.

[0142] In addition, in determining which area a person attempting to pass through gate 10 has entered, the above-described methods may be used in combination, selectively, or the same method may be used at all times.

[0143] In addition, in the above-described embodiment, the success or failure of each of the first-stage face authentication process and the second-stage face authentication process may be output by voice and / or image. At this time, different voices and / or images may be output depending on the success or failure of the first-stage face authentication process and the success or failure of the face authentication process after the second stage. By doing so, it is possible to make a person attempting to pass through gate 10 aware that two or more stages of authentication are being performed. Further, when a person's face authentication process fails, it is possible to distinguish and notify at which stage (the success or failure of the first-stage face authentication process) of obtaining permission to pass through gate 10 and at which stage (the success or failure of the face authentication process after the second stage) of confirming passage through gate 10 the failure occurred.

[0144] Also, for example, in a gate provided at a railway station to manage the entry and exit of railway users into and out of the station premises, no charge is levied when entering the station premises, but a charge is levied when exiting. Therefore, when a railway user passes through the gate and enters the station premises, the face authentication process may be simplified. For example, if the authentication of the entering user fails, the information of the image for which the authentication has failed is recorded, and when the said user exits, the place of entry is identified using the information of the image recorded at the time of entry, and retrospective charging may be carried out.

[0145] In the above-described embodiment, a system for managing the passage of a person passing through gate 10 has been described, but the present disclosure is not limited thereto. For example, the present disclosure may be applied when there are no side walls of the passageway and a regulating unit (for example, a door) for regulating the passage of a person. For example, the present disclosure may be applied to a movement path from a certain zone to another zone where entry of a person is permitted according to an authentication process. In this case, a camera for photographing the face of a person passing through the movement path may be installed, for example, on a support unit or the like provided on the movement path.

[0146] Also, in the present embodiment, an example where the authentication target is a person has been shown, but the present disclosure is not limited thereto. For example, it may be applied to moving bodies such as animals and vehicles. Also, in the present embodiment, an example of performing face authentication has been shown, but the present disclosure is not limited thereto. For example, the present disclosure may be applied to authentication using an ID card indicating that the bearer has the right to pass through the gate, and other authentication methods such as biometric authentication.

[0147] Also, face authentication and other authentication methods may be used in combination. Even if passage is not permitted by face authentication according to the disclosure of the above-described embodiment, if the information of the ID card is input, passage may be exceptionally permitted.

[0148] In the above-described embodiments, the camera 11 is not limited to the example of being provided on the side wall V of the gate 10. For example, the camera 11 may be attached to a support provided on the gate 10. The support may be, for example, a pole extending vertically from the gate 10, or an arch-shaped member provided so as to cover the side wall of the gate 10.

[0149] In the above-described embodiments, a door is used as the means for restricting passage through the gate 10, but passage may be restricted directly or indirectly by other means. For example, by sounding an alarm or turning on a warning light, it may be indicated to a person attempting to pass through the gate 10 that passage through the gate 10 is restricted. Further, by sending a notification to a terminal or the like owned by an employee near the gate 10, the employee may restrict passage.

[0150] Also, depending on the congestion situation, control to block passage may or may not be performed, or the means for restricting passage may be switched. For example, in an environment where blocking or restricting the passage of people, such as when a large number of people enter and exit, may reduce safety, passage through the gate 10 may not be blocked, and information indicating that there has been unauthorized passage may be recorded. In this case, the face image or face authentication result of the person who made the unauthorized passage may be recorded in association with the information indicating that there has been unauthorized passage. This makes it possible to track down the person who made the unauthorized passage later and claim the consideration for the right of passage or the like.

[0151] In the above-described embodiments, the access control system 1 manages both entry to and exit from a facility at the entrance and exit of a facility such as an airport, a station, or an event venue. However, at the entrance or the exit, only one of entry to the facility or exit from the facility may be managed, and the other may not be managed.

[0152] The present disclosure can be realized by software, hardware, or software in cooperation with hardware.

[0153] Each functional block used in the description of the above embodiment may be realized, partially or entirely, as an LSI which is an integrated circuit, and each process described in the above embodiment may be controlled, partially or entirely, by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip so as to include part or all of the functional blocks. The LSI may be provided with data input and output. Depending on the degree of integration, the LSI may also be referred to as an IC, a system LSI, a super LSI, or an ultra LSI.

[0154] The method of integrating into an integrated circuit is not limited to LSI, and it may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, after manufacturing the LSI, an FPGA (Field Programmable Gate Array) which can be programmed, or a reconfigurable processor which can reconfigure the connection and setting of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0155] Furthermore, if a technology for integrating into an integrated circuit that replaces the LSI appears due to the progress of semiconductor technology or another derived technology, naturally, the integration of functional blocks may be performed using that technology. The application of biotechnology, etc. may be possible as an example.

[0156] The present disclosure can be implemented in any type of apparatus, device, system having a communication function (collectively referred to as a communication apparatus). The communication apparatus may include a wireless transceiver (transceiver) and a processing / control circuit. The wireless transceiver may include a receiving unit and a transmitting unit, or may include them as functions. The wireless transceiver (transmitting unit, receiving unit) may include an RF (Radio Frequency) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of the communication apparatus include a telephone (mobile phone, smartphone, etc.), a tablet, a personal computer (PC) (laptop, desktop, notebook, etc.), a camera (digital still / video camera, etc.), a digital player (digital audio / video player, etc.), a wearable device (wearable camera, smartwatch, tracking device, etc.), a game console, a digital book reader, a telehealth / telemedicine (remote healthcare / medical prescription) device, a vehicle or mobile transportation means with a communication function (automobile, airplane, ship, etc.), and combinations of the various apparatuses described above.

[0157] The communication apparatus is not limited to being portable or movable, and includes any type of apparatus, device, system that is not portable or is fixed, such as a smart home device (home appliance, lighting device, smart meter or measuring device, control panel, etc.), a vending machine, and any "Thing" that can exist on the IoT (Internet of Things) network.

[0158] In recent years, in IoT (Internet of Things) technology, CPS (Cyber Physical Systems), which is a new concept of creating new added value through information cooperation between the physical space and the cyber space, has attracted attention. This CPS concept can also be adopted in the above-described embodiments.

[0159] That is, as a basic configuration of the CPS, for example, an edge server arranged in the physical space and a cloud server arranged in the cyber space are connected via a network, and the processors mounted on both servers can distribute and process the processing. Here, each piece of processing data generated in the edge server or the cloud server is preferably generated on a standardized platform. By using such a standardized platform, it is possible to improve the efficiency when constructing a system including various diverse sensor groups and IoT application software.

[0160] For communication, in addition to data communication by a cellular system, a wireless LAN system, a communication satellite system, etc., data communication by combinations of these is also included.

[0161] In addition, the communication device also includes devices such as a controller and a sensor that are connected or linked to a communication device that executes the communication function described in the present disclosure. For example, a controller and a sensor that generate a control signal or a data signal used by the communication device that executes the communication function of the communication device are included.

[0162] In addition, the communication device also includes infrastructure facilities, such as a base station, an access point, and any other devices, devices, and systems that communicate with or control the above-described various non-limiting devices.

[0163] As described above, various embodiments have been described with reference to the drawings. Needless to say, the present disclosure is not limited to such examples. It is obvious that a person skilled in the art can come up with various modification examples or correction examples within the scope described in the claims, and it is naturally understood that those also belong to the technical scope of the present disclosure. Also, within the scope not departing from the gist of the disclosure, the components in the above embodiments may be arbitrarily combined.

[0164] As described above, specific examples of the present disclosure have been described in detail, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples exemplified above.

Industrial Applicability

[0165] One embodiment of the present disclosure is suitable for a face recognition system.

Explanation of Signs

[0166] 1 Passage management system 10 Gate 11 Camera 12 Face recognition function unit 121 Camera control unit 122 First face matching processing unit 123 Second face matching processing unit 13 Passage management function unit 131 Passage management state transition processing unit 132 History management unit 133 History DB 14 Face recognition server 15 Passage history management server

Claims

1. An acquisition unit that acquires an image including a face image of a person entering the gate, the image being taken from the direction from the exit to the entrance of the gate; A first face authentication processing unit that executes a first face authentication process based on a first face image taken in the first area for a person determined to have entered a first area defined for the gate, and outputs collation process result information including information about the collated face to a second face authentication processing unit; A second face authentication processing unit that, when the first face authentication process is successful, executes a second face authentication process based on a second face image taken in the second area for a person determined to have entered a second area defined between the first area and the exit of the gate; An output unit that outputs that the person has passed through the gate when it is determined by the second face authentication process that the person who succeeded in the first face authentication process and the person included in the second face image are the same person; An information processing apparatus comprising the above.

2. The information about the collated face includes at least one of information about a registered face image collated in the first face authentication process, face frame detection information, shooting time, and cropped face area information. The information processing apparatus according to Claim 1.

3. At least one of the first face authentication processing unit or the second face authentication processing unit determines that the person has entered the first area and the second area based on the size of the area of the face image in the image. The information processing apparatus according to Claim 1.

4. At least one of the first face authentication processing unit or the second face authentication processing unit determines that the person has entered the first area and the second area based on the detection result of an optical sensor. The information processing apparatus according to Claim 1.

5. The first face authentication processing unit: In the first face authentication process, transmits the image information of the first face image to a server having information of a registered person permitted to pass through the gate; Receives from the server a collation result of collating the image information of the first face image and the information of the registered person; Based on the collation result, determines whether the first face authentication process has succeeded; When the collation of the image information of the first face image and the information of the registered person is successful in the server, receives from the server information of a candidate corresponding to the registered person; The second face authentication processing unit: In the second face authentication process, the information of the candidate and the image information of the second face image are compared. The information processing apparatus according to claim 1.

6. In the second face authentication process, the second face authentication processing unit compares the image information of the first face image and the image information of the second face image. The information processing apparatus according to claim 1.

7. The second face authentication processing unit executes the second face authentication process using a number of feature amounts smaller than the number of feature amounts of the face used in the first face authentication process. The information processing apparatus according to any one of claims 1 to 5.

8. The second face authentication processing unit uses, for face comparison in the second face authentication process, a threshold smaller than the threshold for comparison with the matching score of face comparison in the first face authentication process. The information processing apparatus according to any one of claims 1 to 6.

9. An acquisition unit that acquires an image including a face image of a user who is photographed from a certain zone and is heading toward another zone where entry of a person is permitted according to an authentication process; A first face authentication processing unit that executes a first face authentication process based on a first face image photographed in the first area for a person determined to have entered a first area defined for the other zone, and outputs collation processing result information including information about the collated face to a second face authentication processing unit; When the first face authentication process is successful, a second face authentication processing unit that executes a second face authentication process based on a second face image photographed in the second area for a person determined to have entered a second area defined between the first area and the other zone; An output unit that outputs that the person has entered the other zone when it is determined by the second face authentication process that the person who succeeded in the first face authentication process and the person included in the second face image are the same person; An information processing apparatus comprising the above.

10. The acquisition unit of the information processing apparatus acquires an image including a face image of a person entering the gate, photographed from the direction from the exit to the entrance of the gate, and when the first face authentication processing unit of the information processing apparatus determines that the person has entered a first area defined for the gate, the first face authentication processing unit executes a first face authentication process based on the first face image of the person photographed in the first area, and outputs collation processing result information including information about the collated face to the second face authentication processing unit. When the second face authentication processing unit of the information processing apparatus determines that the person has entered a second area defined between the first area and the exit of the gate when the first face authentication processing is successful, the second face authentication processing is executed based on a second face image captured in the second area. When the output unit of the information processing apparatus determines, by the second face authentication processing, that the person who succeeded in the first face authentication processing and the person included in the second face image are the same person, the output unit outputs that the person has passed through the gate. Information processing method.

Images