Information processing device and information processing method

Abstract

This contributes to providing an information processing device and an information processing method that realizes, with a simple configuration, management of the passage of objects (for example, people) through a gate. [Solution] The information processing device comprises an acquisition unit that acquires an image of a person entering a gate taken from the direction from the gate exit toward the gate entrance; a processing unit that performs a second facial recognition process for a person who has successfully passed a first facial recognition process based on the facial image of the person in a first image taken in a first area defined relative to the gate, based on the facial image of the person in a second image taken in a second area defined between the first area and the gate exit; and an output unit that outputs a message indicating that the person has passed through the gate when the second facial recognition process determines that the person who has successfully passed the first facial recognition process and the person included in the second image are the same person, and outputs different sounds or images depending on whether the first facial recognition process is successful or the second facial recognition process is successful.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] There is known technology for managing the entry and exit of people passing through gates installed at stations, airports, etc. Patent Document 1 describes a device that, when a person obtains permission to pass through a gate using a wireless card and enters the gate from the gate entrance, tracks whether the person has passed through the gate (whether the person has returned to the gate entrance) based on the change in the position of the wireless card. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 09-330440 Summary of the Invention [Problem to be solved by the invention]

[0004] At a gate that manages entry and exit, it is desirable to manage who has passed through the gate. Hereinafter, this management is sometimes abbreviated to "passage management" or "tracking management." There is room for consideration on how to realize passage management with a simple configuration.

[0005] Non-limiting embodiments of the present disclosure contribute to providing an information processing device and an information processing method that realize, with a simple configuration, passing management of an object (for example, a person) through a gate. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present disclosure includes an acquisition unit that acquires an image of a person entering a gate taken from a direction from the exit of the gate toward the entrance of the gate; a processing unit that executes a second face recognition process for a person who has succeeded in a first face recognition process based on a face image of the person in a first image taken in a first area defined relative to the gate, based on a face image of the person in a second image taken in a second area defined between the first area and the exit of the gate; and an output unit that outputs a message indicating that the person has passed through the gate when it is determined by the second face recognition process that the person who has succeeded in the first face recognition process and the person included in the second image are the same person, and outputs different sounds or images depending on whether the first face recognition process is successful or the second face recognition process is successful.

[0007] In an information processing method according to one embodiment of the present disclosure, an information processing device acquires an image of a person entering a gate taken from a direction from the exit of the gate toward the entrance, and for the person who has succeeded in a first face recognition process based on a face image of the person in a first image taken in a first area defined relative to the gate, executes a second face recognition process based on a face image of the person in a second image taken in a second area defined between the first area and the exit of the gate, and if the second face recognition process determines that the person who has succeeded in the first face recognition process and the person included in the second image are the same person, outputs a message that the person has passed through the gate, and outputs different sounds or images depending on whether the first face recognition process is successful or the second face recognition process is successful.

[0008] In addition, these comprehensive or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0009] A non-limiting embodiment of the present disclosure can achieve passage management of objects (e.g., people) through a gate with a simple configuration.

[0010] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings, in which such advantages and / or benefits are provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of which are provided to obtain one or more identical features. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates an example of a gate according to an embodiment. [Figure 2A] FIG. 1 is a diagram illustrating an example of a conceptual configuration of a passage management system according to an embodiment. [Figure 2B] FIG. 1 is a block diagram showing an example of the configuration of a currency management system according to an embodiment; [Figure 3A] Diagram showing examples of zones defined by gates [Figure 3B] Diagram showing examples of zones defined by gates [Figure 4] Flowchart showing an overview of the two-stage face recognition process [Diagram 5] A flowchart showing a first example of a flow of a two-stage face recognition process. [Figure 6] A flowchart showing a second example of the flow of a two-stage face recognition process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant description will be omitted.

[0013] (One embodiment) <Knowledge that led to this disclosure> For gates installed at facilities such as stations and airports to manage entrance and exit to the facility, the use of passage management to accurately manage whether or not a person has passed through the gate is being considered. If passage management is insufficient, for example, a person who entered through the entrance of the gate but turned back to the entrance of the gate instead of going to the exit of the gate may be erroneously determined to have passed through, or a person who actually passed through may be erroneously recognized as not having passed through. For example, such errors may lead to erroneous billing in a service that charges people who pass through a gate, such as a ticket barrier at a station.

[0014] To implement passage management, for example, an authentication process for authenticating a person attempting to pass through (which may include, for example, a process for determining that authentication is not possible) and a tracking process for recording the history of the person's movements are implemented. It is desirable to implement these processes early in order to secure time for, for example, recording the passage of a person or performing processes such as regulating the movement of a person, such as opening and closing a door.

[0015] For example, a camera may be installed above the person and the gate (e.g., on the ceiling), photograph the person and the gate with the camera on the ceiling, and the captured image may be analyzed to track the person.

[0016] However, when installing a camera on the ceiling, the location where the camera can be installed may be limited depending on the structure or environment of the installation location. Even if installation is possible, if large-scale construction work is required for installation, the installation cost will increase. For this reason, it may be difficult to introduce a passage management system using a camera installed on the ceiling.

[0017] For example, it is conceivable to provide a gate with an arch-shaped or pole-shaped support that extends above the gate and to install a camera on the support, but a gate with such a support is larger in height than a gate without a support, so the installation location of the gate may be limited. Also, from a design perspective, providing a support on a gate may not be desirable.

[0018] In this embodiment, a passage management process including authentication and tracking processes is implemented by performing tracking processes using an image captured by a camera used for face authentication processes of people attempting to pass through the gate. By using the camera used for authentication processes for tracking processes as well, there is no need to provide a separate device for tracking processes (e.g., a camera on the ceiling, etc.). This makes it possible to suppress increases in the introduction costs of passage management. Also, compared to the case of providing equipment such as a dedicated camera for tracking processes, restrictions on installation locations are relaxed, improving the freedom of installation locations and making it easier to introduce a passage management system.

[0019] <Gate configuration example> Fig. 1 is a diagram showing an example of a gate 10 according to the present embodiment. Fig. 1 is a diagram showing the gate 10 as viewed from above, and illustrates a state in which a person h enters the gate 10 through an entrance / exit E1 and exits the gate 10 through an entrance / exit E2. Note that, with respect to the gate 10 shown in Fig. 1, a person may enter through the entrance / exit E2 and exit through the entrance / exit E1. In other words, the gate 10 allows people to pass in both directions.

[0020] Gate 10 has, for example, side walls V facing each other, and a passage L is formed between the side walls V to guide people passing through gate 10. At the top of one of the side walls V, which is about 1 m high, cameras 11 are provided, for example, at two positions closer to each of entrances E1 and E2 than the center of the side wall V, so that a total of four cameras 11 (11-R1, 11-R2, 11-L1, 11-L2) are provided on the two side walls V.

[0021] Cameras 11-R1 and 11-L1 are installed, for example, on side wall V at a position closer to entrance / exit E2 than the center of gate 10, and are used to photograph people entering gate 10 from entrance / exit E1 on the opposite side and passing through entrance / exit E2.

[0022] On the other hand, cameras 11-R2 and 11-L2 are installed, for example, on side wall V at a position closer to entrance / exit E1 than the center of gate 10, and are used to photograph people entering gate 10 from entrance / exit E2 on the opposite side and passing through to entrance / exit E1.

[0023] For example, the camera 11-R1 is installed at a position where it can capture an image of a person entering through the entrance E1 from the right front of the person. The camera 11-L1 is installed at a position where it can capture an image of a person entering through the entrance E1 from the left front of the person.

[0024] For example, the camera 11-R2 is installed at a position where it can photograph a person from the right front of the person entering from the entrance E2 opposite the entrance E1. The camera 11-L2 is installed at a position where it can photograph a person from the left front of the person entering from the entrance E2 opposite the entrance E1.

[0025] Therefore, a person entering gate 10 through entrance / exit E1 and passing through entrance / exit E2 is photographed from two directions (e.g., left and right) by two cameras 11-R1 and 11-L1 installed at positions separated from each other on the upper parts of two side walls V, separated by an aisle L.

[0026] On the other hand, a person passing through gate 10 in the opposite direction, i.e., a person entering gate 10 from entrance / exit E2 and passing through gate 10 to entrance / exit E1, is photographed from two directions (e.g., left and right) by two cameras 11-R2 and 11-L2 installed at positions separated from each other on the upper parts of two side walls V, separated by the aisle L.

[0027] 1 shows an example of a configuration in which a person can enter from both entrance E1 and entrance E2 of gate 10, but the present disclosure is not limited thereto. For example, gate 10 may be configured such that a person can enter from one entrance (e.g., entrance E1) but cannot enter from the other entrance (e.g., entrance E2). If gate 10 is configured not to allow a person to enter from entrance E2, camera 11-R2 and camera 11-L2 may not be provided. If gate 10 is configured not to allow a person to enter from entrance E1, camera 11-R1 and camera 11-L1 may not be provided.

[0028] 1 shows an example in which a person entering through the entrance of gate 10 is photographed from two directions by two cameras 11, but the present disclosure is not limited to this. For example, a person entering through the entrance of gate 10 may be photographed from one direction (the direction from the exit to the entrance of gate 10) by one camera 11. Alternatively, when a person entering through the entrance of gate 10 is photographed from two directions by two cameras 11, one of the images photographed by the two cameras 11 may be selected, and authentication processing and tracking processing may be performed based on the selected image.

[0029] In the following, as an example, a case will be described in which the passage management of a person who enters from entrance E1 and passes through entrance E2 at gate 10 shown in Fig. 1 is performed using an image captured by camera 11-R1 or camera 11-L1. Note that camera 11-R1 and camera 11-L1 may be collectively referred to as camera 11 for convenience.

[0030] Moreover, a person entering the gate 10 corresponds to a person who is a target of processing including face recognition processing. Hereinafter, a person who is a target of processing will be referred to as a "target person."

[0031] 1 is an example, and the present disclosure is not limited thereto. For example, the gate 10 may be provided with five or more cameras 11, or three or less cameras 11. By varying the shooting direction and / or angle of the cameras 11, it is possible to capture a wider range of a person's face.

[0032] Note that the multiple cameras 11 do not have to be identical to each other. For example, the multiple cameras 11 may be configured to capture images with different resolutions, angles of view, and image quality. Furthermore, the installation position and / or the capture direction of the camera 11 may be fixed or adjustable.

[0033] <System configuration> Fig. 2A is a diagram showing an example of a conceptual configuration of a passage management system according to the present embodiment. Fig. 2B is a block diagram showing an example of the configuration of a passage management system according to the present embodiment. The passage management system 1 according to the present embodiment is a system that manages the passage of people through a gate 10 (e.g., an entrance gate, a ticket gate, etc.) installed at the entrance / exit of a facility such as an airport, a station, or an event venue.

[0034] In the passage management system 1 according to the present embodiment, for example, management of entrance and exit of users who use the facility is performed by facial recognition. For example, when a user passes through gate 10 to enter the facility, facial recognition determines whether the user is a person permitted to enter the facility. Also, when a user passes through the gate to exit the facility, facial recognition determines which user is exiting the facility. Note that "facial recognition" may be considered as a concept included in "matching using a facial image."

[0035] 1, the passage management system 1 includes, for example, a gate 10, a camera 11 (right camera 11 or left camera 11), a face authentication function unit 12, a passage management function unit 13, a face authentication server 14, and a passage history management server 15. Note that in the passage management system 1, the number of gates 10 may be one or more.

[0036] Gate 10 is installed in facilities such as airports, stations, and event venues. Users authorized to use the facility pass through gate 10 when entering and / or leaving the facility. Gate 10 also performs control to prevent the passage of people who are not authorized to enter the facility.

[0037] 1, camera 11 is provided, for example, on a side wall V of gate 10. Camera 11 captures an image of a person passing through gate 10 and, if there is a person about to pass through gate 10, a shooting range including the face of the person. For example, the shooting range of camera 11 is a range in which the front view of a person's face can be captured.

[0038] The face authentication function unit 12 performs face authentication processing on the image. In the present embodiment, for example, the face authentication function unit 12 performs at least two stages of face authentication processing.

[0039] For example, the face authentication function unit 12 includes a camera control unit 121, a first face matching processing unit 122, and a second face matching processing unit 123. The first face matching processing unit 122 performs a first stage of face authentication processing, and the second face matching processing unit 123 performs second and subsequent stages of face authentication processing.

[0040] The camera control unit 121 controls, for example, the timing of shooting of the camera 11. For example, the camera 11 shoots at a speed of about 5 fps under the control of the camera control unit 121. Also, the right camera 11 and the left camera 11 may shoot simultaneously or with a difference in shooting timing within an allowable range under the control of the camera control unit 121, and the shooting timings of the right camera 11 and the left camera 11 may be synchronized with each other. Alternatively, the right camera 11 and the left camera 11 may shoot asynchronously at shooting timings independent of each other under the control of the camera control unit 121, for example.

[0041] The camera control unit 121 detects a face frame from an image captured by the camera 11, for example. The method of detecting the face frame is not particularly limited, but may be, for example, a method of detecting features (eyes, nose, and mouth) included in a face from an image, and detecting a boundary between the face area and an area outside the face based on information on the position and color of the detected features, thereby detecting a frame (face frame) surrounding the face area. For example, when the face frame is detected, the camera control unit 121 outputs information on the detected face frame (face frame information) and the captured image to the first face matching processing unit 122 and / or the second face matching processing unit 123. For example, the camera control unit 121 determines the output destination to either the first face matching processing unit 122 or the second face matching processing unit 123 depending on the positional relationship between the gate 10 and the person. In the following, the description of "face frame" may refer to a frame surrounding the face area, or may refer to the face area surrounded by the frame.

[0042] The first face matching processing unit 122 cuts out a face area included in the image based on, for example, face frame information, and notifies a face authentication request including information on the cut-out face area to the face authentication server 14. The information on the face area may be, for example, an image of the face area, or information indicating feature points extracted from the image of the face area.

[0043] In the face authentication server 14, for example, face images of persons who are permitted to pass through the gate 10 are registered. The face images registered in the face authentication server 14 may be referred to as registered face images. The registered face images may be associated with information capable of uniquely identifying or specifying a person, such as an ID of the registered person. The registered face images may also be information indicating feature points extracted from an image, for example.

[0044] For example, when the face authentication server 14 receives a face authentication request from the first face matching processing unit 122, it determines (matches) whether or not a face of a person identical to the face in the face area included in the face authentication request is included in the registered face image. The face authentication server 14 notifies the first face matching processing unit 122 of a face matching result including the determination result, for example. Note that the face matching result may include, for example, information indicating whether or not a face of a person identical to the face in the face area is included in the registered face image (for example, a flag indicating "OK" or "NG"), and information (for example, an ID) of a person associated with the registered face image when a face of a person identical to the face in the face area is included in the registered face image.

[0045] Matching involves, for example, comparing a registered face image with a face image of a person passing through gate 10 to determine whether a pre-registered registered face image matches the face image of the person passing through gate 10, or whether a pre-registered registered face image and a face image of the person passing through gate 10 are the face images of the same person.

[0046] On the other hand, authentication means proving to an outside party (e.g., gate 10) that a person whose facial image matches a previously registered facial image is the person in question (in other words, that the person is someone who should be allowed to pass through gate 10).

[0047] However, in this disclosure, "verification" and "authentication" may be used interchangeably.

[0048] For example, the matching process is a process of identifying who the face in the image data is by comparing the feature points of a preregistered face image with the feature points extracted from the detected face area. For example, a machine learning method may be used for this matching process. Also, the matching process may be performed, for example, in the face authentication server 14, but may also be performed in another device such as the gate 10, or may be distributed among multiple devices.

[0049] The first face matching processing unit 122 outputs, for example, information including the matching process result to the second face matching processing unit 123 and the passage management function unit 13. The matching process result may include, for example, information on the registered face image and a matching score. In addition, the information output from the first face matching processing unit 122 may include, for example, face frame detection information, the shooting time of the face camera image in which the face frame was detected, and information on the face area cut out by the first face matching processing unit 122.

[0050] It should be noted that the output destination of the first face matching processing unit 122 is not limited to the second face matching processing unit 123 and the passage management function unit 13. For example, the first face matching processing unit 122 outputs information including the matching processing result to a display device that presents image information to a person who is about to pass through the gate 10 and / or an audio output device that presents audio information to the person.

[0051] The second face matching processing unit 123 cuts out a face region included in the image based on, for example, face frame information, and performs face matching using information on the cut-out face region.

[0052] For example, the second face matching processing unit 123 may compare information of the face area cut out in the second face matching processing unit 123 with information of the face area cut out in the first face matching processing unit 122, and determine whether the person corresponding to the face area cut out in the second face matching processing unit 123 is the same as the person corresponding to the face area cut out in the first face matching processing unit 122.

[0053] Alternatively, the second face matching processing unit 123 may compare information about the face area cut out in the second face matching processing unit 123 with information about the registered face image included in the matching processing result, and determine whether the person corresponding to the face area cut out in the second face matching processing unit 123 is the same as the person corresponding to the registered face image included in the matching processing result.

[0054] The second face matching processing unit 123 outputs information including a determination result as to whether or not the people corresponding to the two face areas are the same to the passage management function unit 13. Note that the output destination of the second face matching processing unit 123 is not limited to the passage management function unit 13. For example, the second face matching processing unit 123 outputs information including the determination result to a display device that presents image information to a person attempting to pass through the gate 10 and / or an audio output device that presents audio information to the person.

[0055] The passage management function unit 13 manages the passage of a person located around the gate 10 based on, for example, information output from the face authentication function unit 12. The person located around the gate 10 includes, for example, a person passing through the gate 10, a person who is about to pass through, and a person passing around the gate 10. Here, the person who is about to pass through the gate 10 is not limited to a person who is permitted to pass through the gate 10 (for example, a person whose face image has been registered in the face authentication server 14), but may be, for example, a person whose face image is not registered in the face authentication server 14 but is trying to pass through. Furthermore, a person passing around the gate 10 is, for example, a person who is not about to pass through the gate 10 but is passing through the shooting range of the camera 11, or a person who is not about to pass through the gate 10 but is entering the shooting range. Furthermore, the state of a person may be, for example, a state related to the movement of a person, such as whether the person is moving or stationary, and the direction of movement if the person is moving.

[0056] The passage management function unit 13 includes, for example, a passage management state transition processing unit 131, a history management unit 132, and a history database (DB) 133.

[0057] For example, in the person passage management process, the passage management state transition processing unit 131 transmits to the gate 10 control information regarding the control of the gate 10 when a person permitted to pass through the gate 10 passes through the gate 10. Also, the passage management state transition processing unit 131 transmits to the gate 10 control information regarding the control of the gate 10 when a person not permitted to pass through the gate 10 attempts to pass through the gate 10.

[0058] The history management unit 132, for example, stores and manages information (passage history information) indicating the history of a person who passed through the gate 10. Furthermore, the history management unit 132, for example, stores the passage history information in a history DB 133 and transmits the passage history information to the passage history management server 15. For example, in a railway network, the history management unit 132 may manage local passage history information for each station (or each ticket gate).

[0059] The passage history management server 15, for example, stores and manages information (passage history information) indicating the history of a person who has passed through the gate 10. For example, the passage history management server 15 may manage the passage history information of multiple gates 10. For example, in a large facility with multiple entrances and exits, the passage history information of the gates 10 provided at each of the multiple entrances and exits may be aggregated and managed by the passage history management server 15. Also, for example, in a railway network, the passage history information of each of the gates 10 provided at the ticket gates of multiple stations may be aggregated and managed by the passage history management server 15.

[0060] The passage management function unit 13 may output, for example, information related to passage management (passage management information) to a display device not shown. The passage management information may include, for example, information output from the face authentication function unit 12. The display device may display, for example, the state of a person (for example, the result of face authentication of the person and the direction of movement). For example, the display device may display an image and superimpose a detected face frame on the image. The display device may also display, for example, information related to the person obtained by face authentication (the person's ID) by superimposing it on the image.

[0061] The face authentication function unit 12 described above may operate, for example, synchronously with or asynchronously with the passage management function unit 13. In the case of asynchronous operation, for example, the face authentication function unit 12 may operate when a face frame is detected in the camera control unit 121.

[0062] At least a part of the functions of the face authentication function unit 12 and the passage management function unit 13 described above may be replaced with each other. For example, a part of the functions of the passage management function unit 13 may be included in the face authentication function unit 12. For example, the face authentication function unit 12 may include an output unit that outputs a message indicating that the person to be determined has passed through the gate 10 when it is determined that the persons corresponding to the two face areas are the same. Information output from the output unit may include control information regarding the control of the gate 10 when a person permitted to pass through the gate 10 passes through, for example, control information regarding the control of the gate 10 when a person not permitted to pass through the gate 10 attempts to pass through the gate 10.

[0063] The configuration of the face authentication function unit 12 and the passage management function unit 13 described above may each have the form of a single information processing device (e.g., a server device), or the two may be included in a single information processing device. For example, the face authentication function unit 12 may have the form of a single information processing device, and the passage management function unit 13 may be included in a single information processing device. Furthermore, the face authentication function unit 12 and / or the passage management function unit 13 having the form of an information processing device may be included in the gate 10.

[0064] The information processing device described above may include a processor, a memory, and an input / output interface used for transmitting various types of information. The processor is an arithmetic device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory is a storage device realized using a RAM (Random Access Memory) or a ROM (Read Only Memory). The processor, the memory, and the input / output interface are connected to a bus, and various types of information are exchanged via the bus. The processor reads out, for example, programs and data stored in the ROM onto the RAM and executes processing, thereby realizing the functions of the configuration included in the information processing device.

[0065] In the above-mentioned face recognition function unit 12, the timing of face recognition processing may be determined depending on where a person is located in a zone defined around the gate 10. An example of the zone defined around the gate 10 will be described below.

[0066] <Gate area management> 3A and 3B are diagrams showing examples of zones defined for the gate 10. In Fig. 3A and Fig. 3B, examples of multiple zones are shown when the gate 10 is viewed from above. Note that Fig. 3A and Fig. 3B show an example in which the side wall V of the gate 10 that forms the passage L extends in the vertical direction of the paper.

[0067] As shown in FIG. 1, of the entrances E1 and E2 of the gate 10, for example, the upstream side along a specific approach direction (eg, entry direction) corresponds to the entrance, and the downstream side corresponds to the exit.

[0068] 3A shows an example of a zone defined when a person enters from entrance E2 at gate 10 where people can enter from both entrances E1 and E2, and FIG 3B shows an example of a zone defined when a person enters from entrance E1.

[0069] When the gate 10 allows entry in both directions, it is conceivable that the movement direction may differ depending on the entrance / exit through which the person enters, whether it is normal or not. For example, the movement direction of a person from entrance / exit E1 to entrance / exit E2 is normal for a person who entered through entrance / exit E1, but is abnormal for a person who entered through entrance / exit E2. In response to such differences in regulations, the passage management function, for example, defines entrance / exit E1 as the "north side" and entrance / exit E2 as the "south side."

[0070] The expressions "north side" and "south side" are merely examples, and the present disclosure is not limited to these expressions. For example, the expressions "north side" and "south side" do not limit the arrangement of the gate 10 to an arrangement along the geographic north-south direction. For example, even if the passage L of the gate 10 is located along a direction different from the north-south direction, or if the passage includes a curve, one side may be specified as the "north side" and the other side may be specified as the "south side".

[0071] For example, Fig. 3A shows an example of zones defined when a person enters through entrance / exit E2. In Fig. 3A, "Zone outside-S" (outside south zone area), "Zone A", "Zone B", and "Zone C" are defined for gate 10.

[0072] In contrast, Fig. 3B shows an example of zones defined when a person enters through entrance / exit E1. In Fig. 3B, "Zone outside-N" (northern outside zone area), "Zone A", "Zone B", and "Zone C" are defined for gate 10.

[0073] Below, each zone will be described with reference to the example of Fig. 3A. The example of Fig. 3B is the same as Fig. 3A, except that a person enters from the entrance E1 and "Zone outside-S" (area outside the south zone) is replaced with "Zone outside-N" (area outside the north zone).

[0074] The boundary between the southern outer zone area and Zone A may be referred to as, for example, the "face recognition start line."

[0075] The "face recognition start line" is used, for example, to determine whether or not to start face recognition processing (face matching processing) in the first face matching processing unit 122. For example, when a person crosses the "face recognition start line" and enters the gate 10, the face recognition processing is started. For example, the first face matching processing unit 122 issues a face recognition request from face frame information, links the matching result (face recognition ID) with person detection information, and starts tracking the person. The "face recognition start line" may also be referred to as the "A LINE."

[0076] The "face authentication start line" may be provided outside the gate 10 (for example, upstream along the path of the gate 10). The "face authentication start line" is not limited to one line segment, but may have multiple line segments, such as a U-shape. The shape having multiple line segments is not limited to a shape corresponding to a part of a rectangular shape such as a U-shape, but may be a shape corresponding to a part of a side of another polygonal shape. Alternatively, the "face authentication start line" may have an arc, or may have a shape in which straight lines and curves are mixed. For example, by having the "face authentication start line" have multiple line segments and / or arcs, face authentication processing can be started not only when a person enters from the front of the gate 10, but also when a person enters from a direction deviated from the front, such as from the side.

[0077] The boundary between Zone A and Zone B may be referred to, for example, as the "door closing limit line."

[0078] The "door closing limit line" indicates, for example, the position at which the exit gate door will close in response to a door closing command in time for a person to pass through. The "door closing limit line" is determined, for example, taking into consideration the maximum speed (e.g., 6 km / h, hereinafter referred to as the "maximum possible passing speed") assumed for a person to pass through the gate 10 and the time required to physically close the gate door (e.g., 0.5 seconds). For example, the "door closing limit line" is set a distance before the physical gate door position ("gate door position") that corresponds to the maximum passing speed times the time required to physically close the gate door. In this way, when a person who is not permitted to pass through the gate 10 passes through the "door closing limit line" and moves at the maximum possible passing speed, the exit gate door is closed before the person passes through the exit gate door.

[0079] The "closed door limit line" may also be referred to as the "unauthorized intrusion detection line" or the "B line."

[0080] The boundary between Zone B and Zone C may be referred to as the "exit line."

[0081] The "exit line" indicates, for example, the position at which it is determined that the person has exited the gate 10. The "exit line" may be provided outside the gate 10, similar to the above-mentioned "face recognition start line." The "exit line" is not limited to, for example, a single line segment, and may have multiple line segments, such as a U-shape. Alternatively, the "exit line" may have an arc. The "exit line" may be referred to as, for example, a "Z line."

[0082] In passage management, the physical gate door position ("gate door position") may simply be a passage point, in which case the physical gate door position may be different from or the same as the logically set "exit line." For example, in actual operation, the "gate door position" and the "exit line" may be set to be the same.

[0083] For example, in the case of a gate 10 that charges a fee to people passing through, the "exit line" may correspond to the "charge line."

[0084] For example, if a person who enters gate 10 crosses the charging line (e.g., if he or she enters zone C from zone B), the person is charged. In other words, if the person has not crossed the charging line (e.g., has not entered zone C), the person is not charged. By providing this charging line, it is possible to avoid the mistake of charging a person who enters gate 10 but turns back before crossing the charging line.

[0085] In the above description, an example has been given in which the "charging line" corresponds to the "Z line" ("exit line"); however, for example, the "charging line" may correspond to the "B line".

[0086] In the above example, three zones are defined, excluding the north side outside zone area and the south side outside zone area, but the present disclosure is not limited to this. The number, size, position, and shape of the zones may be changed depending on the situation to which the present disclosure is applied.

[0087] The face authentication function unit 12 in this embodiment executes a first-stage face authentication process when a person enters the above-mentioned zone A, and executes a second-stage face authentication process when the person enters zone B from zone A. By performing the authentication process in two stages, it is possible to confirm whether or not the person authenticated in the first stage has passed through the gate 10 by the second-stage face authentication process. The flow of the two-stage face authentication process will be described below.

[0088] Fig. 4 is a flowchart showing an outline of the flow of a two-stage face authentication process. The flow shown in Fig. 4 may be started, for example, every time a passerby approaches.

[0089] The face authentication processing unit 12 detects that a pedestrian approaches the gate 10 and enters a first authentication area (e.g., zone A) (S101). The first authentication area is not limited to zone A shown in Figs. 3A and 3B. The first authentication area may be, for example, an area upstream of the charge line of the gate 10, where a face frame of a size that allows face authentication processing can be obtained in a captured image. Whether or not a pedestrian has entered the first authentication area may be determined, for example, by a sensor (e.g., a photoelectric sensor) provided at the gate 10.

[0090] The face authentication function unit 12 executes a first face authentication process on a passerby who has entered the first authentication area (S102). For example, the face authentication function unit 12 transmits a face authentication request including information on a face frame in an image captured by the camera 11 to the face authentication server 14. When the face authentication server 14 receives the face authentication request, the face authentication server 14 determines whether or not the passerby is present among the registrants, for example, based on the information on the face frame. If the result of the determination indicates that the passerby is present among the registrants, the face authentication server 14 transmits, for example, information indicating the corresponding registrant to the face authentication function unit 12. On the other hand, if the passerby is not present among the registrants, the face authentication server 14 transmits information indicating the result of the determination to the face authentication function unit 12.

[0091] The face authentication function unit 12 determines whether or not the authentication has been successful as a result of the first face authentication process (S103).

[0092] If the authentication fails (NO in S103), the face authentication function unit 12 executes the process of S102 again. Here, a case where the authentication fails is, for example, a case where the passerby is not included in the registered persons held in the face authentication server 14. If the passerby is not included in the registered persons held in the face authentication server 14, the passerby may be treated as a person who is not permitted to pass through the gate 10. Alternatively, a case where the authentication fails may be a case where, even though the passerby is included in the registered persons, it is erroneously determined in the face authentication process that the passerby and the registered person are not the same person.

[0093] If the authentication is successful (YES in S103), the system waits until it detects that the pedestrian has entered the second authentication area (zone B) (S104). Note that the second authentication area is not limited to zone B shown in Figures 3A and 3B. For example, the second authentication area may be an area beyond a charging line set up at gate 10.

[0094] The face authentication processing unit 12 detects that the passerby has entered the second authentication area (S105). Whether or not the passerby has entered the second authentication area may be determined by a sensor such as a photoelectric sensor.

[0095] The face authentication function unit 12 executes a second face authentication process on a passerby who has entered the second authentication area (S106). For example, instead of requesting the face authentication server 14 to execute the second face authentication process, the face authentication function unit 12 may execute the second face authentication process locally at the gate 10 itself (or the face authentication function unit 12 itself).

[0096] The face authentication function unit 12 determines whether or not the authentication has been successful as a result of the second face authentication process (S107).

[0097] If the authentication fails (NO in S107), the face authentication function unit 12 determines, for example, that the pedestrian is not allowed to pass (S109). In this case, a process of closing the door of the gate 10 (door closing process) is executed. Note that, if the authentication fails (NO in S107), the first face authentication process similar to S103 may be performed again (re-authentication or re-matching) before determining that the pedestrian is not allowed to pass. Note that in this case, similar to S103, the face authentication function unit 12 transmits a face authentication request to the face authentication server 14, but the information included in the face authentication request may be information related to the face frame in the image in S107. Then, the flow shown in FIG. 4 ends.

[0098] If the authentication is successful (YES in S107), the face authentication function unit 12 determines that the pedestrian may be permitted to pass through the gate 10 (S108). In response to this determination, the face authentication function unit 12 may, for example, instruct the gate 10 to open the door that regulates passage through the gate 10. Then, the flow shown in FIG. 4 ends.

[0099] By performing the above-described two-stage face authentication process, it is possible to confirm whether or not a person who has been successfully authenticated in the first stage has passed through gate 10 by performing the second stage face authentication process.

[0100] In addition, when performing face recognition processing in two stages, there are several methods that differ in the method and / or timing of the second stage face recognition processing. For example, as shown below, there are a multiple local recognition method, a local face recognition tracking method, and a post-batch method.

[0101] In the multiple local authentication method, the information obtained from the face recognition server in the first stage of face recognition processing is registered in the local face recognition engine, and the registered information is used in the second stage and subsequent face recognition processing.

[0102] For example, in the multiple local authentication method, when receiving a face matching result from the face recognition server 14, the face matching result including the information of the facial feature amount of the candidate is received. Then, the information of the facial feature amount of the candidate is registered in the local face recognition engine. In the face recognition process after the information is registered in the local face recognition engine, the degree of match indicating whether the passerby and the candidate are the same person is confirmed by the image of the passerby and the registered information. Note that, in the multiple local authentication method, the information of the facial feature amount of the candidate does not necessarily need to be included in the face matching result. The face recognition function unit 12 may receive the information of the facial feature amount of the candidate as information separate from the face matching result, or may receive the information at a timing different from the face matching result. In other words, in the face recognition server 14, if the matching between the image information of the first face image area and the information of the registrant is successful, the data structure and / or timing of the received information are not limited as long as the face recognition function unit 12 can receive the facial feature amount of the candidate corresponding to the registrant.

[0103] In the multiple local authentication method, face recognition processing is performed multiple times, which can improve the legitimacy that the passerby is a person permitted to pass through gate 10. Here, in the multiple local authentication method, face recognition processing from the second stage onwards is performed using features registered in face recognition server 14, so that the face recognition processing from the second stage onwards is also performed with the same accuracy as face recognition processing using face recognition server 14. Therefore, it can be more reliably confirmed that the passerby is a person permitted to pass through gate 10.

[0104] In the local facial recognition tracking method, information about passersby contained in the image used in the first facial recognition process is registered in the local facial recognition engine, and the registered information is used in the second and subsequent stages of facial recognition processing.

[0105] For example, in the local face recognition tracking method, the face of a passerby is registered in the local face recognition engine at the timing of sending a face recognition request to the face recognition server 14. Here, the face registration of a passerby may correspond to extracting the face area of ​​the passerby from an image of the passerby and registering information on the image of the extracted area (i.e., face image), or may correspond to registering features obtained from the extracted face image. In the face recognition process after registering the information in the local face recognition engine, face matching (1:1 face matching) is performed between the image of the passerby and the registered face image to confirm that the photographed passerby and the registered passerby are the same person. Then, if the size of the face frame and the photographed area in the image of the passerby are equal to or larger than the designated size and fall within the designated area, and the matching result from the face recognition server 14 is OK, the passerby is allowed to pass.

[0106] In the local face recognition tracking method, a passerby who has been successfully authenticated as a person permitted to pass through the gate 10 in the first face recognition process can be tracked as entering and passing through the gate 10 in the second and subsequent authentication processes, so that passing through the gate 10 can be guaranteed. In addition, in the local face recognition tracking method, the feature amount used in the first face recognition process is used in the second and subsequent authentication processes, so that authentication reflecting the most recent environment (e.g., lighting, sunlight) can be performed in the second and subsequent authentication processes. This allows appropriate passage management even when there is a discrepancy between the face image registered in the face recognition server 14 and the latest face image (e.g., face area in the image captured by the camera 11) due to the influence of shadows near the gate 10 and aging.

[0107] In the post-batch method, similarly to the above-mentioned local face recognition tracking method, information about passersby contained in the image used in the first face recognition process is registered in the local face recognition engine, and the registered information is used in the second and subsequent stages of face recognition process. However, unlike the local face recognition tracking method, in the post-batch method, the second and subsequent stages of face recognition process are not executed in real time. For example, the second and subsequent stages of face recognition process are executed all at once at a certain specific timing. The certain specific timing may be a time period when there are few people passing through the gate 10 (for example, at night), or the end of the month, etc.

[0108] The post-batch method does not pursue real-time performance, and therefore can reduce waste of computer resources (eg, CPU processing power) related to face recognition processing in control devices, servers, etc.

[0109] The two-step authentication method is not limited to the above example. In the above example, the second and subsequent steps of authentication are performed by a local face recognition engine instead of sending a face recognition request to the face recognition server 14, but the present disclosure is not limited to this. For example, in the second step, a face recognition request may be sent to the face recognition server, and the face recognition server 14 may determine whether or not a candidate who is the same person as the passerby is present.

[0110] Note that, in the two-step authentication method, FIG. 4 shows an example in which the timing of authentication is determined by detecting, by a photoelectric sensor, that a passerby has entered a specific area, but the present disclosure is not limited to this.

[0111] For example, camera 11 captures an image of a person entering through the entrance of gate 10 from the direction from the exit of gate 10 toward the entrance. Therefore, the face frame of a person present in an area where a second stage of face authentication is performed (for example, zone B in FIG. 3A ) has a larger size than the face frame of a person present in an area where a first stage of face authentication is performed (for example, zone A in FIG. 3A ).

[0112] By utilizing the fact that the size of the face frame differs between the area where the first stage face authentication is performed and the area where the second stage face authentication is performed, the timing of authentication may be specified based on the size of the face frame instead of the photoelectric sensor. An example of a two-stage face authentication process based on the size of the face frame will be described below.

[0113] 5 is a flowchart showing a first example of the flow of a two-stage face recognition process. The flow shown in FIG 5 may be started, for example, every time a passerby approaches.

[0114] The face recognition function unit 12 acquires an image and detects a face frame (S201).

[0115] The face authentication function unit 12 judges whether the size of the face frame is equal to or larger than a predetermined first threshold (S202). The size of the face frame and the first threshold to be compared with the size of the face frame may be determined by the area in the image region, or may be determined by the vertical or horizontal length of the face frame. The first threshold may be determined based on, for example, the lower or upper limit of the size of the face frame used in the face authentication process in S203 described later.

[0116] If the size of the face frame is not equal to or larger than the first threshold (NO in S202), the face recognition function unit 12 executes the process of S201.

[0117] If the size of the face frame is equal to or larger than the first threshold (YES in S202), the face recognition function unit 12 executes a first face recognition process for the detected face frame (S203). In the first face recognition process, a face recognition request is sent to the face recognition server 14, and the face recognition server 14 determines whether or not there is a candidate who is the same person as the passerby among the registered persons.

[0118] As a result of the first face authentication process, the face authentication function unit 12 judges whether the authentication has been successful or not (S204). Whether the authentication has been successful or not may be the same as in S104 of FIG.

[0119] If the authentication fails (NO in S204), the face authentication function unit 12 executes the process of S201.

[0120] If the authentication is successful (YES in S204), the face authentication function unit 12 acquires an image at the next timing and detects a face frame (S205).

[0121] The face authentication function unit 12 judges whether a timeout has occurred (S206). For example, in the process of S206, a timer is started at the timing when authentication is successful in the first face authentication process in S203, and if the time indicated by the timer is equal to or longer than a predetermined time, it is judged that a timeout has occurred. Here, the time indicated by the timer is the time from the timing when authentication is successful in the first face authentication process in S203 to the timing when the judgment of S206 is executed. Also, for example, the predetermined time may be an upper limit time allowed for a passerby who has been successfully authenticated in the first face authentication process to pass through the gate 10. This judgment of timeout allows an appropriate process (for example, a warning to the passerby) to be executed in a situation where, for example, a passerby who has been successfully authenticated in the first stage remains in the passage of the gate 10.

[0122] If a timeout occurs (YES in S206), the flow ends.

[0123] If a timeout has not occurred (NO in S206), the face authentication function unit 12 determines whether the size of the face frame is equal to or larger than a predetermined second threshold (S207). For example, the second threshold is larger than the first threshold. For example, the second threshold may be the lower limit or upper limit of the size of the face frame used in the face authentication process in S208 described below. Alternatively, the second threshold may be defined by the size of the face frame of an image of a person having an average face size who has reached the charging line. The second threshold may also be defined by a value obtained by multiplying the size of the face frame to be processed in S203 by a coefficient greater than 1.

[0124] If the size of the face frame is not equal to or larger than the second threshold (NO in S207), the face recognition function unit 12 executes the process of S205.

[0125] If the size of the face frame is equal to or larger than the second threshold (YES in S207), the face recognition function unit 12 executes 1:1 face recognition processing, which is the second face recognition processing, for the face frame detected in S205 (S208).

[0126] For example, in the second face recognition process of the multiple local authentication method, the facial feature information of the candidate acquired in S203 is compared with the facial frame information of the passerby detected in S205 to determine whether the candidate and the passerby are the same person.

[0127] Also, for example, in the second face recognition process of the local face recognition tracking method, the face frame information used in the first face recognition process in S203 is compared with the face frame information of the passerby detected in S205 to determine whether the candidate and the passerby are the same person.

[0128] As a result of the second face authentication process, the face authentication function unit 12 determines whether or not the authentication has been successful (S209).

[0129] If the authentication fails (NO in S209), the face authentication function unit 12 treats the pedestrian as not allowed to pass (S211). In this case, a process of closing the door of the gate 10 (door closing process) is executed. Note that if the authentication fails (NO in S209), the first face authentication process similar to S203 may be performed again (re-authentication or re-matching) before treating the pedestrian as not allowed to pass. Note that in this case, similar to S203, the face authentication function unit 12 sends a face authentication request to the face authentication server 14, but the face authentication request may include information regarding the face frame in the image in S208. Then, the flow ends.

[0130] Here, the case where the authentication fails may be, for example, when the information of the candidate identified by the first face recognition process in S203 does not match the information of the image of the passerby. When the information does not match, for example, when the score obtained by the face recognition process is lower than the threshold value for determining that the two are the same person. When the information of the identified candidate does not match the information of the image of the passerby, it may be when the identified candidate and the passerby included in the image are not the same person (for example, when they are switched in the middle). Alternatively, when the information of the identified candidate does not match the information of the image of the passerby, it may be when the identified candidate and the passerby included in the image are the same person, but it is erroneously determined that the identified candidate and the passerby included in the image are not the same person.

[0131] If the authentication is successful (YES in S209), the face authentication function unit 12 may determine to permit the pedestrian to pass through the gate 10 (S210). In response to this determination, the face authentication function unit 12 may, for example, output to the gate 10 an instruction to open the door that regulates passage through the gate 10. Then, the flow ends.

[0132] In addition, although Fig. 5 shows an example in which the second-stage face recognition process is executed locally, the present disclosure is not limited to this. For example, as shown below with reference to Fig. 6, in the second-stage face recognition process, face recognition may be requested from the face recognition server 14. In this way, the function of performing face recognition can be omitted from the gate 10, and the configuration of the gate 10 can be simplified.

[0133] Fig. 6 is a flow chart showing a second example of the flow of two-stage face recognition processing. The flow shown in Fig. 6 may be started, for example, every time a passerby approaches. In Fig. 6, the same processes as those shown in Fig. 5 are denoted by the same reference numerals and the description thereof will be omitted. In Fig. 6, S208 in Fig. 5 is replaced with S301.

[0134] If the size of the face frame is equal to or larger than the second size (YES in S207), the face authentication function unit 12 executes the second face authentication process for the face frame detected in S205 (S301). Here, in S301, the face authentication function unit 12 transmits a face authentication request to the face authentication server 14. The face authentication request includes, for example, information on the face frame detected in S205. In addition, the face authentication server 14 may determine whether or not there is a candidate who is the same person as the passerby among the registrants. Alternatively, the face authentication server 14 may perform the process simply by using the result of the matching performed based on the face authentication request in S203. For example, the information on the registrants narrowed down by the result of the matching performed based on the face authentication request in S203 may be matched with the information on the face frame included in the newly received face authentication request.

[0135] As described above, in the passage management system 1 of this embodiment, a first face authentication process is executed based on a face frame in an image of a person present in a first authentication area, and for a person who has been successful in the first face authentication process, a second face authentication process is executed based on a face frame in an image of the person present in a second authentication area between the first authentication area and the exit of the gate. The two-stage face authentication process executed by this configuration makes it possible to realize passage management of targets (e.g., people) through the gate with a simple configuration.

[0136] In the above-described embodiment, an example of performing two-stage face recognition processing has been shown, but the present disclosure is not limited thereto, and three or more stages of face recognition processing may be performed. For example, for a person who has successfully completed the first stage of face recognition processing, face frame detection and face recognition processing may be performed on each image captured after the first stage of face recognition processing, and the person may be tracked. In this case, the second and subsequent stages of face recognition processing may be performed locally. With this configuration, the movement of the person can be tracked sequentially, and it can be accurately determined whether the person has passed through the gate 10. For example, in a configuration in which the person is tracked, when a sensor provided at the exit of the gate 10 (or the charge line of the gate 10) detects the passage of the person, the face recognition processing may be stopped and tracking may be terminated.

[0137] In addition, since the accuracy of face recognition (face matching) is guaranteed in the first stage face recognition process, the processing may be simplified in the second stage face recognition process. For example, the number of face features used in the second stage face recognition process may be reduced compared to the number of face features used in the first stage face recognition process. For example, if the features of each face part are used in the first stage face recognition process, the features of the eyes of the face may be used in the second stage face recognition process, and the features of parts other than the eyes may not be used. This reduces the processing load related to the comparison of features, and improves the processing speed of the second stage.

[0138] Also, for example, in face recognition processing, when a score calculated by comparing feature amounts of two faces (e.g., a face of a registered person and a face of a passerby) is compared with a threshold, and when the score is equal to or greater than the threshold, it is determined that the two faces are the faces of the same person, the threshold may be adjusted according to the stage of face recognition processing. The threshold used in the second stage of face recognition processing may be smaller than the threshold used in the first stage of face recognition processing. By adjusting the threshold, the processing load related to the score comparison can be reduced, and the processing speed of the second stage can be improved.

[0139] In the above-described embodiment, an example in which the door closing process is performed when authentication fails in the second-stage face authentication process has been described, but the present disclosure is not limited thereto. For example, when authentication fails in the second-stage face authentication process, the door closing process may not be performed depending on the installation status and / or congestion status of the gate 10. For example, in the case of a gate installed at a railway station, since a large number of people pass through the gate, stopping the flow of people by closing the door will cause congestion. In such a case, rather than detecting a person who passes through the gate illegally, it is possible to not perform the door closing process so as not to stop the flow of people.

[0140] In the above-described embodiment, whether or not a person has entered the first area and / or the second area is determined depending on whether or not the size of the face frame is equal to or larger than the first threshold and / or the second threshold, but the present invention is not limited to this. The rate of change in the size of the face frame obtained based on the size of the face frame may be used to determine whether or not a person has entered the first area and / or the second area. The rate of change in the size of the face frame represents the rate of change in the size of the face frame in an image that changes with the passage of time (or the movement of a person). For example, the rate of change in the size of the face frame at a certain time point t is calculated by comparing the size of the face frame acquired at time point t with the size of the face frame of the same person that is acquired one time point before (or multiple times before) the time point t. In general, the rate of change in the size of the face frame increases as the face approaches the camera, so the rate of change in the size of the face frame changes depending on the distance between the person and the camera. Furthermore, the rate of change in the size of the face frame indicates the rate of change (or the amount of change), and is therefore less susceptible to the influence of differences in the size of the face of each person. Therefore, by determining whether the rate of change in the size of the face frame is equal to or greater than a predetermined threshold, it is possible to determine the approach of a face (for example, which area a person has entered) while absorbing differences in face size between people.

[0141] In the above-described embodiment, the information of the face frame is used to determine whether or not the person has entered the first area and / or the second area, but the present invention is not limited to this. The distance between the facial features may be used to determine whether or not the person has entered the first area and / or the second area. Generally, the closer the face is to the camera, the greater the distance between the facial features in the captured image. Therefore, the approach of the face can be determined by determining whether the distance between the facial features, such as the distance between the eyes and the distance between the eyes and the mouth, is equal to or greater than a predetermined threshold. Examples of the facial features include facial organs such as the eyes, nose, mouth, and ears, but features other than general organs, such as age spots, moles, and wrinkles, may be used as facial parts. In addition, the size of the facial features in the captured image may be used instead of the distance between the parts. This is because, like the size of the face frame, the size of the facial features also increases as the face approaches the camera. In addition, the rate of change in the size of the facial features may be used instead of the distance between the parts. This is because, like the rate of change in the size of the face frame, the rate of change in the size of the facial features also increases as the face gets closer to the camera.

[0142] In addition, when determining which area a person attempting to pass through gate 10 has entered, the above-mentioned methods may be used in combination, may be used selectively, or the same method may be used all the time.

[0143] In the above-described embodiment, the success or failure of the first-stage face authentication process and the second-stage face authentication process may be output by sound and / or image. In this case, different sounds and / or images may be output depending on whether the first-stage face authentication process is successful or not and whether the second-stage face authentication process or later is successful or not. In this way, a person who is about to pass through the gate 10 can be made aware that two or more stages of authentication are being performed. In addition, if a person's face authentication process fails, it is possible to notify the person whether the failure occurred at the stage of obtaining permission to pass through the gate 10 (success or failure of the first-stage face authentication process) or at the stage of confirming passage through the gate 10 (success or failure of the second-stage face authentication process or later).

[0144] Also, for example, at a gate installed at a railway station that manages the entrance and exit of railway users to and from the station premises, no charge is made when entering the station premises, but a charge is made when exiting. Therefore, when a railway user passes through the gate to enter the station premises, the face authentication process may be simplified. For example, if authentication of an entering user fails, information on an image where authentication failed may be recorded, and when the user leaves the station, the location of the user's entry may be identified using information on the image recorded at the time of entry, and the user may be charged retroactively.

[0145] In the above-described embodiment, a system for managing the passage of a person passing through the gate 10 has been described, but the present disclosure is not limited thereto. For example, the present disclosure may be applied to a case where there is no side wall of the passageway and no restricting portion (e.g., a door) restricting the passageway of a person. For example, the present disclosure may be applied to a movement path from a certain zone to another zone where the person's entry is permitted according to authentication processing. In this case, a camera for capturing an image of a face of a person passing through the movement path may be installed, for example, on a support portion or the like provided on the movement path.

[0146] In addition, in the present embodiment, an example in which the authentication target is a person is shown, but the present disclosure is not limited thereto. For example, the present disclosure may be applied to moving objects such as animals and vehicles. In addition, in the present embodiment, an example in which face authentication is performed is shown, but the present disclosure is not limited thereto. For example, the present disclosure may be applied to authentication using an ID card indicating that the person has the right to pass through a gate, and other authentication methods such as biometric authentication.

[0147] In addition, face authentication may be used in combination with other authentication methods. Even if the face authentication does not permit passage as disclosed in the above-mentioned embodiment, passage may be permitted exceptionally if ID card information is input.

[0148] Furthermore, in the above-described embodiment, the camera 11 is not limited to being provided on the side wall V of the gate 10. For example, the camera 11 may be attached to a support provided on the gate 10. The support may be, for example, a pole extending vertically from the gate 10, or an arch-shaped member provided so as to cover the side wall of the gate 10.

[0149] In the above embodiment, a door is used as a means for restricting passage through gate 10, but passage may be restricted directly or indirectly by other means. For example, an alarm may be sounded or a warning light may be turned on to inform a person attempting to pass through gate 10 that passage through gate 10 has been restricted. Also, an employee near gate 10 may restrict passage by sending a notification to a terminal or the like owned by the employee.

[0150] Also, depending on the congestion situation, whether or not to perform control to prevent passage, or the means for restricting passage may be switched. For example, in an environment where the safety may be reduced by preventing or restricting the passage of people, such as when a large number of people enter or exit, the passage through the gate 10 may not be prevented, and information indicating that there has been an illegal passage may be recorded. In this case, the face image or face recognition result of the person who has illegally passed through may be recorded in association with the information indicating that there has been an illegal passage. This makes it possible to later track down the person who has illegally passed through and claim compensation for the right of passage, etc.

[0151] Furthermore, in the above-described embodiment, the passage management system 1 managed both entry to and exit from a facility at the entrances and exits of the facility, such as an airport, train station, or event venue, but it is also possible to manage either entry to or exit from a facility at the entrance or exit, and not manage the other.

[0152] The present disclosure can be realized in software, hardware, or software in conjunction with hardware.

[0153] Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by one LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of one chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, a system LSI, a super LSI, or an ultra LSI.

[0154] The method of integration is not limited to LSI, and may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, after LSI manufacturing, a programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used. The present disclosure may be realized as digital processing or analog processing.

[0155] Furthermore, if a new integrated circuit technology that can replace LSI appears due to the progress of semiconductor technology or a derivative technology, it is possible to integrate the functional blocks using that technology. The application of biotechnology is also a possibility.

[0156] The present disclosure may be implemented in any type of apparatus, device, or system having a communication function (collectively referred to as a communication apparatus). The communication apparatus may include a radio transceiver and processing / control circuitry. The radio transceiver may include a receiver and a transmitter, or both as functions. The radio transceiver (transmitter and receiver) may include a radio frequency (RF) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of communication devices include telephones (e.g., cell phones, smartphones, etc.), tablets, personal computers (PCs) (e.g., laptops, desktops, notebooks, etc.), cameras (e.g., digital still / video cameras), digital players (e.g., digital audio / video players), wearable devices (e.g., wearable cameras, smart watches, tracking devices, etc.), game consoles, digital book readers, telehealth and telemedicine devices, communication-enabled vehicles or mobile conveyances (e.g., cars, airplanes, boats, etc.), and combinations of the above devices.

[0157] Communications Equipment includes, but is not limited to portable or mobile equipment, non-portable or fixed equipment, devices and systems of any kind, such as smart home devices (appliances, lighting equipment, smart meters or metering devices, control panels, etc.), vending machines and any other "Things" that may exist on an Internet of Things (IoT) network.

[0158] In recent years, in the field of IoT (Internet of Things) technology, CPS (Cyber ​​Physical Systems) is a new concept that creates new added value by linking information between physical space and cyber space. This CPS concept can also be adopted in the above-mentioned embodiment.

[0159] That is, as a basic configuration of a CPS, for example, an edge server located in a physical space and a cloud server located in a cyberspace can be connected via a network, and processing can be distributed and processed by the processors installed in both servers. Here, each processing data generated in the edge server or the cloud server is preferably generated on a standardized platform, and the use of such a standardized platform can improve the efficiency of building a system including a variety of sensors and IoT application software.

[0160] Communications include data communications via cellular systems, wireless LAN systems, communications satellite systems, etc., as well as data communications via combinations of these.

[0161] A communications apparatus also includes devices, such as controllers and sensors, connected or coupled to a communications device that performs the communications functions described in this disclosure, such as controllers and sensors that generate control and data signals used by the communications device to perform the communications functions of the communications apparatus.

[0162] The communication apparatus also includes infrastructure facilities, such as base stations, access points, and any other apparatus, device, or system that communicates with or controls the various apparatuses listed above, but are not limited to these.

[0163] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the disclosure.

[0164] Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Industrial Applicability]

[0165] An embodiment of the present disclosure is suitable for a face recognition system. [Explanation of symbols]

[0166] 1. Currency Management System 10 Gates 11 Camera 12 Face recognition function unit 121 Camera control unit 122 First face matching processing unit 123 Second face matching processing unit 13 Passage management function section 131 Passing management state transition processing unit 132 History Management Department 133 History DB 14 Face Recognition Server 15 Passing history management server

Claims

1. an acquisition unit that acquires an image including a face image of a person entering a gate, the image being captured from a direction from an exit of the gate toward an entrance of the gate; performing a first face authentication process on a person who is determined to have entered a first area defined for the gate, based on a first face image captured in the first area; a processing unit that, when the first face authentication process is successful, executes a second face authentication process on a person determined to have entered a second area defined between the first area and an exit of the gate, based on a second face image captured in the second area; an output unit that outputs a message indicating that the person has passed through the gate when it is determined by the second face authentication process that the person who has succeeded in the first face authentication process and the person included in the second face image are the same person, and outputs different sounds or images depending on whether the first face authentication process is successful or the second face authentication process is successful; An information processing device comprising:

2. the processing unit determines that the person has entered the first area and the second area based on a size of an area of ​​the face image in the image; The information processing device according to claim 1 .

3. The processing unit determines that the person has entered the first area and the second area based on a detection result of the photoelectric sensor. The information processing device according to claim 1 .

4. The processing unit includes: In the first face authentication process, image information of the first face image is transmitted to a server having information of registered persons permitted to pass through the gate; receiving from the server a collation result obtained by comparing image information of the first face image with information of the registrant; determining whether the first face authentication process has been successful based on the matching result; If the server successfully matches the image information of the first face image with the information of the registrant, the server receives information of the candidate corresponding to the registrant from the server; In the second face authentication process, information of the candidate is compared with image information of the second face image. The information processing device according to claim 1 .

5. The processing unit compares image information of the first face image with image information of the second face image in the second face authentication process. The information processing device according to claim 1 .

6. the processing unit executes the second face authentication process using a number of facial features that is smaller than a number of facial features used in the first face authentication process. The information processing device according to claim 1 .

7. the processing unit uses a threshold value for face matching in the second face authentication processing that is smaller than a threshold value for comparing with the matching score of the face matching in the first face authentication processing, The information processing device according to claim 1 .

8. An acquisition unit that acquires an image including a face image of a user moving from a certain zone to another zone where the person is allowed to enter in accordance with an authentication process; performing a first face authentication process on a person who is determined to have entered a first area defined for the different zone, based on a first face image captured in the first area; a processing unit that, when the first face authentication process is successful, executes a second face authentication process on a person determined to have entered a second area defined between the first area and the other zone, based on a second face image captured in the second area; an output unit that outputs a message indicating that the person who has succeeded in the first face recognition process and the person included in the second face image are determined to be the same person by the second face recognition process, and outputs different sounds or images depending on whether the first face recognition process is successful or the second face recognition process is successful; An information processing device comprising:

9. An information processing device, Obtaining an image including a facial image of a person entering a gate, the image being captured from a direction from the exit of the gate toward the entrance of the gate; when it is determined that the person has entered a first area defined for the gate, a first face authentication process is executed based on a first face image of the person photographed in the first area; if the first face authentication process is successful, when it is determined that the person has entered a second area defined between the first area and an exit of the gate, a second face authentication process is executed based on a second face image captured in the second area; When it is determined by the second face recognition process that the person who has succeeded in the first face recognition process and the person included in the second face image are the same person, a message is output to the effect that the person has passed through the gate, and a different sound or image is output depending on whether the first face recognition process is successful or the second face recognition process is successful. Information processing methods.

Images