Information processing apparatus and information processing method

Abstract

To make it possible to appropriately determine whether a photographed image is an image obtained by photographing a person's actual face.SOLUTION: An information processing apparatus comprises: an acquisition unit that acquires from a camera a photographed image obtained by photographing a user attempting to enter through an entrance of a gate; and a processing unit that extracts image information from the photographed image and determines whether or not the camera has photographed the actual face of the user based on a change in the image information in response to a change in a positional relation between the entrance and the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] In stations, offices, and the like, there are specific areas where entry is restricted to people other than certain individuals. Gates are installed at the entrances and exits of such specific areas, and the entry of people who pass through the gates to enter the area and the exit of people who pass through the gates to leave the area are managed. For example, ticket gates are installed at the entrances and exits of stations, and the ticket gates manage the entry of people into the station premises depending on whether or not they have a ticket.

[0003] In recent years, facial recognition gates that use facial recognition technology to determine whether or not a person can pass through the gate have been under consideration.Technology has been considered for preventing so-called impersonation, in which a person who is not permitted to pass through the gate attempts to pass through the gate by using a facial image of a person who is permitted to pass through the gate (for example, Patent Document 1 and Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-219703 [Patent Document 2] Japanese Patent Publication No. 2022-168759 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in order to prevent impersonation, there is room for improvement in how to appropriately determine whether an image of a person attempting to pass through a facial recognition gate is an image of the person's actual face.

[0006] Non-limiting examples of the present disclosure contribute to providing an information processing device and an information processing method that can appropriately determine whether a captured image is an image of a person's actual face. [Means for solving the problem]

[0007] An information processing device according to one embodiment of the present disclosure includes an acquisition unit that acquires from the camera an image of a user attempting to enter through the entrance of a gate, and a processing unit that extracts image information from the captured image and determines whether the camera has captured an actual face of the user based on changes in the image information in response to changes in the positional relationship between the entrance and the user.

[0008] In an information processing method according to one embodiment of the present disclosure, an information processing device acquires from a camera an image of a user attempting to enter through a gate entrance, extracts image information from the captured image, and determines whether the camera has captured an actual face of the user based on changes in the image information in response to changes in the positional relationship between the entrance and the user.

[0009] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0010] A non-limiting example of the present disclosure can appropriately determine whether a captured image is an image of a person's actual face.

[0011] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of a face authentication ticket gate system according to an embodiment of the present disclosure. [Figure 2] Diagram showing an example of camera installation in a facial recognition ticket gate system [Figure 3] FIG. 10 is a diagram showing an example of distances between feature points calculated in inside detection; [Figure 4A] A diagram showing an example of a zone provided in a facial recognition ticket gate. [Figure 4B] FIG. 10 is a diagram showing an example of a correspondence relationship between the position and / or size of a face in a captured image and the zone in which the user is present. [Figure 5] A diagram showing an example of a change in the ratio of distances between feature points as a user moves. [Figure 6] A flowchart showing an example of the flow of an inside detection process. [Figure 7] FIG. 10 is a diagram showing an example of image processing used for outside detection. [Figure 8] FIG. 10 is a diagram showing an example of tilt correction. [Figure 9] 10 is a flowchart showing an example of the flow of an outside detection process. [Figure 10] FIG. 10 shows an example of a variation of the determination method. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0014] (One embodiment) <Knowledge that led to this disclosure> In recent years, facial recognition gates that use facial recognition technology to determine whether or not a person can pass through the gate have been considered. For example, the introduction of facial recognition gates as transportation ticket gates at the entrances and exits of train stations and at departure and arrival gates at airports is being considered.

[0015] At a facial recognition gate, authentication can be easily performed by having a person passing through the gate point their face at a camera installed at the gate that takes an image for facial recognition. However, there is a risk of spoofing, where a person who is not permitted to pass through the gate uses the facial image of a person who is permitted to pass through the gate to impersonate that person and attempt to pass through the facial recognition gate. For this reason, it is desirable to detect spoofing (hereinafter referred to as spoofing detection). In spoofing detection, it is desirable to determine whether the image taken by the camera is an image of the actual person's face.

[0016] On the other hand, transportation ticket gates must quickly process the passage of a large number of users. One known countermeasure against spoofing is to ask users to cooperate by changing their facial orientation or posture. However, this type of countermeasure that requires cooperation slows down the speed at which users pass through, making it unsuitable for transportation ticket gates. Therefore, what is desired is the ability to pass through the ticket gate without requiring users to be particularly conscious or cooperative, i.e., so-called walk-through performance.

[0017] Therefore, it is desirable to perform spoofing detection without impeding walkthrough performance.

[0018] As a gate user moves as they pass through the gate, the state of their face as seen by the camera changes. For example, the relative angle of the direction the camera is pointing (e.g., the direction of the camera's image) to a specific direction of the face (e.g., the direction directly in front of the gate) changes. On the other hand, in an image for impersonation (e.g., an image of the face of a person permitted to pass through the gate), it is expected that the state of the face as seen by the camera will change little. In a non-limiting example of the present disclosure, this difference in the state of the face (change in the angle of the face) is utilized to detect impersonation.

[0019] Furthermore, since a gate user moves as they pass through the gate, the state of the surroundings of the face (e.g., the background) as seen by the camera changes. On the other hand, in an image for impersonation (e.g., an image of the face of a person permitted to pass through the gate), it is expected that the state of the surroundings of the face as seen by the camera will not change much. In a non-limiting example of the present disclosure, impersonation detection is performed by utilizing this difference in the state of the surroundings of the face.

[0020] In the following description, spoofing detection using information about the inside of an image corresponding to the face area included in a captured image (hereinafter, referred to as a face image) is referred to as "inside detection." An example of inside detection is spoofing detection that uses differences in the state of the face in the image (changes in the angle of the face). Spoofing detection using information about the outside of the face image is referred to as "outside detection." An example of outside detection is spoofing detection that uses differences in the state around the face in the image. The range of the face image may be any area that includes at least the entire face, and various ranges are possible depending on the algorithm used to detect faces from captured images. For example, if a face frame that surrounds the area where the user's face is detected with a frame of a predetermined shape, such as a rectangle or a circle, is detected, the range of the face frame may be used as the range of the face image. Furthermore, if the outline of the user's face is detected, the range of the outline may be used as the range of the face image. The range of the face image may be different between "inside detection" and "outside detection." For example, "inside detection" may use information about the inside of the face frame, while "outside detection" may use information about the outside of the face outline.

[0021] The term "detect" may be substituted with terms such as "monitor," "estimate," "identify," and "determine."

[0022] <Example of system configuration> Fig. 1 is a diagram showing an example of the system configuration of a face recognition ticket gate system 1 according to this embodiment. Fig. 2 is a diagram showing an example of camera installation in the face recognition ticket gate system 1.

[0023] 1 shows a camera 10, a ticket gate control device 20, a server device 30, a face recognition ticket gate 40, and a registration terminal 50. The face recognition ticket gate 40 is connected to the ticket gate control device 20 by wire and / or wirelessly. The camera 10 is connected to the ticket gate control device 20 by wire and / or wirelessly. The server device 30 may be connected to the ticket gate control device 20 by wire and / or wirelessly, or may be connected via a network (e.g., the Internet).

[0024] As shown in Fig. 2, cameras 10 are provided in the facial recognition ticket gate 40. In the example of Fig. 2, four cameras 10, cameras 10-1 to 10-4, are provided in the facial recognition ticket gate 40. Cameras 10-1 and 10-2 capture images including the face of a user who is about to enter the facial recognition ticket gate 40 from the direction indicated by arrow K1 in Fig. 2. Cameras 10-3 and 10-4 capture images including the face of a user who is about to enter the facial recognition ticket gate 40 from the direction indicated by arrow K2 in Fig. 2.

[0025] It should be noted that the number of cameras 10 and / or the positions of the cameras 10 are not limited to the example shown in Fig. 2. For example, it is sufficient that at least one camera 10 is installed in the facial recognition ticket gate 40. For example, if the facial recognition ticket gate 40 is configured so that users enter from only one direction rather than from both directions, the number of cameras 10 does not need to be sufficient and / or the cameras 10 do not need to be installed in positions that allow them to capture images of users entering from both directions.

[0026] In the following, a case where a user attempts to enter the face authentication ticket gate 40 from only one direction, as indicated by the arrow K1 in FIG. 2, will be described as an example.

[0027] The camera 10 captures an image including the face of a user who is about to enter through the entrance of the face authentication ticket gate 40. The camera 10 transmits the captured image to the ticket gate control device 20.

[0028] An image captured by camera 10 is referred to as a "captured image." An image used by a person committing an act of impersonation is referred to as a "spoofed image." A spoofed image includes the face of a person permitted to pass through facial recognition ticket gate 40. Specifically, a spoofed image may be a life-size printed photograph of the face of a person permitted to pass through facial recognition ticket gate 40, or an image or video of a photograph or video of the face of a person permitted to pass through facial recognition ticket gate 40 displayed on a terminal such as a tablet.

[0029] When camera 10 captures a photo of a user who is impersonating someone else, the user will overlay an impersonation image over their own face to hide it, and the captured image will include the "impersonation image" captured by camera 10. Note that in this case, the captured image does not include the face of the user who is impersonating someone else.

[0030] When the camera 10 captures an image of a user who is not impersonating another user, the captured image includes the face of the user captured by the camera 10.

[0031] The ticket gate control device 20 controls the face authentication ticket gate 40 based on face authentication. The ticket gate control device 20 includes a communication unit 201, a spoofing detection unit 202, a face authentication processing unit 203, and a ticket gate control unit 204. The spoofing detection unit 202, the face authentication processing unit 203, and the ticket gate control unit 204 may be collectively referred to as a control unit (or a processing unit).

[0032] The communication unit 201 performs communication between the ticket gate control device 20 and the camera 10, and between the ticket gate control device 20 and the server device 30. For example, the communication unit 201 acquires a captured image from the camera 10. The communication unit 201 also transmits information indicating a request for face authentication to the server device 30, and receives a response to the request for face authentication from the server device 30.

[0033] The spoofing detection unit 202 performs face detection processing on the captured image acquired from the camera 10. When a face image is detected by the face detection processing, the spoofing detection unit 202 determines whether or not the captured image is an image of the face of an actual user. For example, the spoofing detection unit 202 performs at least one of inside detection and outside detection, which will be described later, and determines whether or not the captured image is an image of the face of an actual user (for example, a living person). A case in which the captured image is not an image of the face of an actual user (for example, a living person) may correspond to a case in which an act of spoofing has occurred.

[0034] The face authentication processing unit 203 performs face authentication processing. For example, when a face image is detected in a captured image, the face authentication processing unit 203 generates information indicating a face authentication request including facial feature amounts of the face image for the server device 30, and transmits the information to the server device 30 via the communication unit 201.

[0035] The face authentication processing unit 203 acquires a response to the request for face authentication via the communication unit 201. The response to the request for face authentication includes the determination result of the server device 30 as to whether or not the person to be authenticated has the right to pass through the face authentication ticket gate 40. Note that here, if the captured image includes the face of a user who is about to pass through the face authentication ticket gate 40, the person to be authenticated is the user who is actually about to pass through the face authentication ticket gate 40. On the other hand, if the captured image includes a "spoofed image," the person to be authenticated is the person whose face is included in the spoofed image, that is, a person who has the right to pass through the face authentication ticket gate 40, and not the user who is actually about to pass through the face authentication ticket gate 40.

[0036] The ticket gate control unit 204 controls the face authentication ticket gate 40 depending on whether the authentication target has the right to pass and whether the captured image is an image of the actual user's face. The determination result of whether the authentication target has the right to pass is included in the response to the request for face authentication. Whether the captured image is an image of the actual user's face is determined by the impersonation detection unit 202.

[0037] If the person to be authenticated has the right to pass and the captured image is an image of the actual face of the user, the ticket gate control unit 204 performs control to permit passage through the facial recognition ticket gate 40. The control to allow the person to be verified to pass through the facial recognition ticket gate 40 includes control to open the door provided in the passage of the facial recognition ticket gate 40, and control to output audio and / or display to notify the user that they have been permitted to pass.

[0038] If the authentication target does not have the right to pass, the ticket gate control unit 204 performs control to block passage through the face authentication ticket gate 40. The control to block passage through the face authentication ticket gate 40 includes control to close the doors provided in the passage of the face authentication ticket gate 40, control to output audio and / or display to notify the user that passage is prohibited, and control to notify the manager of the face authentication ticket gate 40, such as a station staff member. Note that it is not necessary to perform all of these controls; it is sufficient to perform any one or more of these controls.

[0039] If the captured image is not an image of the actual face of the user, the ticket gate control unit 204 performs control to block passage through the facial recognition ticket gate 40. The control to block passage through the facial recognition ticket gate 40 includes control to close the doors provided in the passage of the facial recognition ticket gate 40, and control to output audio and / or display a message to notify the user that passage is prohibited.

[0040] The ticket gate control unit 204 may perform different control depending on whether the authentication target does not have the right to pass or the captured image is not an image of the actual user's face. For example, if the captured image is not an image of the actual user's face, the ticket gate control unit 204 may notify the administrator of the face recognition ticket gate system 1 (e.g., station staff, airport staff, or security guard) via the communication unit 201 that the user is attempting to impersonate someone else. Furthermore, if the captured image is not an image of the actual user's face, the ticket gate control unit 204 may output an alarm from the face recognition ticket gate 40.

[0041] The server device 30 includes a storage unit 301, a face authentication unit 302, and a communication unit 303. The server device 30 may be a cloud server.

[0042] The storage unit 301 stores information about registrants, including facial images acquired from the registrants, in advance. A registrant is a person who registers information in order to receive a service that allows the registrant to pass through the facial recognition ticket gate 40 based on the results of facial recognition processing. The registered information includes facial features of the registrant. The storage unit 301 also stores information about whether each registrant has the right to pass.

[0043] The face authentication unit 302 receives a face authentication request from the ticket gate control device 20 via the communication unit 303 and performs face authentication. The face authentication unit 302 performs face authentication on the authentication target included in the face authentication request. For example, the face authentication unit 302 compares the facial features of the authentication target included in the face authentication request with the facial features of registrants stored in the storage unit 301, and calculates a score indicating the similarity between the facial features of the authentication target and the facial features of the registrants for each registrant. If the highest score is equal to or greater than a threshold, the face authentication unit 302 identifies the authentication target as the registrant corresponding to the highest score. After identifying which of the registrants the authentication target is, the face authentication unit 302 determines whether the identified registrant has the right to pass. For example, if the identified registrant has purchased a ticket, the face authentication unit 302 determines that the person to be matched corresponding to the identified registrant has the right to pass. Note that the face authentication method used by the face authentication unit 302 is not limited to the above, and other known face authentication methods may be used.

[0044] In addition, if the face authentication unit 302 is unable to identify who among the registered persons the person to be authenticated is, or if it is able to identify who among the registered persons the person to be authenticated is but the identified registered person does not have the right to pass, it determines that the person to be authenticated does not have the right to pass.

[0045] The face authentication unit 302 transmits a response including the result of face authentication to the ticket gate control device 20. For example, the response includes a determination result indicating whether or not the subject of authentication has the right to pass.

[0046] The registration terminal 50 is owned by a person who registers information in order to receive a service that allows passage through the facial authentication ticket gate 40 based on the result of facial authentication processing, for example. Alternatively, the registration terminal 50 may be a terminal installed in a location that provides a service that allows passage through the facial authentication ticket gate 40 based on the result of facial authentication processing. The person who registers information registers the information by using the registration terminal 50 to photograph their face, input information, etc. The registered information is transmitted to the server device 30.

[0047] 1 shows an example in which the ticket gate control device 20 and the server device 30 are separate devices, but the present disclosure is not limited to this. The ticket gate control device 20 may be configured integrally with the server device 30. Also, in the system configuration shown in FIG. 1, an example in which the ticket gate control device 20 is connected to the facial recognition ticket gate 40 by wire or wirelessly is shown, but the ticket gate control device 20 may be built into the facial recognition ticket gate 40. Also, in the facial recognition ticket gate system 1 according to this embodiment, some of the configuration shown in FIG. 1 may be omitted, or configuration not shown in FIG. 1 may be added.

[0048] In the system configuration shown in FIG. 1, an example is shown in which the ticket gate control device 20 includes the spoofing detection unit 202, but the present disclosure is not limited to this.

[0049] For example, the server device 30 may have an impersonation detection unit 202 and perform impersonation detection. In this case, the ticket gate control device 20 may transmit an impersonation detection request including a captured image acquired from the camera 10 to the server device 30, and the impersonation detection unit 202 of the server device 30 may perform impersonation detection.

[0050] For example, an information processing device (e.g., a PC) different from the server device 30 and the ticket gate control device 20 may have the spoofing detection unit 202 and perform spoofing detection. In this case, the ticket gate control device 20 may send an spoofing detection request including a photographed image acquired from the camera 10 to the information processing device, and the spoofing detection unit 202 of the information processing device may perform spoofing detection.

[0051] <Inside detection> Here, the inside detection performed by the spoofing detection unit 202 of the ticket gate control device 20 will be described.

[0052] When a face is detected in a captured image, the spoofing detection unit 202 identifies facial feature points and calculates the distance between at least two feature points. For example, the distance between feature points on the left and right sides of the face is calculated. Then, the spoofing detection unit 202 calculates the ratio of the left and right distances between the feature points on the left and right sides of the face.

[0053] Fig. 3 is a diagram showing examples of distances between feature points calculated in inside detection. Fig. 3 shows four examples of facial feature points and distances between feature points to be calculated. Note that Fig. 3 shows facial feature points extracted from a photographed image, so the left side of the illustrated facial feature points corresponds to the right side of the actual face, and the right side of the illustrated facial feature points corresponds to the left side of the actual face.

[0054] In Example 1, the distance between the feature point corresponding to the outer corner of the eye (hereinafter simply referred to as "outer corner of the eye") and the feature point corresponding to the temple (hereinafter simply referred to as "temple") is calculated for each of the right and left sides of the face. That is, the distance between the outer corner of the right eye and the right temple, and the distance between the outer corner of the left eye and the left temple are calculated.

[0055] In Example 1, the ratio of the distance R1 between the outer corner of the right eye and the right temple to the distance L1 between the outer corner of the left eye and the left temple is calculated.

[0056] In Example 2, the distance between the feature point corresponding to the inner corner of the eye (hereinafter simply referred to as "inner corner of the eye") and the feature point corresponding to the top of the nose (hereinafter simply referred to as "top of the nose") is detected on both the right and left sides of the face. That is, the distance between the inner corner of the right eye and the top of the nose, and the distance between the inner corner of the left eye and the top of the nose are detected. Note that the top of the nose is located in the center, and not on either the left or right side.

[0057] In Example 2, the ratio of the distance R2 between the inner corner of the right eye and the top tip of the nose to the distance L2 between the inner corner of the left eye and the top tip of the nose is calculated.

[0058] In Example 3, the distance between the feature point corresponding to the temple (hereinafter simply referred to as "temple") and the feature point corresponding to the top of the nose (hereinafter simply referred to as "top of nose") is detected on each of the right and left sides of the face. That is, the distance between the right temple and the top of the nose and the distance between the left temple and the top of the nose are detected.

[0059] In Example 3, the ratio of the distance R3 between the right temple and the top of the nose to the distance L3 between the left temple and the top of the nose is calculated.

[0060] In Example 4, the distance between the feature point corresponding to the corner of the mouth (hereinafter simply referred to as "mouth corner") and the feature point corresponding to the cheek (hereinafter simply referred to as "cheek") is detected on each of the right and left sides of the face. That is, the distance between the right corner of the mouth and the right cheek, and the distance between the left corner of the mouth and the left cheek are detected.

[0061] In Example 4, the ratio of the distance R4 between the right corner of the mouth and the right cheek to the distance L4 between the left corner of the mouth and the left cheek is calculated.

[0062] Note that the distance between feature points to be calculated is not limited to the example shown in Fig. 3. Furthermore, the distance between feature points of the detection target may be one (one set), or two or more (two sets or more).

[0063] Furthermore, although the above description shows an example in which the distance between two feature points is calculated for each of the left and right sides, and a distance ratio, which is the ratio of the distances between the left and right feature points, is calculated, the present disclosure is not limited to this. For example, the area of ​​a range formed by connecting three or more feature points may be calculated for each of the left and right sides, and an area ratio, which is the ratio of the left and right areas, may be calculated. For example, the area of ​​an eye formed by connecting feature points surrounding the eye may be calculated for each of the left and right sides, and an area ratio between the left and right eyes may be calculated. Furthermore, for example, the face may be divided into left and right halves using a plurality of feature points corresponding to the facial contour and feature points corresponding to the center of the face (for example, feature points corresponding to the top of the nose and the bridge of the nose), and the areas of the divided left and right faces may be calculated, and an area ratio between the left and right faces may be calculated.

[0064] The spoofing detection unit 202 calculates the ratio of the distance between feature points on the left and right sides of the face, as shown in FIG. 3, and performs spoofing detection based on changes in the ratio as the user moves.

[0065] Regarding the movement of a user through the facial recognition ticket gate 40, a zone may be set that defines the positional relationship between the entrance of the facial recognition ticket gate 40 and the user. The movement of the user may be indicated by the zone. In this case, the movement of the user corresponds to a change in the zone in which the user is located.

[0066] The following describes the zones provided in the face authentication ticket gate 40 and a method for estimating which zone a user is in.

[0067] Fig. 4A is a diagram showing an example of zones provided in the facial recognition ticket gate 40. Fig. 4A shows zones defined for a user moving in the direction of arrow K1 when the facial recognition ticket gate 40 is viewed from above.

[0068] 4A shows four zones: an outer zone, a start zone, an intermediate zone, and a charging zone. Whether the user is moving or not may be determined by estimating the location of the user in the four zones.

[0069] 4A are zones used for controlling the face recognition ticket gate 40 in the face recognition ticket gate system 1. For example, as a person moves between these zones, the response to the person changes in the control of the face recognition ticket gate 40 (for example, face recognition, charging processing, and control of opening and closing the doors of the face recognition ticket gate 40).

[0070] The outer zone is a zone corresponding to a stage before face recognition is started by the face recognition ticket gate system 1. For example, the face recognition ticket gate system 1 does not need to perform face recognition on people present in the outer zone.

[0071] The start zone is a zone corresponding to the stage at which face recognition is started by the face recognition ticket gate system 1. For example, when a person enters the start zone, the face recognition ticket gate system 1 (e.g., the ticket gate control device 20) starts face recognition for that person.

[0072] The intermediate zone is a zone corresponding to the stage where facial recognition is being performed by the facial recognition ticket gate system 1. A person (e.g., a user) who wishes to pass through the facial recognition ticket gate 40 enters the intermediate zone from the start zone, and while the person moves through the intermediate zone, the facial recognition ticket gate system 1 is performing facial recognition.

[0073] The charging zone is a zone corresponding to the stage where the facial recognition ticket gate system 1 finishes facial recognition and charges the user. A person attempting to pass through the facial recognition ticket gate 40 enters the charging zone from the intermediate zone, and while the person is moving through the charging zone, the facial recognition ticket gate system 1 completes facial recognition. If the facial recognition determines that the person is permitted to pass, the facial recognition ticket gate system 1 (e.g., the ticket gate control device 20) performs a charging process for the person moving through the charging zone, and performs control to permit the person from the charging zone to pass through the facial recognition ticket gate 40 (e.g., control to open the door). If the facial recognition determines that the person is not permitted to pass through the facial recognition ticket gate 40, the facial recognition ticket gate system 1 (e.g., the ticket gate control device 20) does not perform a charging process for the person moving through the charging zone, but instead performs control to block the person from the charging zone to the exit of the facial recognition ticket gate 40 (e.g., control to close the door).

[0074] An entrance and an exit may be defined for each zone. FIG. 4A illustrates an entrance to an outer zone, an exit from the outer zone that is also an entrance to the start zone, and an exit from the start zone that is also an entrance to an intermediate zone. The entrance to a certain zone is a part of the periphery of the zone, and is the part through which a user passes when entering the zone. For example, the entrance to a certain zone includes the part of the periphery of the zone that is farthest from the exit of the facial recognition ticket gate 40 (e.g., one side of a rectangle). The exit to a certain zone is also a part of the periphery of the zone, and is the part through which a user passes when leaving the zone. For example, the exit to a certain zone includes the part of the periphery of the zone that is closest to the exit of the facial recognition ticket gate 40 (e.g., one side of a rectangle).

[0075] The method for estimating the zone where the user is present is not particularly limited. For example, if the shooting range of a captured image is fixed, the position and / or size of a face in the captured image is associated with the zone where the user is present. The spoofing detection unit 202 may estimate the zone where the user is present based on the correspondence between the position and / or size of a face in the captured image and the zone where the user is present.

[0076] Fig. 4B is a diagram showing an example of the correspondence relationship between the position and / or size of a face in a captured image and the zone that the user is in. Fig. 4B exemplarily shows the correspondence relationship between the position and size of a face in a captured image captured by camera 10-2 (see Figs. 2 and 4A) and the zones shown in Fig. 4A.

[0077] When camera 10-2 moves in the direction of arrow K1 to continuously photograph a user passing through facial recognition ticket gate 40, the user's face becomes smaller the farther it is from facial recognition ticket gate 40. In other words, as shown in Fig. 4B, the face in the image photographed when the user is in the start zone is relatively small, and the face in the image photographed when the user is in the charging zone is relatively large in the photographed image.

[0078] When camera 10-2 moves in the direction of arrow K1 to continuously photograph a user passing through facial authentication ticket gate 40, the user's face moves from the left to the right of the photographed image. In other words, as shown in Fig. 4B, the face in the photographed image taken when the user is in the start zone is located relatively to the left in the photographed image, and the face in the photographed image taken when the user is in the charging zone is located relatively to the right in the photographed image.

[0079] Although not shown in Figure 4B, the face in the image captured when the user is in the outer zone is smaller than when the user is in the start zone. Also, the face in the image captured when the user is in the outer zone is located to the left of the image compared to when the user is in the start zone.

[0080] For example, the spoofing detection unit 202 estimates a zone corresponding to the position and / or size of a face in a captured image using the correspondence relationship described above. The estimated zone may be a zone where the user is present.

[0081] The method for estimating the zone is not particularly limited. For example, the zone in which the user is present may be estimated from an image captured by a camera that photographs the user from above the facial recognition ticket gate 40. Alternatively, the zone in which the user is present may be estimated from a sensor that is provided around the facial recognition ticket gate 40 and detects the position of the user. Alternatively, the zone in which the user is present may be estimated from a sensor (or camera) that detects the skeleton.

[0082] The spoofing detection unit 202 calculates the ratio of the distance between feature points on the left and right sides of the face, as shown in FIG. 3, for each zone in which the user is present. The spoofing detection unit 202 then performs spoofing detection based on the change in ratio that occurs depending on the zone in which the user is present. For example, the spoofing detection unit 202 calculates the ratio of the distance between feature points on the left and right sides in a captured image when the user is present in the outer zone, and calculates the ratio of the distance between feature points on the left and right sides in a captured image when the user is present in the start zone, and performs spoofing detection based on the change in these two ratios. If multiple captured images are acquired while the user is present in each zone, a representative value of the ratio of the distances in each zone is used as the distance ratio used for spoofing detection. Specific examples of the representative value include a value obtained by statistically processing the distance ratios obtained from multiple captured images, such as the average, median, maximum, or minimum value of the ratio. Furthermore, if the configuration allows detection of the user's position, it is possible to use the ratio at a predetermined position within each zone (such as the entrance to the zone, the center of the zone, or the exit of the zone) as the representative value. The value used as the representative value may differ for each zone, but in order to accurately grasp changes in the ratio across zones, it is better to use a value calculated using the same standard.

[0083] FIG. 5 is a diagram showing an example of a change in the ratio of the distances between feature points as the user moves.

[0084] Fig. 5 shows the change in ratio in the captured image of the actual user's face in each of the four zones shown in Fig. 4A. Note that Fig. 5 exemplarily shows the captured image captured by camera 10-2 (see Fig. 2 and Fig. 4A). Fig. 5 also shows the ratio between the feature point distance between the rightmost edge of the facial contour and the top tip of the nose and the feature point distance between the leftmost edge of the facial contour and the top tip of the nose.

[0085] In the example of Figure 5, the ratio A when the user is in the outer zone is 1 (= 1 / 1), the ratio B when the user is in the starting zone is 2 (= 2 / 1), the ratio C when the user is in the intermediate zone is 2.5 (= 2.5 / 1), and the ratio D when the user is in the charging zone is 3 (= 3 / 1).

[0086] 5, in a captured image of an actual user's face, the ratio of the distances between feature points changes as the user moves (e.g., changes zones). For example, the difference between ratio A and ratio B is 1, the difference between ratio B and ratio C is 0.5, and the difference between ratio C and ratio D is 0.5.

[0087] On the other hand, in a captured image of a spoofed image, the ratio of the distances between feature points does not change, or changes only slightly, as the user who owns the spoofed image moves (for example, moving from one zone to another). This is because the actual user's face has unevenness and curves, so the ratio of the distances between feature points is likely to change even with a slight change in angle, whereas in a flat spoofed image, the ratio of the distances between feature points is unlikely to change even with a change in angle.

[0088] Using such differences, the spoofing detection unit 202 performs spoofing detection. For example, the spoofing detection unit 202 calculates the ratio of the distances between feature points for each zone, calculates the difference between the ratios of different zones, and calculates the standard deviation of the difference. If the standard deviation is equal to or greater than a threshold, the spoofing detection unit 202 determines that the captured image is an image of the actual user's face. On the other hand, if the calculated standard deviation is less than the threshold, the spoofing detection unit 202 determines that the captured image is not an image of the actual user's face. In other words, if the calculated standard deviation is less than the threshold, the spoofing detection unit 202 determines that the captured image is an image of a spoofed image.

[0089] Note that a single distance between feature points may be used for the determination, or multiple distances between feature points may be used for the determination. For example, the distance between the outer corner of the eye and the temple shown in Example 1 of Fig. 3 may be used for the determination, or the distance between the outer corner of the eye and the temple shown in Example 1 of Fig. 3 and the distance between the inner corner of the eye and the top tip of the nose shown in Example 2 may each be used for the determination.

[0090] Furthermore, the method of calculating the ratio, the number of ratios to be calculated, the determination threshold, etc. may be set arbitrarily.

[0091] In calculating the ratio, the number and size of the specified zones may be set. For example, the number and size of the zones may be set so that the facial state also changes significantly. Furthermore, information about each zone (e.g., the distance between feature points) does not need to be used in order to calculate the standard deviation. For example, the difference in ratio may be calculated for combinations with large differences in distance (e.g., the combination of the charging zone and the outer zone in the example of FIG. 4A). Furthermore, sequentially acquired captured images may be held for a specific period of time and calculated later. For example, in the example of FIG. 4A, when the user reaches the intermediate zone, the distance between feature points may be calculated using a captured image corresponding to the user being located halfway between the start zone and the intermediate zone.

[0092] <Inside detection process flow> Fig. 6 is a flowchart showing an example of the flow of the inside detection process. The process shown in Fig. 6 may be started, for example, when a certain user is detected in a certain zone. Note that Fig. 6 corresponds to the flow of the process when cameras 10-1 and 10-2 capture an image of a user moving in the direction of arrow K1, as shown in Fig. 2.

[0093] The ticket gate control device 20 (e.g., the impersonation detection unit 202) determines whether face detection has been performed for the first time in a certain zone based on the image captured by the camera 10-1 (S101). For example, if the user is in a zone where face detection has already been performed, the ticket gate control device 20 determines that face detection has been performed in that zone. If the user is in a zone where face detection has not already been performed, the ticket gate control device 20 determines that face detection has been performed for the first time in that zone.

[0094] If face detection is not being performed for the first time in a certain zone (NO in S101), the flow returns to S101.

[0095] When face detection is performed for the first time in a certain zone (YES in S101), the ticket gate control device 20 acquires feature points from the detected face (S102).

[0096] The ticket gate control device 20 calculates the distance between the feature points (S103).

[0097] The ticket gate control device 20 calculates the ratio (distance ratio) between the distances between the feature points calculated for the left and right sides of the face (S104).

[0098] The ticket gate control device 20 determines whether a predetermined number of distance ratios have been calculated (S105). The predetermined number may be, for example, the number of set zones, or may be a number different from the number of set zones.

[0099] If the predetermined number of distance ratios have not been calculated (NO in S105), the flow returns to S101.

[0100] If a predetermined number of distance ratios have been calculated (YES in S105), the ticket gate control device 20 calculates the standard deviation of the distance ratios (S106). For example, the difference between the distance ratios calculated in two different zones is calculated multiple times, and the standard deviation is calculated from the multiple differences.

[0101] The ticket gate control device 20 determines whether the calculated standard deviation is less than a threshold value (S107).

[0102] If the standard deviation is not less than the threshold (NO in S107), for example, if the standard deviation is equal to or greater than the threshold, the change in the distance ratio is equal to or greater than a predetermined level, so the ticket gate control device 20 determines that the authentication target is a living person (S108). In other words, it determines that the captured image including the authentication target is an image of the user's actual face. Then, the flow ends.

[0103] If the standard deviation is less than the threshold (YES in S107), the change in the distance ratio is less than a predetermined level, and therefore the ticket gate control device 20 provisionally determines that the authentication target is a fake face (S109). In other words, the provisional determination is that the captured image including the authentication target is not an image of the user's actual face. In other words, a fake face refers to a facial image that is not an image of the user's actual face, and specifically, a face that the user is impersonating. However, a fake face is not limited to a face that the user is impersonating, and may include, for example, a face printed on the user's clothing, etc.

[0104] The determination based on the image captured by camera 10-1 is completed by S108 or S109. Note that a determination based on the image captured by camera 10-2 may be performed in parallel. The determination based on the image captured by camera 10-2 may be the same as S101 to S109 described above.

[0105] Next, the ticket gate control device 20 determines whether or not to perform a judgment (for example, an AND judgment) on the images captured by the multiple cameras 10 (S110). For example, this judgment is made based on the settings of the administrator of the face recognition ticket gate system 1 (for example, a station staff member, etc.).

[0106] If the AND judgment is not performed (NO in S110), the ticket gate control device 20 judges that the authentication target is a fake face based on the provisional judgment made in S109 (S111), and the flow then ends.

[0107] When performing an AND determination (YES in S110), the ticket gate control device 20 determines whether or not the determination based on the image captured by the other camera, that is, the camera 10-2, has been completed (S112).

[0108] If the determination based on the images captured by the other cameras has not been completed (NO in S112), the ticket gate control device 20 holds the provisional determination made in S109 and waits for a determination (S113). Then, the flow returns to S112.

[0109] When the judgment based on the image taken by the other camera is completed (YES in S112), the ticket gate control device 20 judges whether the result of the judgment based on the image taken by the other camera indicates that the face to be authenticated is a fake face (S114).

[0110] If the result of the determination based on the image captured by the other camera indicates that the authentication target is a fake face (YES in S114), the ticket gate control device 20 determines that the authentication target is a fake face based on the provisional determination in S109 and the determination in S114 (S115), and the flow then ends.

[0111] If the result of the determination based on the image captured by the other camera does not indicate that the authentication target is a fake face (NO in S114), the ticket gate control device 20 determines that the authentication target is a living person (S116), and the flow ends.

[0112] In a case where there are two cameras, if both of the results of the determination based on the images captured by the two cameras (i.e., the two results) indicate that the authentication target is a fake face, the authentication target is determined to be a fake face. In this case, if at least one of the two results indicates that the authentication target is a living person, the authentication target is determined to be a living person. However, the present disclosure is not limited to this. For example, in a case where there are two cameras, if at least one of the two results indicates that the authentication target is a fake face, the authentication target may be determined to be a fake face, and if both of the two results indicate that the authentication target is a living person, the authentication target may be determined to be a living person. In addition, the fact that the authentication target is a fake face may correspond to the fact that the authentication target is not a living person.

[0113] In addition, in a case where there are three or more cameras, if all of the results of the determination based on the images captured by the three or more cameras (i.e., three or more results) indicate that the authentication target is a fake face, the authentication target may be determined to be a fake face. In this case, if at least one of the three or more results indicates that the authentication target is a living person, the authentication target may be determined to be a living person.

[0114] In addition, in the case where there are three or more cameras, if more than half of the results of the determination based on the images captured by each of the three or more cameras (i.e., three or more results) indicate that the authentication target is a fake face, the authentication target may be determined to be a fake face. Alternatively, in the case where there are three or more cameras, if more than half of the results of the determination based on the images captured by the three or more cameras (i.e., three or more results) indicate that the authentication target is a living person, the authentication target may be determined to be a living person.

[0115] As described above, in inside detection, the communication unit 201 (an example of an acquisition unit) of the ticket gate control device 20 acquires from the camera 10 a captured image of a user attempting to enter through the entrance of the facial recognition ticket gate 40 (an example of a gate). The impersonation detection unit 202 (an example of a processing unit) of the ticket gate control device 20 extracts image information from the captured image and determines whether the camera 10 has captured the user's actual face based on changes in the image information in response to changes in the zone. The zone indicates the positional relationship between the entrance and the user. Note that the image information in inside detection is the distance between two feature points of the facial image detected in the captured image. In inside detection, the impersonation detection unit 202 extracts feature point distances on the left and right sides of the facial image, calculates the ratio between the distance between the feature points on the left side and the distance between the feature points on the right side, and determines that the camera 10 has captured the user's actual face if the change in the ratio in response to changes in the zone (e.g., standard deviation) is equal to or greater than a threshold.

[0116] The inside detection described above makes it possible to appropriately determine whether a captured image is an image of a person's actual face. Furthermore, the inside detection described above can avoid actions that would stop the user's movement, making it possible to detect impersonation without impeding walk-through performance.

[0117] In inside detection, the greater the change in the relative angle between the camera 10 and the user, the greater the change in the distance between feature points, allowing for more accurate spoofing detection. Therefore, the camera 10 may be positioned so that the user does not gaze at it, making it easier for the relative angle between the user and the camera 10 to change. For example, the camera 10 may be positioned in a location where the user cannot see it. This prevents the user from recognizing the location of the camera 10, thereby reducing the likelihood that the user will gaze at the camera 10. Furthermore, by positioning multiple cameras so that they can capture images of the user from different angles, an environment may be created in which the user cannot gaze at all cameras simultaneously. For example, multiple cameras 10 may be positioned in different positions, such as the right, left, upper, or lower, as viewed from the user. Then, inside detection may be performed based on images captured by cameras 10 excluding the camera 10 that the user gazes at. Using images captured by cameras 10 that the user does not gaze at can improve the accuracy of inside detection.

[0118] When images captured by each of the multiple cameras 10 are used for the judgment, the judgment method, such as whether to perform the judgment by AND of the respective judgment results, by OR, or by majority vote, may be set or changed by an administrator of the face recognition ticket gate system 1. When images captured by each of the multiple cameras 10 are used for the judgment, the calculation method of the information to be judged (for example, a calculation method of the distance between feature points, the ratio of the distance between feature points, the standard deviation of the ratio, etc.) may be set commonly for each of the images captured by the multiple cameras 10, or may be set independently for each of the images captured by the multiple cameras 10.

[0119] Furthermore, the distance between feature points used for detecting the inside may be the distance between feature points at different parts on the left and right sides. For example, the distance between the inner corner of the eye and the top of the nose on the left side of the face (e.g., distance L2 in Example 2 of Figure 3) may be used, and the distance between the corner of the mouth and the cheek on the right side of the face (e.g., distance R4 in Example 4 of Figure 3) may be used.

[0120] Furthermore, the distance between feature points used for interior detection may be the distance between feature points on only one side of the face. For example, spoofing detection may be performed using the ratio between the feature point distance between the inner corner of the eye and the top of the nose on the right side of the face (e.g., distance R2 in Example 2 of Figure 3) and the feature point distance between the corner of the mouth and the cheek on the right side of the face (e.g., distance R4 in Example 4 of Figure 3). Because a person's face has unevenness and curves, the ratio of feature point distances between feature points on different parts of the face may change as the user moves. Therefore, even if a configuration is used to evaluate changes in feature point distances focusing on only one side of the face, it is still possible to sufficiently distinguish between real faces and spoofed images. However, if the targets for calculating the feature point distance are too close, the change in feature point distance may be small even on real faces. Therefore, using the ratio of feature point distances on both the left and right sides of the face allows for more accurate spoofing detection.

[0121] <Outside detection> In a captured image of a normal user, the image information around the user's face changes as the user moves. On the other hand, in a captured image of an image used by an unauthorized user who commits impersonation (i.e., a spoofed image), the image information around the face does not change, or changes only slightly, as the unauthorized user moves, because the area around the face is included in the spoofed image.

[0122] Therefore, in outer detection, spoofing is detected by utilizing changes in image information around the face in the photographed image.

[0123] Fig. 7 is a diagram showing an example of image processing used for outside detection, which shows four stages of image processing.

[0124] In outside detection, the spoofing detection unit 202 identifies a face frame from the result of face detection. Then, the spoofing detection unit 202 creates an image with a margin (hereinafter, a margin image) by taking a margin outside the face frame and trimming the captured image. Note that the margin for the face frame may be determined according to the size of the face frame. For example, the margin may be set so that the ratio between the size of the face frame and the size of the margin is constant.

[0125] Next, the impersonation detection unit 202 removes the inside of the face frame from the margin image. For example, based on the margin image and the face frame, a mask for removing the inside of the face frame from the margin image is created, and the inside of the face frame is removed from the margin image by performing mask processing on the margin image. The image from which the inside of the face frame has been removed from the margin image is hereinafter referred to as an outer image. In outer detection, outer images are detected for each positional relationship (e.g., zone) between the face authentication ticket gate 40 and the user, and multiple outer images corresponding to different zones are compared by pattern matching or the like. Alternatively, for comparison of the outer images, parameters for image comparison (e.g., contrast) may be calculated from the outer images.

[0126] The zones used in outer detection may be the same as the zones used in inner detection, and the method for determining the zones used in outer detection may be the same as the method for determining the zones used in inner detection.

[0127] Furthermore, if photographed images are acquired multiple times while the user is in each zone, a representative outer image for each zone is used as the outer image. Specific examples of the representative outer image include an image created by combining outer images from multiple photographed images, or an outer image from one photographed image selected from the multiple photographed images. Furthermore, if the configuration allows the user's position to be detected, an outer image from an image taken at a predetermined position within each zone (such as the entrance, center, or exit of the zone) can be used as a representative outer image for outer detection. The type of outer image used may differ for each zone, but it is better to use outer images selected or created according to the same criteria in order to accurately grasp changes in the outer image across zones.

[0128] Since it is assumed that the spoofed image is held by the user, the spoofed image may be tilted. If the spoofed image is tilted, the image around the face (i.e., the outer image) cannot be properly extracted. Therefore, the tilt of the captured image of the spoofed image may be corrected using the coordinate information of the eyes, etc.

[0129] Fig. 8 is a diagram showing an example of tilt correction, which shows an example of the flow of correction for a photographed image with no tilt and a spoofed image with a tilt.

[0130] As shown in Fig. 8, when the spoofed image is tilted and a margin image including a face frame is extracted regardless of the tilt (Example 2 in Fig. 8), the outer image cannot be extracted properly, so the tilt may be estimated based on the coordinate information of the eyes, and a margin image may be extracted according to the estimated tilt (Example 3 in Fig. 8). Alternatively, the face frame coordinates including the tilt may be detected at the time the face frame is detected.

[0131] <Outside detection process flow> Fig. 9 is a flowchart showing an example of the flow of the outside detection process. The process shown in Fig. 9 may be started, for example, when a certain user is detected in a certain zone. Note that Fig. 9 corresponds to the flow of the process when cameras 10-1 and 10-2 capture an image of a user moving in the direction of arrow K1, as shown in Fig. 2.

[0132] The ticket gate control device 20 (e.g., the impersonation detection unit 202) determines whether face detection has been performed for the first time in a certain zone based on the image captured by the camera 10-1 (S201). For example, if the user is in a zone where face detection has already been performed, the ticket gate control device 20 determines that face detection has already been performed in that zone. If the user is in a zone where face detection has not already been performed, the ticket gate control device 20 determines that face detection has been performed for the first time in that zone.

[0133] If face detection is not being performed for the first time in a certain zone (NO in S201), the flow returns to S201.

[0134] When face detection is performed for the first time in a certain zone (YES in S201), the ticket gate control device 20 detects a face frame (S202).

[0135] The ticket gate control device 20 extracts the margin image (S203).

[0136] The ticket gate control device 20 performs tilt correction as shown in FIG. 8 (S204).

[0137] The ticket gate control device 20 performs mask processing on the face detected portion and generates an outside image (S205).

[0138] The ticket gate control device 20 determines whether image calculation is necessary (S206). For example, if the administrator of the face recognition ticket gate system 1 sets a method that requires image calculation for comparison of outer images in outer detection (for example, a method of comparing image histograms), it is determined that image calculation is necessary.

[0139] If image calculation is not required (NO in S206), the flow proceeds to S208.

[0140] If image calculation is necessary (YES in S206), the ticket gate control device 20 performs image calculation (S207), and the flow then proceeds to S208.

[0141] The ticket gate control device 20 then stores the results (S208). For example, if image calculation is not required, the stored results include the outer image. If image calculation is required, the stored results include the calculation results.

[0142] The ticket gate control device 20 determines whether a comparison target exists (S209). For example, if results corresponding to two different zones are stored, it is determined that a comparison target exists, and if results corresponding to two different zones are not stored, it is determined that a comparison target does not exist. Note that if results corresponding to two different zones are not stored, this corresponds to, for example, the case where two or more results are not stored, or the two or more stored outside images are for the same zone.

[0143] If there is no comparison target (NO in S209), the flow returns to S201.

[0144] If a comparison target exists (YES in S209), the ticket gate control device 20 performs the comparison (S210). For example, the ticket gate control device 20 may compare the outer images by pattern matching. Alternatively, the ticket gate control device 20 may compare calculated histograms.

[0145] The ticket gate control device 20 stores the comparison result (S211). The comparison result includes the similarity between the outside images.

[0146] The ticket gate control device 20 determines whether a predetermined number of comparison results (for example, similarities) have been calculated (S212).

[0147] If the predetermined number of comparison results have not been calculated (NO in S212), the flow returns to S201.

[0148] If a predetermined number of comparison results have been calculated (YES in S212), the ticket gate control device 20 determines whether the similarity resulting from the comparison is equal to or greater than a threshold (S213). If there are multiple similarities resulting from the comparison, it may determine whether all of the multiple similarities are equal to or greater than a threshold, or it may determine whether some of the multiple similarities are equal to or greater than a threshold. The number of similarities that are equal to or greater than the threshold may be set by the administrator of the face recognition ticket gate system 1, etc.

[0149] If the similarity is not equal to or greater than the threshold (NO in S213), for example, if the similarity is less than the threshold, the change in the external image is equal to or greater than a predetermined level, so the ticket gate control device 20 determines that the authentication target is a living person (S214). In other words, it determines that the captured image including the authentication target is an image of the user's actual face. Then, the flow ends.

[0150] If the standard deviation is equal to or greater than the threshold (YES in S213), the amount of change in the outer image is less than a predetermined level, and therefore the ticket gate control device 20 provisionally determines that the authentication target is a fake face (S215). In other words, the captured image including the authentication target is provisionally determined to be not an image of the user's actual face.

[0151] The determination based on the image captured by camera 10-1 is completed by S214 or S215. Note that a determination based on the image captured by camera 10-2 may be performed in parallel. The determination based on the image captured by camera 10-2 may be the same as S201 to S215 described above.

[0152] Next, the ticket gate control device 20 determines whether or not to perform a judgment (for example, an AND judgment) on the images captured by the multiple cameras 10 (S216). For example, this judgment is made based on the setting of the administrator of the face recognition ticket gate system 1 (for example, a station staff member, etc.).

[0153] If the AND judgment is not performed (NO in S216), the ticket gate control device 20 judges that the authentication target is a fake face based on the provisional judgment made in S215 (S217), and the flow then ends.

[0154] If an AND determination is to be made (YES in S216), the ticket gate control device 20 determines whether or not the determination based on the image captured by the other camera, that is, the camera 10-2, has been completed (S218).

[0155] If the determination based on the images captured by the other cameras has not been completed (NO in S218), the ticket gate control device 20 holds the provisional determination made in S215 and waits for a determination (S219). Then, the flow returns to S218.

[0156] When the judgment based on the image taken by the other camera is completed (YES in S218), the ticket gate control device 20 judges whether the result of the judgment based on the image taken by the other camera indicates that the face to be authenticated is a fake face (S220).

[0157] If the result of the determination based on the image captured by the other camera indicates that the authentication target is a fake face (YES in S220), the ticket gate control device 20 determines that the authentication target is a fake face based on the provisional determination in S215 and the determination in S220 (S221), and the flow then ends.

[0158] If the result of the determination based on the image captured by the other camera does not indicate that the authentication target is a fake face (NO in S220), the ticket gate control device 20 determines that the authentication target is a living person (S222), and the flow ends.

[0159] In a case where there are two cameras, if both of the results of the determination based on the images captured by the two cameras (i.e., the two results) indicate that the authentication target is a fake face, the authentication target is determined to be a fake face. In this case, if at least one of the two results indicates that the authentication target is a living person, the authentication target is determined to be a living person. However, the present disclosure is not limited to this. For example, in a case where there are two cameras, if at least one of the two results indicates that the authentication target is a fake face, the authentication target may be determined to be a fake face, and if both of the two results indicate that the authentication target is a living person, the authentication target may be determined to be a living person. In addition, the fact that the authentication target is a fake face may correspond to the fact that the authentication target is not a living person.

[0160] In addition, in a case where there are three or more cameras, if all of the results of the determination based on the images captured by the three or more cameras (i.e., three or more results) indicate that the authentication target is a fake face, the authentication target may be determined to be a fake face. In this case, if at least one of the three or more results indicates that the authentication target is a living person, the authentication target may be determined to be a living person.

[0161] In addition, in the case where there are three or more cameras, if more than half of the results of the determination based on the images captured by each of the three or more cameras (i.e., three or more results) indicate that the authentication target is a fake face, the authentication target may be determined to be a fake face. Alternatively, in the case where there are three or more cameras, if more than half of the results of the determination based on the images captured by the three or more cameras (i.e., three or more results) indicate that the authentication target is a living person, the authentication target may be determined to be a living person.

[0162] As described above, in outer detection, the communication unit 201 (an example of an acquisition unit) of the ticket gate control device 20 acquires from the camera 10 a captured image of a user attempting to enter through the entrance of the facial recognition ticket gate 40 (an example of a gate). The impersonation detection unit 202 (an example of a processing unit) of the ticket gate control device 20 extracts image information from the captured image and determines whether the camera 10 has captured the user's actual face based on changes in the image information in response to changes in the zone. The zone indicates the positional relationship between the entrance and the user. Note that the image information in outer detection is the image portion outside the facial image detected in the captured image. Then, in outer detection, the impersonation detection unit 202 determines that the camera 10 has captured the user's actual face if the change in the outer image in response to changes in the zone is equal to or greater than a threshold.

[0163] The above-described outside detection makes it possible to appropriately determine whether a captured image is an image of a person's actual face. Furthermore, the above-described outside detection can avoid actions that would stop the user's movement, so it is possible to perform impersonation detection without impeding walk-through performance.

[0164] In outer edge detection, a configuration may be used in which the area surrounding the user's actual face included in the captured image changes as the user moves. For example, the installation position of camera 10 may be adjusted so that the background of the camera's capture range changes as the user moves. Alternatively, multiple cameras 10 may be placed in different positions, such as on the right, left, upper, or lower side as viewed from the user. Then, a camera 10 that captures an image in which the area surrounding the user's actual face changes as the user moves may be selected, and outer edge detection may be performed based on the image captured by the selected camera. By using a configuration in which the area surrounding the face in the captured image changes as the user moves, the accuracy of outer edge detection can be improved.

[0165] When images captured by each of the multiple cameras 10 are used for the determination, the determination method, such as whether to perform the determination using AND of the respective determination results, OR, or by majority vote, may be set or changed by an administrator of the face recognition ticket gate system 1. When images captured by each of the multiple cameras 10 are used for the determination, the determination method (for example, a determination method using a histogram, a determination method using pattern matching, etc.) may be set commonly for each of the images captured by the multiple cameras 10, or may be set independently for each of the images captured by the multiple cameras 10.

[0166] <Variations> In the above-described inner detection, the distance between feature points is calculated for each zone, and in the outer detection, the outer image is detected for each zone. This reduces the possibility of incorrectly detecting feature points from an image or erroneously determining that a calculation result of the distance between feature points or a detection result of the outer image based on a small change in the user's position is spoofing. Furthermore, the zones used for this spoofing detection (e.g., inner detection and outer detection) are zones in which the ticket gate's control (e.g., facial recognition, charging process, etc.) responds differently to people. This allows for spoofing detection to be performed at just the right timing for the ticket gate's control. However, if erroneous determination of spoofing is acceptable, the distance between feature points may be calculated or the outer image may be detected at a timing other than the timing of the ticket gate's control. Other examples of timing include the expiration of a predetermined time period or the timing when a person passes through a predetermined area of ​​the ticket gate as detected by a sensor or the like installed in the ticket gate.

[0167] The above-described inside detection and outside detection may also be performed using images captured by each of the multiple cameras 10. When multiple types of spoofing detection are performed, the final determination method may be selected from a variety of methods.

[0168] Fig. 10 is a diagram showing an example of a variation of the determination method, which exemplarily shows a variation of the determination method when a face authentication ticket gate 40 is equipped with a camera 10-1 and a camera 10-2.

[0169] For example, in the inside detection, an AND determination may be applied between the determination result using the image captured by camera 10-1 and the determination result using the image captured by camera 10-2. For example, if both the determination result using the image captured by camera 10-1 and the determination result using the image captured by camera 10-2 indicate that the authentication target is a fake face, the authentication target is determined to be a fake face. In this case, if at least one of the determination result using the image captured by camera 10-1 and the determination result using the image captured by camera 10-2 indicates that the authentication target is not a fake face, the authentication target is determined to be not a fake face.

[0170] For example, in inside detection, an OR determination may be applied between a determination result using an image captured by camera 10-1 and a determination result using an image captured by camera 10-2. For example, if at least one of the determination result using the image captured by camera 10-1 and the determination result using the image captured by camera 10-2 indicates that the authentication target is a fake face, the authentication target is determined to be a fake face. In this case, if both the determination result using the image captured by camera 10-1 and the determination result using the image captured by camera 10-2 indicate that the authentication target is not a fake face, the authentication target is determined to be not a fake face.

[0171] In outer detection, as in the inner detection described above, an AND judgment may be applied between the judgment result using the image captured by camera 10-1 and the judgment result using the image captured by camera 10-2, or an OR judgment may be applied.

[0172] As shown in FIG. 10, an AND determination may be applied between the determination result based on the inside detection and the determination result based on the outside detection, or an OR determination may be applied.

[0173] The OR method is more likely to result in a spoofed response than the AND method. The OR method may also increase the chance of false positives, where a false positive is detected even when no spoofing has occurred.

[0174] Whether to apply OR determination or AND determination may be set based on the surrounding environment of the facial recognition ticket gate 40, the installation status of the facial recognition ticket gate 40 and the camera 10, etc. For example, if the surrounding environment of the facial recognition ticket gate 40 is one in which erroneous determination of either inside detection or outside detection is likely to occur, OR determination may be used. Also, for example, if the environment requires strict security, AND determination may be used. Furthermore, two or more determination results to which OR determination or AND determination is applied may be selected based on the surrounding environment, the installation status of the facial recognition ticket gate 40 and the camera 10, etc.

[0175] Furthermore, when three or more determination results are obtained, for example, when three or more cameras 10 are installed and determination results are obtained using images captured by each camera 10, a determination may be made by majority vote.

[0176] The present disclosure can be realized in software, hardware, or software in conjunction with hardware.

[0177] Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0178] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0179] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility.

[0180] The present disclosure may be implemented in any type of apparatus, device, or system (collectively referred to as a communications apparatus) that has a communications function. The communications apparatus may include a wireless transceiver and processing / control circuitry. The wireless transceiver may include a receiver and a transmitter, or both functions. The wireless transceiver (transmitter and receiver) may include a radio frequency (RF) module and one or more antennas. The RF module may include an amplifier, an RF modulator / demodulator, or the like. Non-limiting examples of communication devices include telephones (e.g., cell phones, smartphones), tablets, personal computers (PCs) (e.g., laptops, desktops, notebooks), cameras (e.g., digital still / video cameras), digital players (e.g., digital audio / video players), wearable devices (e.g., wearable cameras, smartwatches, tracking devices), game consoles, digital book readers, telehealth / telemedicine devices, communication-enabled vehicles or mobile transportation (e.g., cars, airplanes, ships), and combinations of the above devices.

[0181] Communications equipment is not limited to portable or mobile equipment, but also includes non-portable or fixed equipment, devices, and systems of any kind, such as smart home devices (such as appliances, lighting equipment, smart meters or metering devices, control panels, etc.), vending machines, and any other "things" that may exist on an IoT (Internet of Things) network.

[0182] Furthermore, in recent years, in the field of IoT (Internet of Things) technology, CPS (Cyber ​​Physical Systems) has been attracting attention as a new concept that creates new added value by linking information between physical space and cyberspace. This CPS concept can also be adopted in the above-mentioned embodiments.

[0183] That is, as a basic configuration of a CPS, for example, an edge server located in physical space and a cloud server located in cyberspace can be connected via a network, and processing can be distributed and performed by processors installed on both servers. Here, it is preferable that each piece of processing data generated on the edge server or cloud server is generated on a standardized platform, and the use of such a standardized platform can improve the efficiency of building a system that includes a variety of sensor groups and IoT application software.

[0184] Communications include data communications via cellular systems, wireless LAN systems, communications satellite systems, etc., as well as data communications via combinations of these.

[0185] A communications apparatus also includes devices such as controllers and sensors connected or coupled to a communications device that performs the communications functions described in this disclosure, such as controllers and sensors that generate control and data signals used by the communications device to perform the communications functions of the communications apparatus.

[0186] The communication apparatus also includes infrastructure facilities, such as base stations, access points, and any other apparatus, device, or system that communicates with or controls the various apparatuses listed above, but are not limited to these.

[0187] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the disclosure.

[0188] Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Industrial Applicability]

[0189] An embodiment of the present disclosure is suitable for a face authentication system. [Explanation of symbols]

[0190] 1. Facial recognition ticket gate system 10 Camera 20 Ticket gate control device 30 Server device 40 Facial Recognition Ticket Gate 50 registered terminals 201 Communications Department 202 Spoofing detection unit 203 Face recognition processing unit 204 Ticket gate control unit 301 Storage section 302 Face Recognition Unit 303 Communications Department

Claims

1. an acquisition unit that acquires, from the camera, an image of a user attempting to enter through an entrance of the gate; a processing unit that extracts image information from the captured image and determines whether the camera captured an actual face of the user based on a change in the image information in response to a change in the positional relationship between the entrance and the user; An information processing device comprising:

2. the image information is a distance between two feature points of a face image detected in the captured image; The information processing device according to claim 1 .

3. The processing unit extracts the distances on the left and right sides of the face image, calculates a ratio between the distance on the left side and the distance on the right side, and determines that the camera has captured an actual face of the user when a change in the ratio in response to a change in the positional relationship is equal to or greater than a threshold value. The information processing device according to claim 2 .

4. the processing unit calculates the ratio based on each of the plurality of captured images, and determines that the camera has captured an actual face of the user when the degree of variation in the ratio is equal to or greater than the threshold value. The information processing device according to claim 3 .

5. the image information is an image portion outside the face image detected in the captured image; The information processing device according to claim 1 .

6. The processing unit determines that the camera has captured an image of the user's actual face when a change in the image portion in accordance with the change in the positional relationship is equal to or greater than a threshold. The information processing device according to claim 5 .

7. the processing unit determines that the camera has captured an image of the user's actual face when a change in contrast or a change in pattern between the image portions having different positional relationships is equal to or greater than the threshold value; The information processing device according to claim 6 .

8. the image information includes a distance between two feature points of the face image detected in the photographed image and an image portion outside the face image detected in the photographed image; The information processing device according to claim 1 .

9. The processing unit extracts the distances on the left and right sides of the face image, and determines that the camera has captured the actual face of the user when a change in the ratio between the distance on the left side and the distance on the right side in response to a change in the positional relationship is equal to or greater than a first threshold, and / or when a change in the image portion in response to a change in the positional relationship is equal to or greater than a second threshold. The information processing device according to claim 8 .

10. the processing unit estimates the positional relationship based on a position of a face image in the captured image. The information processing device according to claim 1 .

11. The processing unit determines whether the camera has captured an actual face of the user based on a change between first image information extracted from a first captured image captured when the distance between the entrance and the user is equal to or greater than a first threshold, and second image information extracted from a second captured image captured when the distance between the entrance and the user is less than a second threshold that is smaller than the first threshold. The information processing device according to claim 1 .

12. The information processing device A photographed image of a user attempting to enter through the entrance of the gate is acquired from the camera; extracting image information from the captured image; determining whether the camera has captured an image of the actual face of the user based on a change in the image information in accordance with a change in the positional relationship between the entrance and the user; Information processing methods.

Images