Information processing device, information processing method, and program

Abstract

Provided is an information processing device comprising: an acquisition unit for acquiring time-series data representing a shift in the position, in a three-dimensional coordinate system, of a person included in a captured image in which a prescribed region has been captured; a determination unit that determines whether the person is a biological authentication subject on the basis of the time-series data; and an authentication unit that executes the biological authentication of the subject.

Description

[Technical field]

[0001] This disclosure relates to an information processing device, an information processing method, and program Regarding. [Background technology]

[0002] Patent document 1 discloses an authentication system that tracks people in a video, extracts candidate people who are located at a distance from an authentication camera where biometric authentication can be performed, and controls the execution of the biometric authentication based on whether or not the candidate people have an authentication history. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 065852 Summary of the Invention [Problem to be solved by the invention]

[0004] In an authentication system such as that described in Patent Document 1, even if a person does not intend to be authenticated, there is a risk that biometric authentication will be performed on that person if that person is present within a specified distance from the authentication camera.

[0005] The present disclosure relates to an information processing device, an information processing method, and program The purpose is to provide. [Means for solving the problem]

[0006] According to one aspect of the disclosure, a biometric authentication system includes an acquisition unit that acquires time-series data representing a displacement of a position in a three-dimensional coordinate system of a person included in a captured image of a predetermined area, a determination unit that determines whether or not the person is a subject of biometric authentication based on the time-series data, and an authentication unit that performs the biometric authentication on the subject. the determination unit calculates a score indicating a likelihood that the person is the target based on the time-series data, and determines whether or not the person is the target based on the score; the acquisition unit acquires the time-series data related to the person for each of a plurality of cameras based on the captured images captured by each of the cameras, and the determination unit integrates the scores calculated for each of the cameras. An information processing device is provided.

[0007] According to another aspect of the present disclosure, a method for detecting a subject of biometric authentication comprising: acquiring time series data representing a displacement of a position in a three-dimensional coordinate system of a person included in a captured image of a predetermined area; determining whether or not the person is a subject of biometric authentication based on the time series data; and performing the biometric authentication on the subject. In the determining step, a score indicating the likelihood that the person is the target is calculated based on the time-series data, and whether or not the person is the target is determined based on the score. In the acquiring step, the time-series data related to the person is acquired for each of a plurality of cameras based on the images captured by the respective cameras, and in the determining step, the scores calculated for each of the cameras are integrated. A method for processing information is provided.

[0008] According to another aspect of the present disclosure, a method for detecting a position of a person included in a captured image of a predetermined area is provided, the method comprising: acquiring time series data representing a displacement of a position in a three-dimensional coordinate system of a person included in a captured image of a predetermined area; determining whether or not the person is a subject of biometric authentication based on the time series data; and performing the biometric authentication on the subject. in the determining step, a score indicating a likelihood that the person is the target person is calculated based on the time-series data, and whether or not the person is the target person is determined based on the score; in the acquiring step, the time-series data related to the person is acquired for each of a plurality of cameras based on the images captured by each of the cameras; and in the determining step, the scores calculated for each of the cameras are integrated. A program for executing the information processing method is provided. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of the overall configuration of an iris authentication system according to a first embodiment. [Diagram 2] FIG. 2 is an image diagram showing a person authentication process in the iris authentication system according to the first embodiment. [Diagram 3] 4 is a diagram showing an example of time-series data stored in a storage unit according to the first embodiment; FIG. [Figure 4] 2 is a block diagram showing an example of a hardware configuration of the authentication device according to the first embodiment. FIG. [Diagram 5] 4 is a flowchart showing an outline of authentication processing performed in the authentication device according to the first embodiment. [Figure 6] 2 is a diagram showing an example of a movement trajectory of a person who is determined to be an authentication target by the authentication device according to the first embodiment. FIG. [Figure 7A] 4 is a diagram showing an example of a movement trajectory of a person who is determined by the authentication device according to the first embodiment to not be an authentication target. FIG. [Figure 7B]4 is a diagram showing an example of a movement trajectory of a person who is determined by the authentication device according to the first embodiment to not be an authentication target. FIG. [Figure 8] FIG. 11 is a block diagram showing an example of the overall configuration of an iris authentication system according to a second embodiment. [Figure 9] 13 is a schematic diagram showing an example of the arrangement of image capturing devices in the iris authentication system according to the second embodiment. FIG. [Figure 10] 10 is a flowchart showing an outline of authentication processing performed in an authentication device according to a second embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of the overall configuration of an iris authentication system according to a third embodiment. [Figure 12] FIG. 13 is a diagram illustrating an example of a neural network for explaining machine learning in a learning unit according to the third embodiment. [Figure 13] 13 is a flowchart showing an outline of authentication processing performed in an authentication device according to a third embodiment. [Figure 14] FIG. 13 is a block diagram showing an example of the overall configuration of an iris authentication system according to a fourth embodiment. [Figure 15] FIG. 13 is a diagram showing an example of an image projected by a projector according to a fourth embodiment. [Figure 16] FIG. 13 is a block diagram showing an example of the overall configuration of a face authentication system according to a fifth embodiment. [Figure 17] 13 is a flowchart showing an outline of authentication processing performed in an authentication device according to a fifth embodiment. [Figure 18] 13 is a flowchart showing an outline of authentication processing performed in an authentication device according to a sixth embodiment. [Figure 19] 13A to 13C are diagrams showing an example of a movement trajectory and a line of sight direction of a person who is determined by an authentication device according to a sixth embodiment to not be an authentication target. [Figure 20] FIG. 13 is a functional block diagram showing the overall configuration of an information processing device according to a seventh embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. In the drawings, similar or corresponding elements are denoted by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0011] [First embodiment] First, a description will be given of the configuration of an iris authentication system 1 in the first embodiment. Fig. 1 is a block diagram showing an example of the overall configuration of the iris authentication system 1 according to the first embodiment.

[0012] The iris authentication system 1 photographs the iris of the person to be authenticated and performs authentication by comparing the photographed image with a registered iris image. Iris patterns are unique to everyone and remain unchanged throughout a person's life. Therefore, it is possible to confirm the identity of a person by comparing the iris pattern acquired during the authentication process with an iris image registered in advance in a database.

[0013] The iris authentication system 1 can be applied to, for example, identity verification for immigration at airports, identity verification at government agencies, identity verification for entering and leaving factories and offices, identity verification when entering event venues, and the like.

[0014] 1, the iris authentication system 1 includes an authentication device 10, an image capturing device 20, a human presence sensor 30, and a notification device 40. Each device is connected to a network NW such as a LAN (Local Area Network) or the Internet.

[0015] The authentication device 10 is a computer such as a server that executes biometric authentication. Specifically, the authentication device 10 executes a process of matching an iris image (or feature amount) of a person to be authenticated, photographed by the photographing device 20, with a registered iris image (or feature amount) of a person to be registered in advance in a database, and authenticates the person to be authenticated based on the result of the matching.

[0016] The image capturing device 20 captures an image of a person present in a predetermined area based on control information input from the authentication device 10, and outputs the captured image to the authentication device 10. The image capturing device 20 in the first embodiment includes one overall overhead camera 21 and three iris cameras 22a to 22c. In the first embodiment, the term "predetermined area" refers to a three-dimensional space of a predetermined range and size located in front of the image capturing device 20.

[0017] The overall overhead camera 21 is an imaging device that captures an image overlooking a specific area using visible light. The overall overhead camera 21 may be a digital camera using a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, or the like, so as to be suitable for image processing in the authentication device 10. The overall overhead camera 21 may further include a light source that irradiates illumination light forward.

[0018] Iris cameras 22a to 22c are imaging devices consisting of an infrared light emitting device (not shown) and an infrared light camera (not shown), and capture images of a person's eyes using infrared light. The infrared light emitting device includes a light emitting element that emits infrared light, such as an infrared LED. The wavelength of the infrared light irradiated from the infrared light emitting device may be, for example, in the near-infrared region of about 800 nm.

[0019] The infrared camera includes a light receiving element configured to have sensitivity to infrared light. The infrared camera may be a digital camera using a CMOS image sensor, a CCD image sensor, or the like. An infrared light irradiating device irradiates a person's eye with infrared light, and the infrared light reflected by the iris is photographed by the infrared camera to obtain an image of the eye including an iris image used for iris authentication. By obtaining an iris image photographed by infrared light, a high-contrast image can be obtained regardless of the color of the iris, and the influence of reflection by the cornea can be reduced.

[0020] 2 is an image diagram of a person authentication process in the iris authentication system 1 according to the first embodiment. Here, a case is shown in which when a person P moves from a position O where the photographing device 20 is placed to a position X1 at a focal length FL, photographing process is performed by one of the three iris cameras 22a to 22c.

[0021] 2, the viewing ranges A to C of the three iris cameras 22a to 22c aligned vertically are indicated by dashed lines. The viewing ranges A and B partially overlap at position X1. Similarly, the viewing ranges B and C partially overlap at position P1. The viewing range D is preferably set to include a height of about 3 m from the ground at position P1.

[0022] The human presence sensor 30 is a detector that outputs a detection signal to the authentication device 10 when it detects a person passing through the sensor position. The authentication device 10 can control the shooting of the whole bird's-eye view camera 21 and the iris cameras 22a to 22c using the detection signal from the human presence sensor 30 as a trigger. As shown in FIG. 2, the human presence sensor 30 in the first embodiment is placed at a position X2 that is farther from the position O than the position X1. The human presence sensor 30 outputs a detection signal to the authentication device 10 when a person P passes through the position X2. This allows the authentication device 10 to start the iris cameras 22a to 22c at the timing when the person P approaches the position X1 of the focal length FL, and capture an image of the iris of the person P.

[0023] The notification device 40 is a device that issues various notifications and alerts to a person based on notification control information from the authentication device 10. The notification device 40 includes a display 41, an LED 42, and a speaker 43.

[0024] The display 41 displays a facial image of the person and a text message in the display area to notify the person of the passability of a gate (not shown), etc. The LED 42 notifies the person of the authentication result by switching between on and off and by switching the lighting color.

[0025] The speaker 43 outputs an alarm sound or a guidance voice to a person moving within a predetermined area in order to improve the accuracy of biometric authentication.

[0026] As shown in FIG. 1, the authentication device 10 includes an overall bird's-eye view image acquisition unit 11, a face detection unit 12, a time series data acquisition unit 13, a memory unit 14, an authentication subject determination unit 15, an iris camera setting unit 16, a biometric image acquisition unit 17, and a biometric authentication unit 18.

[0027] The overall overhead image acquisition unit 11 acquires an overall overhead image of a predetermined area captured by the overall overhead camera 21. The overall overhead image acquisition unit 11 continuously acquires the overall overhead images in time series. The face detection unit 12 detects the face of a person included in the overall overhead image and outputs the face image to the time series data acquisition unit 13.

[0028] The time series data acquisition unit 13 performs image analysis processing on the overall overhead image and the face image, calculates the position of the person's face in the image, the position of the eyes, the gaze direction, the distance between the eyes, etc., and generates time series data. When the time series data acquisition unit 13 acquires time series data representing the displacement of the position in a three-dimensional coordinate system of a person included in a captured image of a specified area, the time series data is stored in the storage unit 14. The displacement of the position in the three-dimensional coordinate system corresponds to a trajectory of the person's movement, and is therefore also referred to as a "trajectory of movement" in the following description.

[0029] 3 is a diagram showing an example of time-series data stored in the storage unit 14 according to the first embodiment. The time-series data includes, as data items, a captured image ID, a detected face image ID, a face image link ID, a face position, an eye position, an eye distance, a face direction, and a capture date and time.

[0030] The captured image ID is an identifier unique to each captured image. The detected face image ID is an identifier unique to each face image of a person detected from a captured image. The face image link ID is an identifier that associates multiple detected face images that are considered to be of the same person. The face position is the position coordinates of the person's face in the captured image. The eye position is the position coordinates of the person's eyes in the captured image. The eye distance is the in-image distance between the person's eyes. The face direction is the direction of the person's face in the captured image. Note that data items included in the time series data are not limited to these. The time series data may further include, for example, the person's line of sight direction and distance data to the person detected by a distance sensor (not shown) as data items.

[0031] The information stored in the storage unit 14 is not limited to the time-series data described above. The storage unit 14 stores various data necessary for the operation of the authentication device 10. Specifically, the storage unit 14 further stores biometric information (iris image) of a registered person that is referred to in biometric authentication by the biometric authentication unit 18, data on criteria for determining whether or not the person is a person to be authenticated, and the like.

[0032] The authentication subject determination unit 15 determines whether or not a person is a subject of biometric authentication based on time-series data. In the first embodiment, the authentication subject determination unit 15 determines that a person moving toward a position for capturing a biometric image used for biometric authentication is a subject of authentication. In other words, the authentication subject determination unit 15 determines that a person moving toward the capturing position is a person who has an intention of biometric authentication. Conversely, the authentication subject determination unit 15 determines that a person who simply passes in front of the image capture device 20, a person who stops within a specified area, a person who moves away from the image capture device 20, etc. are a person who does not have an intention of biometric authentication.

[0033] The iris camera setting unit 16 selects one of the iris cameras 22a to 22c based on the position of the person's eyes in a three-dimensional spatial coordinate system represented by time-series data, and sets the shooting conditions. The iris camera setting unit 16 also functions as a control unit that controls shooting by the iris cameras 22a to 22c.

[0034] The biometric image acquisition unit 17 acquires a biometric image (iris image) of a person captured by any of the iris cameras 22a to 22c. The biometric authentication unit 18 compares the biometric image (iris image) of the person to be authenticated acquired by the biometric image acquisition unit 17 with a registered biometric image (registered iris image) stored in the storage unit 14, and performs biometric authentication of the person to be authenticated.

[0035] 4 is a block diagram showing an example of a hardware configuration of the authentication device 10 according to the first embodiment. The authentication device 10 is a computer that performs calculations, control, and storage, and includes a processor 151, a RAM (Random Access Memory) 152, a ROM (Read Only Memory) 153, a storage 154, and a communication I / F (Interface) 155. Each device is connected to each other via a bus, wiring, a drive device, etc.

[0036] The processor 151 has a function of performing a predetermined calculation according to a program stored in the ROM 153, the storage 154, etc., and controlling each part of the authentication device 10. In addition, as the processor 151, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), etc. may be used. In addition, one of the above examples may be used, or a plurality of them may be used in parallel.

[0037] The RAM 152 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of the processor 151. The ROM 153 is composed of a non-volatile storage medium, and stores necessary information such as programs used in the operation of the authentication device 10.

[0038] The storage 154 is configured from a non-volatile storage medium, and stores a database and an operation program for the authentication device 10. The storage 154 is configured from, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0039] The communication I / F 155 is a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), 4G, or 5G, and is a module for communicating with other devices.

[0040] The processor 151 loads a program stored in the ROM 153, the storage 154, etc., into the RAM 152 and executes it. In this way, the processor 151 realizes the functions of the overall bird's-eye view image acquisition unit 11, the face detection unit 12, the time-series data acquisition unit 13, the authentication subject determination unit 15, the iris camera setting unit 16, the biometric image acquisition unit 17, the biometric authentication unit 18, etc.

[0041] The hardware configuration shown in FIG. 4 is an example, and other devices may be added, or some devices may not be provided. Some devices may be replaced with other devices having similar functions. Some functions of the first embodiment may be provided by other devices via a network, or the functions of the first embodiment may be distributed and realized among a plurality of devices. The illustrated hardware configuration can be changed as appropriate.

[0042] FIG. 5 is a flowchart showing an outline of the authentication process performed in the authentication device 10 according to the first embodiment.

[0043] In step S101, the overall overhead image acquisition unit 11 acquires an overall overhead image of a predetermined area by causing the overall overhead camera 21 to capture an image of the predetermined area.

[0044] In step S102, the face detection unit 12 performs image analysis on the overall overhead image to detect face images of people present in the image.

[0045] In step S103, the face detection unit 12 determines whether or not a face image of a person is detected from the overall overhead image. If the face detection unit 12 determines that a face image is detected (step S103: YES), the process proceeds to step S104. On the other hand, if the face detection unit 12 determines that a face image is not detected (step S103: NO), the process returns to step S101.

[0046] In step S104, the face detection unit 12 compares the face images detected consecutively in time series with each other, and if it determines that they are face images of the same person, it issues a face image link ID to associate the face images with each other. Methods for determining whether they are the same person include, for example, a method based on the positional relationship of faces detected in a photographed image, and a method of matching face images detected from each of a plurality of photographed images taken consecutively.

[0047] Also, a projector (not shown) may be used to project an identification image serving as a landmark onto or around a person detected in a captured image so as to track the person, thereby identifying the same person between different captured images. Specific examples of identification images include images that include different colors, symbols, or shapes for each person.

[0048] In step S105, the time series data acquisition unit 13 generates time series data related to the captured image and face image. Specifically, the time series data acquisition unit 13 extracts data on the face position, eye position, eye distance, and face direction of a person by performing image analysis on the captured image. Then, the time series data acquisition unit 13 associates the captured image ID, detected face image ID, face image link ID, face position, eye position, eye distance, face direction, and shooting date and time, and stores them in the storage unit 14 as time series data.

[0049] In step S106, the authentication subject determination unit 15 identifies a trajectory of the person's movement from the time-series data of the same person stored in the storage unit 14. In the first embodiment, the trajectory of the person's movement is identified based on the change in the position of the person's eyes or face, and the eye distance over time.

[0050] In step S107, the authentication subject determination unit 15 determines whether or not the trajectory of the movement of the person identified in step S106 matches a predetermined trajectory pattern related to the authentication subject. The trajectory pattern is a reference displacement (reference trajectory) when moving to a shooting position, which is predefined for the person regarded as the authentication subject.

[0051] Here, if authentication subject determination unit 15 determines that the trajectory of the movement of the detected person matches a predetermined trajectory pattern (step S107: YES), the process proceeds to step S108.

[0052] FIG. 6 is a diagram showing an example of a trajectory of a movement of a person that the authentication device 10 according to the first embodiment determines to be a person to be authenticated. The captured images IMG-11 and IMG-12 are images captured continuously at a predetermined cycle by the overall overhead camera 21. The faces F11 and F2 of the person P1 are detected from the captured images IMG-11 and IMG-12, respectively. The distance between the eyes of the person P1 in the captured image IMG-11 is D11. The distance between the eyes of the person P1 in the captured image IMG-12 is D12, which is longer than D11. Therefore, in FIG. 6, a trajectory L1 of the movement of the person P1 approaching the photographing device 20 in a straight line is specified. As a result, the authentication device 10 determines that the person P1 has the intention of being authenticated and is a person to be authenticated. Note that the pattern of the trajectory of the movement of a person determined to be a person to be authenticated is not limited to this.

[0053] On the other hand, if the authentication subject determination unit 15 determines that the trajectory of the movement of the detected person does not match the predetermined trajectory pattern (step S107: NO), the process returns to step S101.

[0054] 7A and 7B are diagrams showing an example of a trajectory of a person's movement that the authentication device 10 according to the first embodiment determines not to be an authentication target. In FIG. 7A, the captured images IMG-21 and IMG-22 are images captured continuously at a predetermined cycle by the overall overhead camera 21. The faces F21 and F22 of the person P2 are detected from the captured images IMG-21 and IMG-22, respectively. The eye distance of the person P2 in the captured image IMG-21 is D21. The eye distance of the person P2 in the captured image IMG-22 is D22, which is equal to D21. In addition, it can be seen from the position coordinates of the faces F21 and F22 that the person P2 is moving to the right. Therefore, in FIG. 7A, a trajectory L2 of the movement of the person P2 crossing a predetermined area without approaching the image capture device 20 is specified. As a result, the authentication device 10 determines that the person P2 is a person who has no intention of being authenticated, and is not a person to be authenticated.

[0055] In FIG. 7B, the captured image IMG-31 and the captured image IMG-32 are images captured continuously at a predetermined cycle by the overall overhead camera 21. The faces F31 and F32 of the person P3 are detected from the captured image IMG-31 and the captured image IMG-32, respectively. The distance between the eyes of the person P3 in the captured image IMG-31 is D31. The distance between the eyes of the person P3 in the captured image IMG-32 is D32, which is shorter than D31. In addition, it can be seen that the person P3 is moving to the left from the position coordinates of the faces F31 and F32. Therefore, in FIG. 7B, a trajectory L3 of the movement of the person P3 moving away from the photographing device 20 is specified. As a result, in the authentication device 10, the person P3 is determined to be a person who does not intend to be authenticated and is not a person to be authenticated. Note that the trajectory pattern of the movement of a person who is determined to be not a person to be authenticated is not limited to these.

[0056] In step S108, the iris camera setting unit 16 selects one of the iris cameras 22a to 22c that corresponds to the eye position and height of the person determined to be the authentication subject, and sets the shooting conditions such as the focus, shooting angle, etc. For example, if the person's height is the average adult height, the iris camera setting unit 16 selects the central iris camera 22b, and sets the shooting conditions.

[0057] In step S109, biometric image acquisition unit 17 controls the iris camera set in step S108 to capture an iris image of the person. The iris camera captures an iris image at a higher resolution and in a smaller area than the overall overhead camera (first camera) 21, but a high-precision iris image can be acquired by selecting and using an appropriate one of iris cameras (second cameras) 22a to 22c.

[0058] In step S110, the biometric authentication unit 18 compares the iris image of the person with a group of iris images of registered persons pre-registered in the storage unit 14, and performs iris authentication.

[0059] In step S111, the biometric authentication unit 18 outputs the result of the iris authentication, and ends the process.

[0060] According to the first embodiment, prior to the authentication process, the intention of a person to be authenticated is determined based on time-series data representing the displacement of the position (movement trajectory) of a person present in a captured image in a three-dimensional coordinate system. This provides the iris authentication system 1 that can execute the authentication process for a person to be authenticated.

[0061] Furthermore, the authentication device 10 is configured to determine that a detected person moving toward a photographing position is a person to be authenticated by comparing the positional displacement in a three-dimensional coordinate system of the detected person obtained from the time series data with a reference displacement (reference trajectory) that is predefined for a person to be considered as a person to be authenticated when the person moves toward a photographing position of a biometric image used for biometric authentication, thereby making it possible to easily and accurately determine whether or not a detected person is a person to be authenticated.

[0062] [Second embodiment] The iris authentication system 2 according to the second embodiment will be described below. The following mainly describes the differences from the first embodiment, and the description of the common parts will be omitted or simplified.

[0063] Fig. 8 is a block diagram showing an example of the overall configuration of the iris authentication system 2 according to the second embodiment. Fig. 9 is a schematic diagram showing an example of the arrangement of the image capturing device 20 in the iris authentication system 2 according to the second embodiment.

[0064] The iris authentication system 2 according to the second embodiment differs from the iris authentication system 1 according to the first embodiment in that a plurality of image capturing devices 20 are arranged at a distance from each other, and in that the iris authentication system 2 has a configuration for scoring the likelihood that a detected person is a person to be authenticated. As shown in Fig. 8, each of the plurality of image capturing devices 20 has one overall overhead camera 21 and three iris cameras 22a to 22c.

[0065] 9, the multiple image capturing devices 20 are placed apart from each other and capture images of the predetermined area from different positions, so that people present in the predetermined area are captured by the multiple image capturing devices 20 at different angles.

[0066] When the overall overhead image acquisition unit 11 acquires a plurality of captured images acquired from a plurality of image capturing devices 20 (overall overhead cameras 21), it outputs the captured images to the face detection unit 12. The face detection unit 12 detects a person's face image from the captured images acquired in time series. When the time series data acquisition unit 13 generates time series data associated with the camera ID of the overall overhead camera 21, it stores the time series data in the storage unit 14.

[0067] Fig. 10 is a flowchart showing an outline of authentication processing performed in authentication device 10 according to the second embodiment. This processing has steps S101 to S106 and steps S108 to S111 in common with the flowchart shown in Fig. 5. Therefore, the following description will focus on the points that differ from the flowchart in Fig. 5.

[0068] When the processes of steps S101 to S106 are completed, the process proceeds to step S201.

[0069] In step S201, the authentication target person determination unit 15 calculates a score indicating the likelihood that a person is an authentication target person, based on the time-series data of each of the overall overhead cameras 21.

[0070] In step S202, the authentication subject determination unit 15 identifies the same person across cameras. Specific examples of a method for identifying whether or not a person is the same across multiple cameras include a method using characteristics of clothing or gait, and a method of projecting different colors around each person using a projector and identifying people with the same color as the same person.

[0071] In step S203, the authentication target person determination unit 15 determines an integrated score for the person by integrating the scores calculated for each of the overall overhead cameras 21. Examples of the integrated score include an average value, a deviation value, and a maximum value.

[0072] In step S204, the authentication subject determination unit 15 determines whether the total score is equal to or greater than a predetermined threshold. If the authentication subject determination unit 15 determines that the total score is equal to or greater than the predetermined threshold (step S204: YES), the detected person is considered to be an authentication subject, and the process proceeds to step S205.

[0073] On the other hand, if the authentication target person determination unit 15 determines that the total score is less than the predetermined threshold value (step S204: NO), the detected person is considered to be not a person to be authenticated, and the process returns to step S101.

[0074] In step S205, the authentication subject determination unit 15 assigns numbers to the authentication subjects in order of proximity to the authentication area. The numbers correspond to the priority order of biometric authentication. After that, the process proceeds to step S108.

[0075] According to the second embodiment, it is possible to determine whether or not a person is a person to be authenticated based on multiple captured images taken from different angles by multiple imaging devices 20, thereby making it possible to improve the accuracy of the determination process compared to the first embodiment.

[0076] In addition, since the authentication device 10 is configured to calculate a score indicating the likelihood that a person detected from a captured image is a person to be authenticated, the authentication device 10 can flexibly change the criteria for determining a person to be authenticated, for example, by adjusting a threshold value to be compared with the calculated score.

[0077] [Third embodiment] The iris authentication system 3 according to the third embodiment will be described below. The following mainly describes the differences from the first embodiment, and the description of the common parts will be omitted or simplified.

[0078] Fig. 11 is a block diagram showing an example of the overall configuration of the iris authentication system 3 according to the third embodiment. As shown in Fig. 11, the iris authentication system 3 according to the third embodiment is different from the iris authentication system 1 according to the first embodiment in that the authentication device 10 further includes a learning unit 19.

[0079] The learning unit 19 learns a model for determining whether or not a person is a person to be authenticated, based on time series data on a first person determined to be a person to be authenticated and time series data on a second person determined not to be a person to be authenticated. Note that the learning unit 19 can also learn a model using the time series data of either the first person or the second person.

[0080] FIG. 12 is a diagram showing an example of a neural network for explaining machine learning in the learning unit 19 according to the third embodiment. The neural network shown in FIG. 12 includes an input layer having a plurality of nodes, an intermediate layer having a plurality of nodes, and an output layer having one node. Time series data indicating the trajectory of the movement of a person detected from a captured image is input as input values ​​x(1) to x(n) to each node of the input layer. Each node of the intermediate layer is connected to each node of the input layer. Each element of the input value input to the node of the intermediate layer is used for calculation in each node of the intermediate layer. Each node of the intermediate layer calculates a calculation value using, for example, the input value input from each node of the input layer, a predetermined weighting coefficient, and a predetermined bias value. Each node of the intermediate layer is connected to the output layer, and outputs the calculated calculation value to a node of the output layer. The node of the output layer receives the calculation value from each node of the intermediate layer.

[0081] The nodes in the output layer use the calculated values ​​input from each node in the intermediate layer, the weighting coefficients, and the bias values ​​to output a value indicating the result of the determination as to whether or not the person is a person to be authenticated. When training the neural network, for example, the backpropagation method is used. Specifically, the output value when data is input to the input layer is compared with the output value obtained from the teacher data, and the error between the two compared output values ​​is fed back to the intermediate layer. This is repeated until the error falls below a predetermined threshold. By this type of training process, when any time-series data is input to the neural network (training model), it is possible to output a value y indicating the result of the determination as to whether or not the target person is a person to be authenticated.

[0082] As time-series data used for learning, it is preferable to use, for example, the following data depending on the model to be generated. (1) When generating a discrimination model between certified and non-certified subjects (A) Information on the trajectory from when a person is captured by the overall overhead camera 21 to when biometric authentication is completed (the person's movement trajectory and eye position trajectory) and the person's authentication history information (whether or not authentication processing was performed and the authentication result). (B) Information on the trajectory of a person from when they are captured by the overall overhead camera 21 until they leave the camera's range, and information that biometric authentication was not performed on that person. (2) When generating an anomaly detection model Information on the trajectory (movement trajectory and eye position trajectory) from when a person is photographed by the overall bird's-eye view camera 21 until authentication is completed, and authentication history information of that person (whether or not authentication processing was performed and the authentication result). However, only the data of the person to be authenticated is used. For example, when using the anomaly detection model generated from the time series data shown in (2), when the time series data of a person detected from a captured image is input to the anomaly detection model, the degree of anomaly in the time series data of that person can be determined. That is, the discrimination method using the anomaly detection model is a method in which normal data distribution is learned in advance using normal data input, and the difference from normal data is measured during practical use. Therefore, for example, by generating an anomaly detection model by learning a probability density map in advance using normal authentication operation data, the degree of anomaly for data acquired from a detected person can be calculated.

[0083] Fig. 13 is a flowchart showing an outline of authentication processing performed in authentication device 10 according to the third embodiment. This processing has steps S101 to S105 and steps S108 to S111 in common with the flowchart shown in Fig. 5. Therefore, the following description will focus on the points that differ from the flowchart in Fig. 5.

[0084] When the processes of steps S101 to S105 are completed, the process proceeds to step S301.

[0085] In step S301, the authentication subject determination unit 15 inputs time-series data relating to a person detected from a captured image to a discrimination model. It is assumed that the discrimination model has been trained prior to the processing of FIG.

[0086] In step S302, the authentication subject determination unit 15 determines whether or not the person is a person to be authenticated based on the determination result output from the discriminant model. If the authentication subject determination unit 15 determines that the person is a person to be authenticated (step S302: YES), the process proceeds to step S108.

[0087] On the other hand, if the authentication subject determination unit 15 determines that the person is not a subject to authentication (step S303: NO), the process returns to step S101.

[0088] According to the third embodiment, it is possible to quickly and accurately determine whether or not a person detected from a captured image is a person to be authenticated, based on a discrimination model obtained by machine learning.

[0089] The machine learning model in the third embodiment is not necessarily limited to a neural network. Any machine learning method that requires data and a learning process in advance and can perform class classification or binary classification for high-dimensional input data can be used. For example, in addition to the neural network, classifiers such as logistic regression and SVM (Support Vector Machine) may be used.

[0090] For example, the following time-series data D1 of a detection target ID1 is assumed as input data. D1 = {[ID1, time (1), face position (1), eye distance (1)], ... [ID1, time (n), face position (n), eye distance (n)]} All that is required is to perform classification on such two-dimensional array (or one-dimensional time series array) input data.

[0091] In addition, features may be extracted from input high-dimensional time-series data using a neural network or the like, and the above classification method may be combined. In this case, the above time-series data D1 is converted into another-dimensional feature D1' using a predetermined calculation formula (1), and classification is performed on D1'. D1' = f1(D1) (1) Specific examples of f1(x) include feature extractors using models based on Convolutional Neural Networks (CNNs), such as ResNet, VGGNet, and GoogLeNet, as well as feature extractors using ResNeXt, SENet, EfficientNet, etc.

[0092] Alternatively, the above processing may be performed for each image rather than in chronological order, and the obtained scores may be added together in chronological order, or Gaussian weighting may be applied, etc., to integrate the scores into a single score for judgment. For example, the following data string d2_t at each time t is acquired as input data. d2_t=[ID2, time t, face position (t), eye distance (t)] Next, the data string d2_t is substituted into a predetermined calculation formula (2) to obtain a value sc2_t indicating the likelihood of being an authenticated person from various classifiers. sc2_t = f2(d2_t) (2) Specific examples of f2(x) include classifiers using CNN-based models such as ResNet, VGGNet, and GoogleNet, as well as classifiers using ResNeXt, SENet, EfficientNet, and the like, and classifiers such as logistic regression models or SVM models. Then, the value sc2_t indicating the authenticator likelihood is substituted into a predetermined calculation formula (3) to calculate the average (or weighted average), and the total score Total_Score is output as the final result. Total_Score=(1 / T)Sigma sc2_t ···(3)

[0093] [Fourth embodiment] The iris authentication system 4 according to the fourth embodiment will be described below. The following mainly describes the differences from the second embodiment, and the description of the common parts will be omitted or simplified.

[0094] Fig. 14 is a block diagram showing an example of the overall configuration of the iris authentication system 4 according to the fourth embodiment. As shown in Fig. 14, the iris authentication system 4 according to the fourth embodiment differs from the iris authentication system 2 according to the second embodiment in that the authentication device 10 further includes a guide unit 101 and that a projector 50 is further connected to the network NW.

[0095] The guidance unit 101 generates an image for guiding a person determined to be an authentication target to a location where biometric authentication is performed. Since the guidance unit 101 knows the current location based on the time series data of the authentication target, it can guide the authentication target to the closest imaging device 20. When multiple authentication targets are detected, each person may be guided to a less crowded imaging device 20 based on the priority order of the people determined by image analysis of the captured images, for example.

[0096] Projector 50 projects the guidance image generated by guidance unit 101 onto, for example, a floor or wall surface in the vicinity of the person to be authenticated, or onto a part of imaging device 20 that has iris cameras 22a-22c built in to capture an iris image of the person to be authenticated.

[0097] 15 is a diagram showing an example of an image projected by a projector according to the fourth embodiment. Here, a projector 50 is placed above the authentication area, and projects a guidance image IMG onto the floor surface in the forward direction toward a person P who is the subject of authentication. The image IMG includes a guidance message MSG for person P ("Please perform authentication using the camera on the left") and an arrow guiding the direction to proceed.

[0098] According to the fourth embodiment, it is possible to present an image that guides a person who has been determined to be an authentication target to a place where biometric authentication is performed, thereby making it possible to improve the efficiency of biometric authentication.

[0099] [Fifth embodiment] A face authentication system 5 according to the fifth embodiment will be described below. Differences from the first embodiment will be mainly described below, and descriptions of common parts will be omitted or simplified.

[0100] Fig. 16 is a block diagram showing an example of the overall configuration of a face authentication system 5 according to the fifth embodiment. The face authentication system 5 according to the fifth embodiment is different from the iris authentication system 1 according to the first embodiment in that the face authentication system 5 performs face authentication based on a face image of a person acquired from an overall overhead image, rather than iris authentication. For this reason, as shown in Fig. 16, the authentication device 10 does not include an iris camera setting unit 16 and a biometric image acquisition unit 17. Moreover, the photographing device 20 does not include iris cameras 22a to 22c, but only includes an overall overhead camera 21.

[0101] Fig. 17 is a flowchart showing an outline of authentication processing performed in the authentication device 10 according to the fifth embodiment. This processing is common to steps S101 to S107 and step S111 in the flowchart shown in Fig. 5. Therefore, the following description will focus on the points that differ from the flowchart in Fig. 5.

[0102] When the processes of steps S101 to S107 are completed, the process proceeds to step S401.

[0103] In step S401, the face detection unit 12 selects a face image suitable for face authentication from among a plurality of face images of the person to be authenticated stored in the storage unit 14. For example, it is preferable that the face detection unit 12 selects a face image of the person to be authenticated facing the camera.

[0104] In step S402, the biometric authentication unit 18 performs face authentication by comparing the face image of the person to be authenticated selected in step S401 with a group of face images of registered persons pre-registered in the storage unit 14. After that, the process proceeds to step S111.

[0105] According to the fifth embodiment, after a detected person is determined to be an authentication target based on the overall overhead image, face authentication can be performed based on the face image of the same person acquired from the overall overhead image. Since it is not necessary to separately capture the iris image of the person to be authenticated by the iris cameras 22a to 22c, there is an advantage that the biometric authentication system can be introduced easily and at low cost compared to the above-mentioned first to fourth embodiments.

[0106] [Sixth embodiment] The biometric authentication system according to the sixth embodiment will be described below. The following mainly describes the differences from the first embodiment, and the description of the common parts will be omitted or simplified.

[0107] The biometric authentication system according to the sixth embodiment differs from the iris authentication system 1 according to the first embodiment in that it determines whether or not a person is a person to be authenticated by taking into account not only the displacement of a person's position (movement trajectory) in a three-dimensional coordinate system but also the orientation of the person's body parts such as the face and eyes.

[0108] Fig. 18 is a flowchart showing an outline of authentication processing performed in authentication device 10 according to the sixth embodiment. This processing is common to the flowchart shown in Fig. 5 in steps S101 to S107 and steps S108 to S111. Therefore, the following description will focus on the points that differ from the flowchart in Fig. 5. In the following description, it is assumed that the person to be authenticated is authenticated by the image capture device 20 installed near an entrance / exit gate (not shown).

[0109] When the processes of steps S101 to S107 are completed, the process proceeds to step S501.

[0110] In step S501, the authentication subject determination unit 15 determines whether or not the person is facing the entrance / exit gate based on the face direction or gaze direction included in the time-series data. If the authentication subject determination unit 15 determines that the person is facing the entrance / exit gate (step S501: YES), the process proceeds to step S108.

[0111] On the other hand, if the authentication subject determination unit 15 determines that the person is not facing the entrance / exit gate (step S501: NO), the process returns to step S101. When determining whether the person is facing the location where biometric authentication is performed, such as the entrance / exit gate, it is preferable to consider the duration during which the face direction or line of sight direction was within a predetermined range. For example, if the person continues to face the entrance / exit gate for a predetermined period of time or longer, it can be inferred that the person intends to undergo biometric authentication.

[0112] Fig. 19 is a diagram showing an example of the movement trajectory and gaze direction of a person who is determined by the authentication device 10 according to the sixth embodiment not to be authenticated. In Fig. 19, the captured images IMG-41 and IMG-42 are images captured continuously at a predetermined cycle by the overall overhead camera 21. The faces F41 and F42 of the person P4 are detected from the captured images IMG-41 and IMG-42, respectively. The distance between the eyes of the person P4 in the captured image IMG-41 is D41. The distance between the eyes of the person P4 in the captured image IMG-42 is D41, which is longer than D41. Furthermore, it can be seen from the position coordinates of the faces F41 and F42 that the person P4 is moving diagonally downward to the left.

[0113] 19, a trajectory of the movement of person P4 approaching slightly closer to the photographing device 20 is identified. However, the line of sight direction E41 of person P4 in photographed image IMG-41 and the line of sight direction E42 of person P4 in photographed image IMG-42 do not match the direction of the photographing device 20. That is, it is understood that person P4 is a person who remains within a predetermined area. As a result, in the authentication device 10, person P4 is determined to be a person who does not intend to be authenticated and is not a person to be authenticated. Note that the trajectory pattern of the movement of a person who is determined to be not a person to be authenticated is not limited to this.

[0114] According to the sixth embodiment, prior to biometric authentication, whether or not a person is a target for authentication is determined by taking into account not only the displacement of a person's position in a three-dimensional coordinate system (movement trajectory) but also the orientation of a specific body part of the person, so that it is possible to determine with even greater accuracy whether or not a person is willing to undergo biometric authentication.

[0115] [Seventh embodiment] 20 is a functional block diagram showing the overall configuration of an information processing device 100 according to the seventh embodiment. The information processing device 100 includes an acquisition unit 100A, a determination unit 100B, and an authentication unit 100C. The acquisition unit 100A acquires time-series data representing the displacement of a position in a three-dimensional coordinate system of a person included in a captured image of a predetermined area. The determination unit 100B determines whether or not the person is a subject of biometric authentication based on the time-series data. The authentication unit 100C executes biometric authentication on the subject.

[0116] According to the seventh embodiment, an information processing device 100 capable of performing biometric authentication on a person who is to be authenticated is provided.

[0117] [Modified embodiment] This disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of this disclosure. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of this disclosure.

[0118] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-mentioned embodiment is recorded on a storage medium, the program recorded on the storage medium is read as code, and executed on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. In addition, not only the storage medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment. In addition, one or more components included in the above-mentioned embodiments may be circuits such as ASICs and FPGAs configured to realize the functions of each component.

[0119] Examples of the storage medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disk)-ROM, a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the storage medium, but also those that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0120] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0121] It should be noted that the above-mentioned embodiments are merely examples of the implementation of this disclosure, and the technical scope of this disclosure should not be interpreted as being limited by these embodiments. In other words, this disclosure can be implemented in various forms without departing from its technical concept or main features.

[0122] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0123] (Appendix 1) an acquisition unit that acquires time series data representing a displacement of a position of a person included in a captured image of a predetermined area in a three-dimensional coordinate system; a determination unit that determines whether the person is a subject of biometric authentication based on the time-series data; an authentication unit that performs the biometric authentication on the subject; An information processing device comprising:

[0124] (Appendix 2) the determination unit determines that the person moving toward the photographing position is the subject by comparing the displacement with a reference displacement when the person moves toward the photographing position of a biometric image used for the biometric authentication. 2. An information processing device according to claim 1.

[0125] (Appendix 3) The determination unit determines whether or not the person is the target person based on the time series data and a line of sight or a face direction of the person detected from the captured image. 10. An information processing device according to claim 1 or 2.

[0126] (Appendix 4) the determination unit calculates a score indicating a likelihood that the person is the target person based on the time-series data, and determines whether the person is the target person based on the score; 10. An information processing device according to claim 1 or 2.

[0127] (Appendix 5) The acquisition unit acquires the time-series data related to the person for each of a plurality of cameras based on the captured images captured by the cameras, The determination unit integrates the scores calculated for each of the cameras. 5. An information processing device according to claim 4.

[0128] (Appendix 6) a learning unit that learns a model for determining whether or not a person is the target person based on the time series data on a first person determined to be the target person and the time series data on a second person determined not to be the target person; 6. The information processing device according to any one of claims 1 to 5, further comprising:

[0129] (Appendix 7) a guidance unit that, when the person is determined to be the target person, provides the person with a guidance image regarding the biometric authentication; 7. The information processing device according to any one of claims 1 to 6, further comprising:

[0130] (Appendix 8) acquiring time series data representing a displacement of a position of a person included in a captured image of a predetermined area in a three-dimensional coordinate system; determining whether the person is a subject of biometric authentication based on the time series data; performing the biometric authentication on the subject; An information processing method comprising:

[0131] (Appendix 9) On the computer, acquiring time series data representing a displacement of a position of a person included in a captured image of a predetermined area in a three-dimensional coordinate system; determining whether the person is a subject of biometric authentication based on the time series data; performing the biometric authentication on the subject; A storage medium storing a program for executing an information processing method having the above-mentioned steps. [Explanation of symbols]

[0132] 1, 2, 3, 4... Iris Recognition System 5. Face Recognition System 10 Authentication device 11. Overall bird's-eye view image acquisition section 12 Face detection section 13. Time series data acquisition section 14...Storage section 15. Certification Subject Judgment Section 16 Iris camera setting section 17 Biometric image acquisition unit 18. Biometric authentication section 19. Learning Department 20. Imaging device 21. Overall view camera 22a, 22b, 22c...Iris camera 30. Human presence sensor 40...Notification device 41. Display 42 LED 43 Speaker 50··Projector 100.... Information processing device 100A...Acquisition section 100B...Judgment section 100C···Certification section 101...Information Department

Claims

1. an acquisition unit that acquires time series data representing a displacement of a position of a person included in a captured image of a predetermined area in a three-dimensional coordinate system; a determination unit that determines whether the person is a subject of biometric authentication based on the time-series data; an authentication unit that performs the biometric authentication on the subject; Equipped with the determination unit calculates a score indicating a likelihood that the person is the target person based on the time-series data, and determines whether the person is the target person based on the score; The acquisition unit acquires the time-series data related to the person for each of a plurality of cameras based on the captured images captured by the respective cameras; The information processing device, wherein the determination unit integrates the scores calculated for each of the cameras.

2. the determination unit determines that the person moving toward the photographing position is the subject by comparing the displacement with a reference displacement at the time of moving to the photographing position of a biometric image used for the biometric authentication.

2. An information processing device according to claim 1.

3. The determination unit determines whether or not the person is the target person based on the time series data and a line of sight or a face direction of the person detected from the captured image.

3. An information processing device according to claim 1 or 2.

4. a learning unit that learns a model for determining whether or not a person is the target person based on the time series data on a first person determined to be the target person and the time series data on a second person determined not to be the target person; The information processing device according to claim 1 , further comprising:

5. a guidance unit that provides the person with a guidance image regarding the biometric authentication when the person is determined to be the target person; The information processing device according to claim 1 , further comprising:

6. acquiring time series data representing a displacement of a position of a person included in a captured image of a predetermined area in a three-dimensional coordinate system; determining whether the person is a subject of biometric authentication based on the time series data; performing the biometric authentication on the subject; Equipped with In the determining step, a score indicating a likelihood that the person is the target person is calculated based on the time-series data, and whether or not the person is the target person is determined based on the score; In the acquiring step, the time-series data related to the person is acquired for each of a plurality of cameras based on the captured images captured by the respective cameras; An information processing method, wherein in the determining step, the scores calculated for each of the cameras are integrated.

7. On the computer, acquiring time series data representing a displacement of a position of a person included in a captured image of a predetermined area in a three-dimensional coordinate system; determining whether the person is a subject of biometric authentication based on the time series data; performing the biometric authentication on the subject; having In the determining step, a score indicating a likelihood that the person is the target person is calculated based on the time-series data, and whether or not the person is the target person is determined based on the score; In the acquiring step, the time-series data related to the person is acquired for each of a plurality of cameras based on the captured images captured by the respective cameras; The program for executing an information processing method, wherein in the determining step, the scores calculated for each of the cameras are integrated.

Images