Biometric information acquisition device

Abstract

To provide a biological information acquisition device that supports acquisition of a plurality of different pieces of biological information suitable for biometric authentication in a non-contact state.SOLUTION: A biological information acquisition device 1 includes: a housing 11; a first photographing unit (first camera CM1) that photographs an image of a first biological part of an object person in a non-contact state; a second photographing unit (second camera) that photographs an image of a second biological part of the object person different from the first biological part in a non-contact state; and an acquisition unit that acquires a first photographed image obtained by the first photographing unit and a second photographed image obtained by the second photographing unit; The housing 11 includes: a first opening 16 that is formed by cutting out a part of the housing to form a photographing space for photographing a second biological part; lighting 15B that illuminates the photographing space inside the housing; and a light path change unit 14 that changes a light path of light reflected by the second biological part in the photographing space and makes the light incident on the second photographing unit.SELECTED DRAWING: Figure 2

Description

Technical Field

[0004] , , , , ,

[0005] , , ,

[0003] , , ,

[0001] The present disclosure relates to a biological information acquisition device.

Background Art

[0002] In Patent Document 1, when a user is detected within a measurable predetermined range, after acquiring a plurality of biological data from the non-contact user, the acquired plurality of biological data is compared with the pre-registered biological data, and user authentication regarding each biological data is performed. A non-contact multi-biological recognition method is disclosed. In the non-contact multi-biological recognition method, in user authentication regarding each biological data, when the user approaches, biological data regarding the face and iris is acquired from the non-contact user by physical movement of the camera, and the user authentication results regarding the biological data are integrated. When the user authentication results regarding each biological data satisfy a predetermined condition, the user is determined as an authorized user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the configuration of Patent Document 1, in order to acquire each biological data necessary for user authentication, a dedicated camera corresponding to each of the biological data is required, so the manufacturing cost of the multi-biological recognition device is high. Therefore, there is a demand for realizing a non-contact biological authentication system with reduced manufacturing cost using a highly versatile device.

[0005] The present disclosure has been devised in view of the above-described conventional circumstances, and an object thereof is to provide a biological information acquisition device that supports acquisition of a plurality of different biological information suitable for biological authentication in a non-contact state.

Means for Solving the Problems

[0006] This disclosure comprises a housing, a first imaging unit for imaging a first biological region of a subject in a non-contact state, a second imaging unit for imaging a second biological region of the subject different from the first biological region in the non-contact state, and an acquisition unit for acquiring a first image captured by the first imaging unit and a second image captured by the second imaging unit, wherein the housing has a first opening formed by cutting out a part of the housing to create an imaging space for imaging the second biological region, illumination for illuminating the imaging space inside the housing, and an optical path changing unit for changing the optical path of light reflected by the second biological region in the imaging space and directing it into the second imaging unit. A detachable attachment section for attaching and detaching an imaging device is provided on the front side of the housing. to have The imaging device comprises the first imaging unit, the second imaging unit, and the acquisition unit. To provide a device for acquiring biological information. [Effects of the Invention]

[0007] According to this disclosure, it is possible to support the acquisition of multiple different biometric information suitable for biometric authentication in a contactless manner. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of a use case for the biometric authentication device according to Embodiment 1. [Figure 2] Right-side perspective view of a biometric authentication device [Figure 3] A diagram showing an example of the opening and internal space of a biometric authentication device. [Figure 4] Left-side perspective view of a biometric authentication device [Figure 5] Block diagram showing an example of the internal configuration of the biometric authentication device according to Embodiment 1. [Figure 6] Sequence diagram showing an example of the operation procedure of the terminal device in Embodiment 1. [Figure 7] A diagram showing an example of the overall configuration of the biometric authentication system according to Embodiment 2. [Figure 8] Block diagram showing an example of the internal configuration of the biometric authentication system according to Embodiment 2. [Figure 9] Sequence diagram showing an example of the operation procedure of the biometric authentication system according to Embodiment 2. [Modes for carrying out the invention]

[0009] (Background leading to this disclosure) Conventionally, there is a method for acquiring biometric information from multiple different biological parts (e.g., hands, face, etc.) non-contact using a highly versatile terminal device (e.g., smartphone, tablet, etc.). In such cases, the terminal device acquires (images) biometric information from each biological part non-contact by capturing an image of the user's face with a front camera located on the front of the terminal device (monitor side) and capturing an image of the user's hands with a back camera located on the back of the terminal device (opposite the monitor side).

[0010] However, terminal devices had difficulty controlling focus on biological tissues, and sometimes failed to acquire images suitable for obtaining biological information. In particular, the front camera has an even wider angle of view than the rear camera, resulting in greater distortion of the captured images. Therefore, terminal devices had even more difficulty controlling focus on the front camera, making it difficult to acquire images suitable for obtaining biological information.

[0011] Furthermore, when store employees use terminal devices to acquire biometric information from multiple different body parts of a user, it was difficult for the employees to determine the appropriate imaging position for the user's body parts (hands, face, etc.) relative to the terminal device, making it difficult to acquire (image) the user's biometric information without physical contact. In addition, when store employees or users image the user's face with the front camera and the user's fingers with the rear camera in a store, the light from the LED (Light-Emitting Diode) installed near the rear camera would shine into the employee's eyes, causing them discomfort.

[0012] Therefore, in each of the embodiments described below, we will explain an example of a biometric information acquisition device that supports the acquisition of multiple different biometric information suitable for biometric authentication in a contactless state.

[0013] Hereinafter, each embodiment specifically disclosing the configuration and operation of the biological information acquisition device according to the present disclosure will be described in detail while appropriately referring to the drawings. However, detailed descriptions that are more than necessary may be omitted. For example, detailed descriptions of well-known matters and duplicate descriptions of substantially the same configuration may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the attached drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and it is not intended to limit the subject matter described in the claims by these.

[0014] In the following description, the XY plane indicated by the X direction and the Y direction represents the installation surface H0 (see FIG. 2) (horizontal plane) of the biometric authentication device 1, and may be referred to as the left-right direction or the front-back direction. Further, the Z direction represents the direction perpendicular to the installation surface H0 (horizontal plane) of the biometric authentication device 1, and may be referred to as the up-down direction.

[0015] (Embodiment 1) Referring to FIG. 1, a use case example of the biometric authentication device 1 in Embodiment 1 will be described. FIG. 1 is a diagram showing a use case example of the biometric authentication device 1 according to Embodiment 1.

[0016] The biometric authentication device 1 (an example of a biological information acquisition device) according to Embodiment 1 includes a terminal device P1 (an example of an imaging device) and a housing 11. The biometric authentication device 1 is installed in a store that uses biometric authentication, acquires the biological information of a user, registers the acquired biological information of the user, or executes user authentication based on the acquired biological information of the user.

[0017] The housing 11 has the terminal device P1 detachably placed in the recess 12 and an opening 16 into which the hand H of the user can be inserted. The housing 11 includes an optical system 14 and at least one illumination 15 inside the housing 11. The housing 11 illuminates the hand H of the user inserted into the opening 16, and causes the light reflected by the hand H of the user to enter the second camera CM2 of the terminal device P1 through the optical system 14.

[0018] The terminal device P1 has at least two cameras (first camera CM1, second camera CM2 (see Figure 2)) and is implemented by, for example, a smartphone or tablet device. The terminal device P1 is placed in the recess 12 of the housing 11 and images each of several different biological parts of the user.

[0019] Terminal device P1 registers the captured image (biometric information) and performs user authentication using the acquired image (biometric information). Terminal device P1 generates a notification screen (not shown) to notify the user authentication result and displays it on monitor MN.

[0020] Next, the internal structure of the housing 11 will be described with reference to Figures 2 and 3, respectively. Figure 2 is a right-side perspective view of the biometric authentication device 1. Figure 3 is a diagram showing an example of the opening 16 and the internal space 17 of the housing of the biometric authentication device 1. Note that the biometric authentication device 1 shown in Figures 2 and 3 is a perspective view of the inside of the housing 11 of the biometric authentication device 1 from the direction of arrow V1 shown in Figure 1.

[0021] The housing 11 comprises a recess 12, an opening 13, an optical system 14, illuminations 15A and 15B, and an opening 16. However, the configuration of the housing 11 and the arrangement of each component are not limited to this. The housing 11 only needs to be capable of mounting and holding the terminal device P1 and capable of capturing images of the user's fingers by the second camera CM2 of the terminal device P1.

[0022] The housing 11 has a recess 12 (an example of a detachable part) that is inclined by a first predetermined angle θ1 with respect to the vertical direction L0 (Z direction). The recess 12 holds the terminal device P1 and has an opening 13 on the rear side where the second camera CM2 of the terminal device P1 is located, which leads to the inside of the housing 11.

[0023] The first predetermined angle θ1 is set to, for example, 23°. However, the first predetermined angle θ1 is not limited to this and can be set to an angle (for example, 15° to 23°) that allows the first camera CM1 to capture the user's face. As a result, the optical axis L1 of the first camera CM1 is tilted by the first predetermined angle θ1 with respect to the mounting surface H0 of the housing 11.

[0024] An example of a second opening, the opening 13, is formed by cutting out a portion of the recess 12, which is the mounting surface for the terminal device P1. The opening 13 allows the second camera CM2 to image the inside of the housing 11 when the terminal device P1 is mounted on the recess 12. The opening 13 may be formed in a position and size corresponding to the position of the second camera CM2 on the terminal device P1.

[0025] The housing 11 includes an optical system 14 and illuminations 15A and 15B inside the housing.

[0026] Lights 15A and 15B are inserted through the opening 16 and illuminate the user's fingers positioned in the internal space 17 of the housing.

[0027] An example of an optical system 14, which is a section for changing the optical path, is implemented using, for example, a prism, a mirror, etc. The optical system 14 is illuminated by illuminations 15A and 15B, and reflects or refracts the light reflected by the user's fingers by 90°, causing it to enter the second camera CM2 through the aperture 13.

[0028] When implemented using a prism, the optical system 14 is positioned on the back of the mounting surface of the terminal device P1 in the recess 12. By positioning the optical system 14 such that the distance between it and the second camera CM2 of the terminal device P1 mounted in the recess 12 is 0 (zero), reflections of illumination light from the lights 15A and 15B can be suppressed more effectively.

[0029] On the other hand, if implemented using a mirror, the optical system 14 should be positioned so that it can reflect the light reflected by the user's fingers at a 90° angle and enter the second camera CM2 through the aperture 13.

[0030] Here, the formation positions of the aperture 13 and the optical system 14 will be described. In order to acquire an image (hereinafter referred to as "finger image") that is in focus on the user's fingers (e.g., veins, fingerprints, palm prints, etc.), it is desirable that the housing 11 be designed such that the depth of field (imaging distance) of the second camera CM2 and the imaging area 17A within the housing space 17 into which the user's fingers are inserted overlap, as shown in Figures 2 to 4. In other words, the aperture 13 is formed at a position (for example, within the depth of field) such that the optical path length of the incident light reflected from the user's fingers by the optical system 14 and incident on the second camera CM2 is within the depth of field of the second camera CM2, resulting in an imaging distance that allows for the acquisition of a finger image suitable for biometric authentication. Therefore, in this embodiment 1, the aperture 13 and the optical system 14 are shown as being formed on the upper part (upper surface side) of the recess 12, but the embodiment is not limited to this.

[0031] An example of a first opening, the opening 16 is formed extending from the front to both sides of the housing 11 and leads to the internal housing space 17 inside the housing 11. The opening 16 is formed such that the opening height H1 on the front side into which the user inserts their hand H is greater than the opening height H2 on the rear side. This makes it easier for the user to insert their hand H from the insertion opening (i.e., the front side of the housing 11) toward the internal housing space 17 (i.e., the rear side of the housing).

[0032] The internal space 17 of the housing is a space into which the user's fingertip, which is the object to be imaged, can be inserted. The internal space 17 of the housing includes an imaging area 17A (space), which is a region (space) corresponding to the depth of field of the second camera CM2 of the terminal device P1.

[0033] The imaging region 17A, as an example of an imaging space, is a region (space) determined based on the depth of field of the second camera CM2 and the position of the optical system 14. It goes without saying that the imaging region 17A shown in Figure 3 is just one example and is not limited to it.

[0034] The lower opening surface 18 is the lower surface of the space into which the user's hand H can be inserted through the opening 16. The lower opening surface 18 is formed at a second predetermined angle θ2 toward the front side of the housing 11 in order to guide the direction in which the user's hand H is inserted into the internal space 17 of the housing.

[0035] As described above, by holding the terminal device P1 with the housing 11 tilted by a first predetermined angle θ1, the system can simultaneously or continuously capture images of the user's face and the fingertips of the user's hand H.

[0036] Next, the internal structure of the housing 11 will be described with reference to Figure 4. Figure 4 is a left-side perspective view of the biometric authentication device 1. Note that the biometric authentication device 1 shown in Figure 4 is a perspective view of the inside of the housing 11 of the biometric authentication device 1 as seen from arrow V2 shown in Figure 1.

[0037] The enclosure 11 is equipped with two lights 15A and 15B in the depth direction of the enclosure 11 (from the front to the back). Note that at least one light is sufficient.

[0038] Lights 15A and 15B each illuminate the user's fingertip inserted into the imaging area 17A of the internal space 17 of the housing. This allows the second camera CM2 of the terminal device P1 to acquire a finger image suitable for biometric authentication.

[0039] Next, an example of the internal structure of the terminal device P1 will be described with reference to Figure 5. Figure 5 is a block diagram showing an example of the internal configuration of the biometric authentication device 1 according to Embodiment 1.

[0040] Note that the internal structure of the biometric authentication device 1 shown in Figure 5 is an example and is not limited thereto. For example, the biometric information database DB1 may be implemented by an external storage medium connected to the terminal device P1 in a data communication manner. Also, for example, the monitor 23 may be implemented by an external monitor connected to the terminal device P1 in a data communication manner.

[0041] The terminal device P1 includes a communication unit 20, a processor 21, a memory 22, a first camera CM1 (an example of a first imaging unit), a second camera CM2 (an example of a second imaging unit), a monitor 23, and a biological information database DB1. Note that the communication unit 20 is not mandatory and may be omitted.

[0042] The communication unit 20 may be connected wirelessly or via wired communication to other terminal devices installed in the same or other locations that share user information, such as the same store or affiliated stores. Wireless communication here refers to, for example, short-range wireless communication such as Bluetooth®, NFC®, or communication via a wireless LAN (Local Area Network) such as Wi-Fi®.

[0043] The processor 21 is configured using, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an FPGA (Field Programmable Gate Array), and works in cooperation with the memory 22 to perform various processing and control functions. Specifically, the processor 21 refers to the programs and data held in the memory 22 and executes those programs to realize various functions such as capturing finger images and facial images, and acquiring biometric information (fingerprint features, facial features, etc.).

[0044] As an example of an acquisition unit, the processor 21 generates and outputs control instructions to the first camera CM1 and the second camera CM2 to start imaging of the biological body when the user requests the acquisition of biological information or the initiation of biometric authentication. The processor 21 acquires the captured images output from the first camera CM1 and the second camera CM2, respectively.

[0045] The processor 21 detects and extracts the region showing the user's face or a part of their face (e.g., eyes, nose, mouth, etc.) from the face image output from the first camera CM1, which captures the user's face, and generates a face image. If the biometric information used for biometric authentication is the iris, the processor 21 may detect and extract the region showing the biometric part corresponding to this biometric information (e.g., eyes) from the captured image.

[0046] The processor 21 calculates an evaluation value indicating whether the face (e.g., eyes, nose, mouth, etc.) in the generated face image is in focus, that is, whether the face image is suitable for face recognition. The processor 21 determines whether the calculated evaluation value is equal to or greater than a first threshold, and if it determines that the evaluation value is equal to or greater than the first threshold, it performs face registration or face recognition based on the face image or features that indicate the user's individuality extracted from the face image (hereinafter referred to as "face features").

[0047] When the processor 21 transitions to facial recognition processing, it compares the extracted facial features with the facial features of multiple users that have been pre-registered in the biometric information database DB1, and then performs facial recognition.

[0048] Furthermore, the processor 21 detects a fingertip region including the first joint of at least one fingertip from the captured image output from the second camera CM2, which captures the user's entire hand or at least one finger. The processor 21 then crops the detected fingertip region to generate a finger image. Note that if the biometric information used for biometric authentication is veins, palm prints, etc., the processor 21 may detect and crop a region from the captured image that shows the biometric part corresponding to this biometric information (for example, a predetermined joint of the finger, the palm, etc.).

[0049] The processor 21 calculates an evaluation value indicating whether the fingerprint (i.e., biological part) in the generated finger image is in focus, that is, whether the finger image is suitable for fingerprint authentication. The processor 21 determines whether the calculated evaluation value is above the second threshold, and if it determines that the evaluation value is above the second threshold, it performs fingerprint registration or fingerprint authentication based on the finger image or features that indicate the user's individuality extracted from the finger image (hereinafter referred to as "fingerprint features").

[0050] When the processor 21 proceeds to the fingerprint authentication process, it compares the extracted fingerprint features with the fingerprint features of multiple users pre-registered in the biometric information database DB1 and performs fingerprint authentication. The processor 21 may compare only the fingerprint features of the same finger (e.g., middle finger, index finger, etc.) as the finger corresponding to the extracted fingerprint features from the fingerprint features of multiple users registered in the biometric information database DB1.

[0051] The processor 21 generates a notification screen (not shown) to inform the user of the registration result or authentication result based on the registration results of face registration and fingerprint registration, or the authentication results of face authentication and fingerprint authentication, and outputs it to the monitor 23 for display. The notification of the registration result or authentication result may also be given via voice notification.

[0052] Memory 22 includes, for example, RAM (Random Access Memory) used as work memory when executing each process of the processor 21, and ROM (Read Only Memory) which stores programs and data that define the operation of the processor 21. Data or information generated or acquired by the processor 21 is temporarily stored in RAM. Programs that define the operation of the processor 21 are written in ROM. Memory 22 stores a first threshold value for facial image evaluation values, a second threshold value for finger image evaluation values, and so on.

[0053] Each of the first camera CM1 and the second camera CM2 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is a solid-state image sensor, such as a CCD (Charged-Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), which converts the optical image formed on the imaging surface into an electrical signal. Each of the first camera CM1 and the second camera CM2 starts the imaging process based on an imaging start control instruction input by the processor 21. Each of the first camera CM1 and the second camera CM2 outputs the captured image to the processor 21.

[0054] The monitor 23 is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence). The monitor 23 outputs and displays captured images taken by the first camera CM1 and the second camera CM2, respectively, or various notification screens generated by the processor 21.

[0055] The operation unit 231 is implemented as a touch panel for the monitor 23 described above, and can accept user operations such as application startup operations and user information input operations, as well as operations by people such as store employees, and outputs the results of the input operations to the processor 21. The operation unit 231 may also be equipped with a microphone (not shown) and can accept voice input operations based on the voice of the user or store employee.

[0056] The biometric information database DB1 is a so-called storage system, configured using a storage medium such as flash memory, HDD (Hard Disk Drive), or SSD (Solid State Drive). The biometric information database DB1 stores (registers) and manages biometric information such as facial images, finger images, facial features, or fingerprint features, as well as user information, for each user. The biometric information database DB1 may be externally connected to at least one terminal device P1 and configured as a separate external storage system.

[0057] Next, the operation procedure of terminal device P1 will be described with reference to Figure 6. Figure 6 is a sequence diagram showing an example of the operation procedure of terminal device P1 according to Embodiment 1.

[0058] Terminal device P1 accepts an operation to specify whether to register or authenticate the user's biometric information, based on user operation or operation by an employee of a facility, store, etc. where biometric authentication device 1 is installed, and an input operation for user information (e.g., name, age, gender, address, telephone number, user ID, etc.), and starts acquiring biometric information using the first camera CM1 and the second camera CM2, respectively.

[0059] Terminal device P1 captures the user's face using the first camera CM1. Terminal device P1 detects an area from the captured image of the user in which at least a portion of the user's face is visible, and generates a face image by cropping this area.

[0060] Terminal device P1 calculates an evaluation value based on the generated face image, which is an index indicating whether or not the user's face in the face image is in focus (i.e., whether or not the face image is a face image from which biometric information suitable for biometric authentication can be obtained). Terminal device P1 determines whether or not the face image is a registerable finger image based on whether or not the calculated evaluation value is above a first threshold indicating that the user's fingerprint in the finger image is in focus.

[0061] If terminal device P1 determines that the calculated evaluation value is equal to or greater than the first threshold, it acquires a user's facial image (St1) to be used for biometric information registration or authentication. If terminal device P1 determines that the calculated evaluation value is not equal to or greater than the first threshold, it performs re-imaging of the user's face using the first camera CM1.

[0062] Terminal device P1 captures images of the user's fingers using the second camera CM2. From the captured images of the user, terminal device P1 detects a region that shows at least one of the user's fingertips, including a portion of the first joint of the fingertip, and generates a finger image by cropping this region.

[0063] Terminal device P1 calculates an evaluation value based on the generated finger image, which is an index indicating whether or not the user's fingerprint in the finger image is in focus (i.e., whether or not the finger image is a finger image from which biometric information suitable for biometric authentication can be obtained). Terminal device P1 determines whether or not the finger image is a registrable finger image based on whether or not the calculated evaluation value is above a second threshold indicating that the user's fingerprint in the finger image is in focus.

[0064] If terminal device P1 determines that the calculated evaluation value is equal to or greater than the second threshold, it acquires an image of the user's finger to be used for biometric information registration or authentication (St2). If terminal device P1 determines that the calculated evaluation value is not equal to or greater than the second threshold, it performs re-imaging of the user's finger using the first camera CM1.

[0065] Terminal device P1 initiates registration of the user's biometric information or biometric authentication based on each of the multiple different biometric pieces of information acquired from the user (in this case, a facial image or a finger image) (St3).

[0066] <Biometric information registration process> When performing the biometric information registration process, terminal device P1 determines, based on the user information entered by the user or employee, whether the user corresponding to the face image is a user already stored (registered) in the biometric information database DB1, and whether the face image of this user is already stored (registered) in the biometric information database DB1 (i.e., whether the face image can be registered) (St4A).

[0067] In step St4A, if terminal device P1 determines that the user corresponding to the face image is not a user already stored (registered) in the biometric information database DB1, and that the face image of this user is not already stored (registered) in the biometric information database DB1 (St4A, YES), it stores (registers) the user information and the face image in the biometric information database DB1 in association with each other. Terminal device P1 may also extract facial features that indicate the user's individuality from the face image, and store (register) the extracted facial features and the user information in the biometric information database DB1 in association with each other.

[0068] On the other hand, in the processing of step St4A, terminal device P1 determines that the user corresponding to the face image is a user that has already been stored (registered) in the biometric information database DB1, and that the face image of this user has already been stored (registered) in the biometric information database DB1 (St4A, NO), then generates a notification screen (not shown) that notifies that the registration result of the user's biometric information is "NG" and outputs it to monitor 23 (St6A).

[0069] Furthermore, terminal device P1 determines, based on user information entered by the user or employee, whether or not a finger image of the same finger (e.g., middle finger, ring finger, etc.) corresponding to the finger image has already been stored (registered) in the biometric information database DB1 (i.e., whether or not a finger image can be registered) (St5A).

[0070] If, in the process of step St5A, terminal device P1 determines that the finger image corresponding to the finger image has not already been stored (registered) in the biometric information database DB1 (St5A, YES), it further associates the finger image with user information and stores (registers) it in the biometric information database DB1. In addition, terminal device P1 may extract fingerprint features that indicate the individuality of the user's finger from the finger image and further associate the extracted fingerprint features with user information and store (register) them in the biometric information database DB1.

[0071] Terminal device P1 generates a notification screen (not shown) indicating that the user's biometric information registration result is "OK" and outputs it to monitor 23 (St7A).

[0072] On the other hand, if terminal device P1 determines in step St5A that the finger image corresponding to the finger image has already been stored (registered) in the biometric information database DB1 (St5A, NO), it generates a notification screen (not shown) informing the user that the registration result of the biometric information is "NG" and outputs it to monitor 23 (St6A).

[0073] <Biometric authentication processing of biometric information> When performing biometric authentication processing of biometric information, terminal device P1 extracts the user's facial features from the acquired face image and compares the extracted facial features with the facial features of multiple users stored (registered) in the biometric information database DB1 (St4B).

[0074] If terminal device P1 determines in step St4B that there are facial features identical or similar to the extracted facial features (St4B, YES), it determines that the user's facial recognition result is "OK".

[0075] On the other hand, if terminal device P1 determines in step St4B that there are no facial features identical or similar to the extracted facial features (St4B, NO), it determines that the user's facial recognition result is "NG," generates a notification screen (not shown) informing the user that the user's facial recognition result is "NG," and outputs it to monitor 23 (St6B). Alternatively, if terminal device P1 determines that the calculated facial image evaluation value is not equal to or greater than the first threshold, it may determine that the user's facial recognition result is "NG," generate a notification screen (not shown) informing the user that the user's facial recognition result is "NG," and output it to monitor 23.

[0076] Terminal device P1 extracts the user's fingerprint features from the acquired finger image and compares the extracted fingerprint features with the fingerprint features of multiple users stored (registered) in the biometric information database DB1 (St5B). Terminal device P1 may perform a process to narrow down the fingerprint features to be compared based on the finger information corresponding to the finger image. This reduces the processing load required for the fingerprint feature comparison process and shortens the processing time for terminal device P1.

[0077] If, in the processing of step St5B, terminal device P1 determines that there are fingerprint features identical or similar to the extracted fingerprint features (St5B, YES), and determines that the user's fingerprint authentication result is "OK", it may generate a notification screen (not shown) indicating that user authentication is "OK" and output it to monitor 23 (St18B). Furthermore, if terminal device P1 determines that both the facial authentication result of the facial authentication process and the fingerprint authentication result of the fingerprint authentication process are "OK", and that the authenticated user is the same person, it may generate a notification screen (not shown) indicating that user authentication is "OK" and output it to monitor 23 (St18B).

[0078] Terminal device P1 generates a notification screen (not shown) indicating that the biometric authentication result for the user's biometric information is "OK," and outputs it to monitor 23 (St7B).

[0079] On the other hand, if terminal device P1 determines in step St5B that there are no fingerprint features identical or similar to the extracted fingerprint features (St5B, NO), it determines that the user's fingerprint authentication result is "NG" and generates a notification screen (not shown) informing the user that the biometric authentication result is "NG" and outputs it to monitor 23 (St6B).

[0080] As described above, the terminal device P1 in Embodiment 1, when mounted on the housing 11, becomes capable of acquiring multiple different biometric information. Furthermore, the terminal device P1 can perform multiple different biometric authentications that have been acquired.

[0081] (Embodiment 2) The biometric authentication device 1 according to Embodiment 1 shows an example in which a terminal device P1 acquires multiple different biometric information and performs multiple different biometric authentications. The biometric authentication system 100 according to Embodiment 2 describes an example in which the acquisition of multiple different biometric information and the performance of multiple different biometric authentications are performed by different devices.

[0082] The internal configuration of the biometric information acquisition device 1A according to Embodiment 2 is the same as the internal configuration of the biometric authentication device 1 according to Embodiment 1. Therefore, in describing the biometric authentication system 100 according to Embodiment 2, the same reference numerals are assigned to the same components as those described in Embodiment 1 (the housing 11 and the terminal device P1), and their descriptions are omitted. Instead, the different functions realized by each component are described.

[0083] Next, with reference to Figure 7, an example of the overall configuration of the biometric authentication system 100 according to Embodiment 2 will be described. Figure 7 is a diagram showing an example of the overall configuration of the biometric authentication system 100 according to Embodiment 2.

[0084] The biometric authentication system 100 according to Embodiment 2 includes a biometric information acquisition device 1A, which includes a housing 11 and a terminal device P1A, a server S1, and a biometric information database DB2. The biometric authentication system 100 according to Embodiment 2 acquires multiple different biometric information using the biometric information acquisition device 1A installed in a place where biometric authentication is used, such as a store, and performs multiple different biometric authentications using the server S1, which is connected to the biometric information acquisition device 1A via a network for data communication. The biometric information database DB2 may be configured integrally with the server S1.

[0085] Next, with reference to Figure 8, an example of the internal configuration of the biometric authentication system 100 according to Embodiment 2 will be described. Figure 8 is a block diagram showing an example of the internal configuration of the biometric authentication system 100 according to Embodiment 2.

[0086] The biological information acquisition device 1A includes a housing 11 and a terminal device P1A mounted on the housing 11. The terminal device P1A includes a communication unit 20A, a processor 21A, a memory 22, a first camera CM1, a second camera CM2, and a monitor 23A.

[0087] The communication unit 20A is connected to the server S1 via a network NW, enabling wired or wireless communication. The communication unit 20A transmits biometric information, control commands (electrical signals), etc., output from the processor 21A to the server S1. The communication unit 20A outputs biometric registration result information, biometric authentication result information, etc., transmitted from the server S1 to the processor 21A.

[0088] A processor 21A, as an example of an acquisition unit, is configured using, for example, a CPU, GPU, or FPGA, and works in cooperation with memory 22 to perform various processing and control. Specifically, the processor 21A references programs and data held in memory 22 and executes those programs to realize the function of acquiring biological information.

[0089] The processor 21A extracts a region from the image output from the first camera CM1 that shows at least a portion of the user's face and generates a face image. The processor 21A extracts a region from the image output from the second camera CM2 that shows at least one fingertip and the first joint of that fingertip and generates a finger image. The processor 21A associates the generated face image and finger image with information about the user and a control command requesting registration of the user's biometric information or biometric authentication, outputs them to the communication unit 20A, and has it sent to the server S1.

[0090] The processor 21A may extract facial features from the generated face image and fingerprint features from the finger image, associate the extracted facial features and fingerprint features with user information and a control command requesting registration of the user's biometric information or biometric authentication, output them to the communication unit 20A, and have it sent to the server S1.

[0091] Alternatively, the processor 21A may generate finger information corresponding to a finger image or fingerprint features, associate the finger information with the finger image or fingerprint features, and send it to the server S1. This allows the biometric authentication system 100 to manage biometric information for each finger and to identify fingers and perform biometric authentication.

[0092] Monitor 23A is configured using, for example, an LCD or an organic EL display. Monitor 23A outputs and displays captured images taken by the first camera CM1 and the second camera CM2, as well as various notification screens transmitted from the server S1.

[0093] Server S1 performs either a user biometric information registration process or a user biometric authentication process based on the user's biometric information transmitted from the biometric information acquisition device 1A. Server S1 includes a communication unit 30, a processor 31, and memory 32.

[0094] The communication unit 30 is connected to the biometric information acquisition device 1A via a network NW, enabling wired or wireless communication. The communication unit 30 is also connected to the biometric information database DB2, enabling wired or wireless communication.

[0095] The communication unit 30 outputs the user's biometric information (e.g., face image and finger image, facial features and fingerprint features, etc.) transmitted from the biometric information acquisition device 1A to the processor 31. The communication unit 30 also transmits the biometric information registration result or biometric authentication result information output from the processor 31 to the biometric information acquisition device 1A.

[0096] The processor 31 is configured using, for example, a CPU, GPU, or FPGA, and works in cooperation with the memory 32 to perform various processes and controls. Specifically, the processor 31 refers to the programs and data held in the memory 22 and executes those programs to realize the function of acquiring biological information.

[0097] Memory 32 includes, for example, RAM as work memory used when executing each process of the processor 31, and ROM which stores programs and data that define the operation of the processor 31. Data or information generated or acquired by the processor 31 is temporarily stored in RAM. Programs that define the operation of the processor 31 are written in ROM. Memory 32 stores a first threshold value for facial image evaluation values, a second threshold value for finger image evaluation values, and so on.

[0098] Next, the operation procedure of the biometric authentication system 100 will be described with reference to Figure 9. Figure 9 is a sequence diagram showing an example of the operation procedure of the biometric authentication system 100 according to Embodiment 2.

[0099] The terminal device P1A of the biometric information acquisition device 1A accepts an operation to specify whether to register the user's biometric information or perform biometric authentication, based on a user operation or an operation by an employee of a facility, store, etc. where the biometric authentication device 1 is installed, and an input operation for information about the user (e.g., name, age, gender, address, telephone number, user ID, etc.), and starts acquiring biometric information using the first camera CM1 and the second camera CM2, respectively.

[0100] Terminal device P1A captures the user's face using the first camera CM1. Terminal device P1A detects an area from the captured image of the user in which at least a portion of the user's face is visible, and generates a face image by cropping this area (St11).

[0101] Terminal device P1A calculates an evaluation value based on the generated face image, which is an index indicating whether or not the user's face in the face image is in focus (i.e., whether or not the face image is a face image from which biometric information suitable for biometric authentication can be obtained) (St11). Terminal device P1A determines whether or not the face image is a registrable face image based on whether or not the calculated evaluation value is above a first threshold indicating that the user's face in the face image is in focus (St11).

[0102] If terminal device P1A determines that the calculated evaluation value is equal to or greater than the first threshold, it acquires the user's facial image to be used for biometric information registration or authentication (St11). If terminal device P1A determines that the calculated evaluation value is not equal to or greater than the first threshold, it performs re-imaging of the user's face using the first camera CM1.

[0103] Terminal device P1A captures images of the user's fingers using the second camera CM2. From the captured images of the user, terminal device P1A detects a region that shows at least one of the user's fingers, including a portion of the first joint of the finger, and generates a finger image by cropping this region (St12).

[0104] Terminal device P1A calculates an evaluation value (St12) based on the generated finger image, which is an index indicating whether or not the user's fingerprint in the finger image is in focus (i.e., whether or not the finger image is a finger image from which biometric information suitable for biometric authentication can be obtained). Terminal device P1A determines whether or not the finger image is a registrable finger image (St12) based on whether or not the calculated evaluation value is above a second threshold indicating that the user's fingerprint in the finger image is in focus.

[0105] If terminal device P1A determines that the calculated evaluation value is equal to or greater than the second threshold, it acquires an image of the user's finger to be used for biometric information registration or biometric authentication (St12). If terminal device P1A determines that the calculated evaluation value is not equal to or greater than the second threshold, it performs re-imaging of the user's finger using the first camera CM1.

[0106] Terminal device P1A extracts and obtains facial features from the acquired facial image (St13). Terminal device P1 extracts and obtains fingerprint features from the acquired finger image (St14). Note that steps St13 and St14 may be omitted if terminal device P1A acquires the facial image and finger image as biometric information and sends them to server S1.

[0107] Terminal device P1A generates a control command requesting either biometric registration or biometric authentication based on an operation specifying whether to perform biometric registration or biometric authentication of the user. Terminal device P1A associates the generated control command with information about the user and the user's biometric information and sends it to server S1 (St15).

[0108] Server S1 initiates registration of the user's biometric information or biometric authentication based on control commands transmitted from terminal device P1A. The following description will explain an example in which terminal device P1A transmits both a face image and a finger image.

[0109] <Biometric information registration process> Server S1 determines, based on information about the user, whether the user corresponding to the face image is a user already stored (registered) in the biometric information database DB2, and whether the face image of this user is already stored (registered) in the biometric information database DB2 (i.e., whether the face image can be registered) (St16A).

[0110] If server S1 determines that the user corresponding to the face image is not a user already stored (registered) in the biometric information database DB2, and that the user's face image is not already stored (registered) in the biometric information database DB2, it associates the user information with the face image and stores (registers) it in the biometric information database DB2 (St16A).

[0111] Furthermore, server S1 determines, based on user information entered by the user or employee, whether or not a finger image of the same finger (e.g., middle finger, ring finger, etc.) corresponding to the acquired finger image is already stored (registered) in the biometric information database DB2 (i.e., whether or not the finger image can be registered) (St17A).

[0112] If server S1 determines that the finger image corresponding to the acquired finger image is not already stored (registered) in the biometric information database DB2, it further associates the finger image with the face image and user information and stores (registers) it in the biometric information database DB2 (St18A).

[0113] When the registration of both the face image and the finger image is complete, server S1 generates biometric registration result information (St18A) that notifies the user that the registration result of the user's biometric information is "OK". Alternatively, if only one of the face image or the finger image is registered, server S1 may generate biometric registration result information that notifies the user that only one of the two has been registered as the user's biometric information. This allows server S1 to accept additional and updated registrations of face images and finger images.

[0114] Furthermore, if server S1 determines that both the face image and the finger image have already been stored (registered) in the biometric information database DB2, it generates biometric information registration result information (St18A) that notifies the user that the registration result of the user's biometric information is "NG".

[0115] <Biometric authentication processing of biometric information> Server S1 extracts facial features from the acquired facial image based on control commands transmitted from terminal device P1A. Server S1 then refers to the biometric information database DB2 and compares the extracted facial features with the facial features of multiple users stored (registered) in the biometric information database DB2 (St16B).

[0116] Server S1 determines that the user's face recognition result is "OK" if it finds facial features that are identical or similar to the extracted facial features (St16B).

[0117] On the other hand, if server S1 determines that there are no facial features identical or similar to the acquired facial features, it determines that the user's facial recognition result is "NG" (St16B). In such cases, server S1 may omit the processing in step St17A.

[0118] Server S1 extracts fingerprint features from the acquired finger image based on control commands transmitted from terminal device P1A. Server S1 refers to the biometric information database DB2 and compares the acquired fingerprint features with the fingerprint features of multiple users stored (registered) in the biometric information database DB2 (St17B). Server S1 may perform a process to narrow down the fingerprint features to be compared based on finger information transmitted from terminal device P1A or the acquisition of finger information corresponding to the finger image at the time of fingerprint feature extraction. This reduces the processing load required for the fingerprint feature comparison process on terminal device P1 and shortens the processing time.

[0119] Server S1 determines that the user's fingerprint authentication result is "OK" if it determines that there are fingerprint features that are identical or similar to the extracted fingerprint features (St17B).

[0120] On the other hand, if server S1 determines that there are no fingerprint features identical or similar to the extracted fingerprint features, it determines that the user's fingerprint authentication result is "NG" (St17B).

[0121] Server S1 determines that user authentication is OK if both the facial recognition result from the facial recognition process and the fingerprint recognition result from the fingerprint recognition process are "OK", and if it determines that the authenticated user is the same person, and generates biometric authentication result information that notifies that the biometric authentication result is "OK" (St18B).

[0122] On the other hand, if server S1 determines that at least one of the authentication results, either the facial recognition result from the facial recognition process or the fingerprint recognition result from the fingerprint recognition process, is "NG", it determines that user authentication is "NG" and generates biometric authentication result information that notifies that the biometric authentication result is "NG" (St18B).

[0123] Server S1 transmits information regarding the registration results of the user's biometric information, or information regarding the biometric authentication results, to terminal device P1A (St19).

[0124] The terminal device P1A generates a notification screen (not shown) that notifies the user of the biometric information registration result or biometric authentication result based on the biometric information registration result or biometric authentication result information transmitted from the server S1, and outputs and displays it on the monitor 23A (St20).

[0125] As described above, the biometric authentication system 100 according to Embodiment 2 can register multiple different biometric information and perform multiple different biometric authentications using each of the multiple different biometric information.

[0126] As described above, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 are examples of biometric information acquisition devices and include a housing 11, a first camera CM1 (an example of a first imaging unit) that images a first biological part of a user (an example of a subject) in a non-contact state, a second camera CM2 (an example of a second imaging unit) that images a second biological part of the subject that is different from the first biological part in a non-contact state, and a processor 21 (an example of an acquisition unit) that acquires a first image captured by the first camera CM1 and a second image captured by the second camera CM2. The housing 11 includes an opening 16 (an example of a first opening) formed by cutting out a part of the housing 11 to create an imaging area 17A (an example of an imaging space) for imaging the second biological part, illuminations 15A and 15B that illuminate the imaging area 17A inside the housing 11, and an optical system 14 (an example of an optical path changing unit) that changes the optical path of light reflected by the second biological part in the imaging area 17A and causes it to enter the second camera CM2.

[0127] As a result, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 are mounted on a housing 11 and can acquire multiple different biometric information using terminal devices P1 and P1A, each having two cameras (a first camera CM1 and a second camera CM2). The biometric information referred to here may be an captured image or a feature quantity extracted from an captured image.

[0128] Furthermore, the first camera CM1 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 captures images of a first biological region outside the housing 11. As a result, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 can acquire multiple different biometric information from both inside and outside the housing 11.

[0129] Furthermore, the optical axis L1 of the first camera CM1 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 has an angle of 15° to 23° with respect to the mounting surface H0 of the housing 11 in a side view of the housing 11 (for example, in the lines V1, V2). As a result, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 can capture images of at least a portion of the user's face located diagonally above the front of the housing 11 using the first camera CM1, and at least a portion of the user's fingers inserted into the housing 11 using the second camera CM2.

[0130] Furthermore, the first camera CM1 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 captures at least a portion of the user's face, and the second camera CM2 captures at least a portion of the user's fingers. As a result, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 can capture at least a portion of the user's face, which is positioned diagonally above the front of the housing 11, using the first camera CM1, and at least a portion of the user's fingers inserted into the housing 11, using the second camera CM2.

[0131] Furthermore, the housing 11 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 further includes a recess 12 (an example of a detachable part) on the front side of the housing 11, to which terminal devices P1 and P1A (an example of an imaging device) can be attached and detached. The terminal devices P1 and P1A each include a first camera CM1, a second camera CM2, and a processor 21. As a result, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 can acquire multiple different biometric information by placing the terminal devices P1 and P1A on the housing 11.

[0132] Furthermore, the recess 12 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 has an opening 13 (an example of a second opening). The optical system 14 causes light with a modified optical path to enter the second camera CM2 through the opening 13. As a result, the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 can adjust the optical path length of the light entering the second camera CM2 (the distance between the user's fingers and the image sensor of the second camera CM2) so that it is within the depth of field of the second camera CM2, and the height of the housing 11 in the vertical direction (Z-axis direction) can be reduced.

[0133] Furthermore, the optical system 14 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 is a prism. This allows the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 to adjust the optical path length of the light incident on the second camera CM2 (the distance between the user's fingers and the image sensor of the second camera CM2).

[0134] Furthermore, the optical system 14 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 is a mirror. This allows the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 to adjust the optical path length of the light incident on the second camera CM2 (the distance between the user's fingers and the image sensor of the second camera CM2).

[0135] Furthermore, the opening 16 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 is formed such that, in a side view of the housing 11 (for example, in the lines V1, V2), the opening height decreases from the front to the back of the housing 11. This makes it easier for the user's fingers to be inserted from the front side of the housing 11 toward the internal space 17 (imaging area 17A) of the housing in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2.

[0136] Furthermore, the opening 16 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2 is formed such that, in a side view of the housing 11 (for example, in the lines V1, V2), the angle between the mounting surface H0 of the housing 11 and the lower surface 18 of the opening 16 decreases as you move from the front to the back of the housing 11. This makes it easier for the user's fingers to be inserted into the internal space 17 (imaging area 17A) of the housing from the front side of the housing 11 in the biometric authentication device 1 in Embodiment 1 and the biometric information acquisition device 1A in Embodiment 2.

[0137] Although various embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It will be clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these will also be understood to fall within the technical scope of this disclosure. Furthermore, the components of the various embodiments described above can be combined arbitrarily without departing from the spirit of the invention. [Industrial applicability]

[0138] This disclosure is useful as a presentation of a biometric information acquisition device that supports the acquisition of multiple different biometric information suitable for biometric authentication in a contactless manner. [Explanation of Symbols]

[0139] 1. Biometric authentication device 1A Biometric Information Acquisition Device 11 cabinets 12 recesses 13 Aperture 14 Optical system 15A,15B Lighting 16 Opening 17. Interior space of the enclosure 17A Imaging area 18 Opening bottom surface 20,30 Communications Department 21,31 processors 22,32 memory 23,23A Monitor CM1 Camera 1 CM2 Second Camera DB1, DB2 Biological Information Databases S1 Server P1, P1A Terminal Device

Claims

1. The casing and A first imaging unit that images a first biological region of the subject in a non-contact manner, A second imaging unit that images a second biological site different from the first biological site of the subject in the aforementioned non-contact state, The system includes an acquisition unit that acquires a first image captured by the first imaging unit and a second image captured by the second imaging unit, The aforementioned enclosure is A portion of the housing is cut out to form a first opening that creates an imaging space for imaging the second biological tissue, Lighting for illuminating the imaging space inside the housing, A light path changing unit that changes the optical path of light reflected by the second biological site in the imaging space and directs it into the second imaging unit, The housing has a detachable section on the front side from which an imaging device can be attached and detached, The imaging device comprises a first imaging unit, a second imaging unit, and an acquisition unit. A device for acquiring biological information.

2. The first imaging unit images the first biological tissue outside the housing. A biological information acquisition device according to claim 1.

3. The optical axis of the first imaging unit has an angle of 15° to 23° with respect to the mounting surface of the housing when viewed from the side of the housing. A biological information acquisition device according to claim 1.

4. The first imaging unit captures an image of at least a portion of the subject's face, The second imaging unit images at least a portion of the subject's fingers. A biological information acquisition device according to claim 1.

5. The aforementioned detachable part has a second opening, The optical path changing unit causes the light whose optical path has been changed to enter the second imaging unit through the second opening. A biological information acquisition device according to claim 1.

6. The aforementioned optical path changing section is a prism. A biological information acquisition device according to claim 1.

7. The optical path changing section is a mirror. A biological information acquisition device according to claim 1.

8. The first opening is formed such that, in a side view of the housing, the opening height decreases from the front to the back of the housing. A biological information acquisition device according to claim 1.

9. The first opening is formed such that, in a side view of the housing, the angle between the mounting surface of the housing and the lower surface of the opening of the first opening decreases as you move from the front to the back of the housing. A biological information acquisition device according to claim 1.

Images