Image processing device, image processing method, and recording medium

Abstract

This image processing device comprises: an acquisition unit 11 that acquires three-dimensional luminance data of skin by projecting and two-dimensionally scanning a light beam onto the skin of a finger, and thereby performing optical coherence tomography; a first extraction unit 12 that extracts a pattern image of the pattern of the skin from the three-dimensional luminance data of the skin; a second extraction unit 213 that extracts the normal direction of the surface of the skin from the three-dimensional luminance data of the skin; and a generation unit 214 that, on the basis of the normal direction, moves the positions of pixels included in a pattern image and generates the post-movement pattern image.

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of an image processing device, an image processing method, and a recording medium. [Background technology]

[0002] A fingerprint imaging device for acquiring a fingerprint image of the epidermis without contact while the fingertip passes through a predetermined place without touching a glass plate or the like, and for use in biometric authentication, is described in Patent Document 1. Patent Documents 2 to 4 describe fingerprint imaging devices for acquiring a fingerprint image of the dermis by performing three-dimensional tomographic imaging of the fingertip using optical coherence tomography (OCT) imaging technology, and for use in biometric authentication. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2009 / 112717 [Patent Document 2] International Publication No. 2016 / 204176 [Patent Document 3] International Publication No. 2020 / 170439 [Patent Document 4] International Publication No. 2021 / 019788 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of this disclosure is to provide an image processing device, an image processing method, and a recording medium that aim to improve upon the techniques described in prior art documents. [Means for solving the problem]

[0005] One aspect of the image processing device includes an acquisition means for acquiring three-dimensional luminance data of the skin generated by performing optical coherence tomography imaging by irradiating a light beam onto the skin of a finger while scanning it two-dimensionally, a first extraction means for extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin, a second extraction means for extracting a normal direction of the skin surface from the three-dimensional luminance data of the skin, and a generation means for shifting positions of pixels included in the pattern image based on the normal direction and generating a shifted pattern image.

[0006] One aspect of the image processing method involves irradiating a light beam onto finger skin while scanning it in two dimensions to perform optical coherence tomography imaging, thereby obtaining three-dimensional brightness data of the skin, extracting a pattern image of the skin pattern from the three-dimensional brightness data of the skin, extracting a normal direction of the skin surface from the three-dimensional brightness data of the skin, and shifting the positions of pixels included in the pattern image based on the normal direction to generate a shifted pattern image.

[0007] In one embodiment of the recording medium, a computer program is recorded to cause a computer to execute an image processing method that includes: irradiating a light beam onto finger skin while scanning it in two dimensions to perform optical coherence tomography imaging, acquiring three-dimensional brightness data of the skin, extracting a pattern image of the skin pattern from the three-dimensional brightness data of the skin, extracting a normal direction of the skin surface from the three-dimensional brightness data of the skin, shifting the positions of pixels included in the pattern image based on the normal direction, and generating a pattern image after the shift. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an image processing apparatus according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of an image processing apparatus according to the second embodiment. [Figure 3] FIG. 3 is a block diagram showing the configuration of an optical coherence tomography imaging apparatus. [Figure 4]FIG. 4 is a diagram illustrating three-dimensional luminance data acquired in optical coherence tomography imaging. [Figure 5] FIG. 5 is a flowchart showing the flow of the image processing operation according to the second embodiment. [Figure 6] FIG. 6 is a conceptual diagram of generation of a pattern image according to the second embodiment. [Figure 7] FIG. 7 is a conceptual diagram of generation of a post-movement pattern image according to the second embodiment. [Figure 8] FIG. 8 is a diagram showing an example of an image processing operation according to the second embodiment. [Figure 9] FIG. 9 is a conceptual diagram of an image processing operation according to the fourth embodiment. [Figure 10] FIG. 10 is a block diagram showing the configuration of an image processing apparatus according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of an image processing device, an image processing method, and a recording medium will be described with reference to the drawings. [1: First embodiment]

[0010] An image processing device, an image processing method, and a recording medium according to a first embodiment will be described below. The image processing device, the image processing method, and the recording medium according to the first embodiment will be described below using an image processing device 1 to which the image processing device, the image processing method, and the recording medium according to the first embodiment are applied. [1-1: Configuration of image processing device 1]

[0011] The configuration of an image processing device 1 in the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the image processing device 1 in the first embodiment.

[0012] As shown in FIG. 1, the image processing device 1 includes an acquisition unit 11, a first extraction unit 12, a second extraction unit 13, and a generation unit 14. The acquisition unit 11 acquires three-dimensional intensity data of the skin by performing optical coherence tomography imaging by irradiating a light beam onto the skin of a finger while scanning it two-dimensionally. The first extraction unit 12 extracts a pattern image of the skin pattern from the three-dimensional intensity data of the skin. The second extraction unit 13 extracts the normal direction of the skin surface from the three-dimensional intensity data of the skin. The generation unit 14 shifts the positions of pixels included in the pattern image based on the normal direction of the skin surface, and generates a shifted pattern image. [1-2: Technical Effects of Image Processing Device 1]

[0013] Since the image processing device extracts a pattern image and a normal direction from the same three-dimensional luminance data, it can generate an appropriate pattern image based on the normal direction, compared to a comparative example in which an image acquired from an imaging device is converted using shape information acquired from a device other than the imaging device. [2: Second embodiment]

[0014] A second embodiment of an image processing device, an image processing method, and a recording medium will be described below. The second embodiment of the image processing device, the image processing method, and the recording medium will be described below using an image processing device 2 to which the second embodiment of the image processing device, the image processing method, and the recording medium is applied. The image processing device 2 may be a computer such as a data processing server, a desktop PC (Personal Computer), a notebook PC, or a tablet PC. [2-1: Configuration of image processing device 2]

[0015] The configuration of the image processing device 2 in the second embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the image processing device 2 in the second embodiment.

[0016] 2, the image processing device 2 includes a calculation device 21 and a storage device 22. The image processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the image processing device 2 does not necessarily include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a field programmable gate array (FPGA). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the figure that is provided in the image processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the image processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing operations to be performed by the image processing device 2 are realized within the arithmetic device 21. That is, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) that the image processing device 2 should perform.

[0018] Fig. 2 shows an example of logical functional blocks realized in the arithmetic device 21 to perform image processing operations. As shown in Fig. 2, an acquisition unit 211 which is a specific example of "acquisition means", a first extraction unit 212 which is a specific example of "first extraction means", a second extraction unit 213 which is a specific example of "second extraction means", and a generation unit 214 which is a specific example of "generation means" are realized in the arithmetic device 21. The operations of the acquisition unit 211, the first extraction unit 212, the second extraction unit 213, and the generation unit 214 will be described later with reference to Figs. 5 to 7.

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the image processing device 2 stores long-term. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0020] The communication device 23 is capable of communicating with devices external to the image processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), or Bluetooth (registered trademark). The communication device 23 may acquire three-dimensional information indicating the three-dimensional shape of the skin from the optical coherence tomography imaging device 100 via the communication network. The optical coherence tomography imaging device 100 will be described later with reference to FIGS. 3 and 4.

[0021] The input device 24 is a device that accepts information input to the image processing device 2 from outside the image processing device 2. For example, the input device 24 may include an operation device that can be operated by an operator of the image processing device 2 (for example, at least one of a keyboard, a mouse trackball, a touch panel, a pointing device such as a pen tablet, a button, etc.). For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the image processing device 2.

[0022] The output device 25 is a device that outputs information to the outside of the image processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. Examples of the display device include a liquid crystal display and an OLED (Organic Light Emitting Diode) display. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. Furthermore, the input device 24 and the output device 25 may be integrally formed as a touch panel.

[0023] Note that the hardware configuration shown in FIG. 2 is an example, and devices other than those shown in FIG. 2 may be added, or some devices may not be provided. Also, some devices may be replaced with other devices having similar functions. Also, some functions of this embodiment may be provided by other devices via a network. The functions of this embodiment may be distributed and realized among multiple devices. Or, for example, the image processing device 2 and the optical coherence tomography imaging device 100 may be an integrated device. In this way, the hardware configuration shown in FIG. 2 can be changed as appropriate.

[0024] Note that, although the following describes the case where three-dimensional brightness data of the skin generated by OCT imaging is used, in this embodiment, the first extraction operation, the second extraction operation, and the generation operation may also be performed using three-dimensional information indicating the three-dimensional shape of the skin generated by other methods. [2-2: Optical coherence tomography imaging device 100]

[0025] The optical coherence tomography imaging apparatus 100 performs optical coherence tomography imaging by irradiating the skin of the finger with a light beam while scanning it two-dimensionally, and generates three-dimensional intensity data of the skin.

[0026] OCT imaging is a technology that uses the interference between object light and reference light to identify the position in the optical axis direction, i.e., the depth direction, of the part of the object being measured where the object light is scattered (light scattering point), and obtain spatially resolved structural data in the depth direction inside the measurement target. OCT technologies include time domain (TD-OCT) and Fourier domain (FD-OCT). In FD-OCT, the interference light spectrum over a wide wavelength band is measured when the object light and reference light interfere, and structural data in the depth direction is obtained by Fourier transforming this. Methods for obtaining the interference light spectrum include spectral domain (SD-OCT), which uses a spectroscope, and swept source (SS-OCT), which uses a wavelength-swept light source.

[0027] Furthermore, by scanning the irradiation position of the object light R3 on the object to be measured in an in-plane direction perpendicular to the depth direction of the object to be measured, it is possible to obtain tomographic structure data that is spatially resolved in the in-plane direction and spatially resolved in the depth direction, i.e., three-dimensional tomographic structure data of the object to be measured. The optical coherence tomography imaging apparatus 100 may be configured to capture an image from above by placing a hand with the palm facing up on a mounting table, or by holding the hand with the palm facing down over the imager. Alternatively, the optical coherence tomography imaging apparatus 100 may be configured to capture an image from above by not placing the hand with the palm facing up on a mounting table. In this case, the mounting table does not need to be included in the optical coherence tomography imaging apparatus 100.

[0028] Fig. 3 is a diagram showing the configuration of an optical coherence tomography imaging device 100. The optical coherence tomography imaging device 100 captures an image of a part of a person's body, such as a finger, based on a three-dimensional measurement technique such as OCT imaging, and generates three-dimensional brightness data including the inside of the skin. Note that the configuration diagram shown in Fig. 3 merely shows one example of a measuring device using OCT imaging, and measuring devices with configurations other than that shown in Fig. 3 may also be used.

[0029] Fig. 3 illustrates an SS-OCT optical coherence tomography imaging apparatus 100. As shown in Fig. 3, the optical coherence tomography imaging apparatus 100 includes a wavelength swept laser light source 110, an optical coherence light receiving unit 120, an optical beam scanning unit 130, and a signal processing control unit 140. The optical coherence light receiving unit 120 includes a circulator 121, a branching / merging unit 122, a reference beam mirror 123, and a balanced optical receiver 124. The optical beam scanning unit 130 includes a fiber collimator 131 and an irradiation optical system 132. The irradiation optical system 132 has a scanning mirror and a lens.

[0030] The wavelength-swept laser source 110 is a laser that emits light while sweeping its wavelength. The wavelength-swept laser source 110 generates and outputs wavelength-swept optical pulses. The wavelength-swept laser source 110 generates optical pulses by sweeping the wavelength from 1250 nm to 1350 nm over a duration of 5 μs. The wavelength-swept laser source 110 generates optical pulses with a repetition frequency of 100 kHz. The optical coherence tomography imaging device 100 generates optical pulses that repeat every 10 μs.

[0031] Light emitted from the wavelength swept laser light source 110 passes through the optical interference light receiving unit 120 and the light beam scanning unit 130, and is irradiated onto and scattered by the measurement object O. The optical interference light receiving unit 120 photoelectrically converts part of the scattered light and outputs an electrical signal. The signal processing control unit 140 processes the electrical signal output by the optical interference light receiving unit 120 into data, and sends the processed data to the image processing device 2. [Operation of the optical interference receiving unit 120]

[0032] The branching / converging device 122 branches the light emitted from the wavelength swept laser light source 110, which has passed through the circulator 121, into an object light R1 and a reference light R2. The object light R1 passes through a fiber collimator 131 and an irradiation optical system 132 and is irradiated onto the measurement object O. The object light R1 scattered by the measurement object O is referred to as object light R3. The object light R3 returns to the branching / converging device 122. The reference light R2 is irradiated onto and reflected by the reference light mirror 123. The reference light R2 reflected by the reference light mirror 123 is referred to as reference light R4. The reference light R4 returns to the branching / converging device 122. The object light R3 scattered from the measurement object O and the reference light R4 reflected by the reference light mirror 123 interfere with each other in the branching / converging device 122, generating interference light R5 and interference light R6. That is, the intensity ratio between the interference light R5 and the interference light R6 is determined by the phase difference between the object light R3 and the reference light R4. The balanced optical receiver 124 has two inputs, and receives the interference light R6 and the interference light R5 that has passed through the circulator 121. The balanced photodetector 124 outputs a voltage corresponding to the difference in intensity between the interference light R5 and the interference light R6. The voltage output by the balanced photodetector 124 is input to the signal processing control unit 140. [A-scan]

[0033] The signal processing control unit 140 generates interference light spectrum data based on information regarding changes in the wavelength of the light emitted by the wavelength swept laser light source 110 and information regarding changes in the intensity ratio between the interference light R5 and the interference light R6. The signal processing control unit 140 performs a Fourier transform on the generated interference light spectrum data to obtain data indicating the intensity of the backscattered light (object light) at different depth positions in the depth direction (also referred to as the "Z direction").

[0034] Hereinafter, the operation of acquiring data indicating the intensity of backscattered light (object light) in the depth direction (Z direction) of the irradiation position of object light R3 in the measurement object O will be referred to as an "A-scan." The signal processing control unit 140 receives an electrical signal with a repetition frequency of 100 kHz as an A-scan trigger signal from the wavelength swept laser light source 110. In response to this, the signal processing control unit 140 generates an A-scan waveform with an optical pulse repetition period of 10 μs. The signal processing control unit 140 generates a waveform indicating the object light backscattering intensity at Nz locations as the A-scan waveform. [B-scan]

[0035] The signal processing control unit 140 controls the irradiation optical system 132 in response to the A-scan trigger signal supplied from the wavelength swept laser light source 110. The irradiation optical system 132 scans the irradiation position of the object light R3 on the measurement object O. The irradiation optical system 132 moves the irradiation position of the object light R3 in the scanning line direction (also referred to as the "fast axis direction of scanning" and the "X direction").

[0036] The signal processing control unit 140 repeatedly performs an A-scan operation for each irradiation position of the object light R3 and connects the A-scan waveforms for each irradiation position of the object light R3. As a result, the signal processing control unit 140 acquires a two-dimensional map of the intensity of the backscattered light (object light) in the scanning line direction (X direction) and the depth direction (Z direction) as a tomographic image. Hereinafter, the operation of repeatedly performing A-scan operations while moving in the scanning line direction (the fast axis direction of scanning, X direction) and connecting the measurement results is referred to as a "B scan." If the irradiation positions of the object light R3 for each B scan are Nx locations, the tomographic image obtained by the B scan is two-dimensional brightness data indicating the object light backscattering intensity at Nz × Nx points. Figure 4(a) illustrates one B-scan tomographic image. [C-scan]

[0037] The illumination optical system 132 moves the irradiation position of the object light R3 not only in the scanning line direction (X direction) but also in a direction perpendicular to the scanning line (also referred to as the "slow axis direction of scanning" or "Y direction"). The signal processing control unit 140 repeatedly performs B-scan operations and connects the B-scan measurement results. As a result, the signal processing control unit 140 acquires three-dimensional tomographic structure data. Hereinafter, the operation of repeatedly performing B-scan operations while moving in a direction perpendicular to the scanning line (Y direction) and connecting the measurement results is referred to as a "C-scan." If the number of B-scans performed per C-scan is Ny, the tomographic structure data obtained by the C-scan is three-dimensional brightness data indicating the backscattering intensity of the object light at Nz × Nx × Ny points. Figure 4(b) is a conceptual diagram of a C-scan operation in which B-scan operations are repeatedly performed while moving in a direction perpendicular to the scanning line (Y direction). Figure 4(c) shows the epidermal curve Z(X, Y) obtained based on the epidermal position Z extraction result extracted for each point (X, Y). [Effects of OCT imaging]

[0038] OCT imaging allows for the acquisition of fingerprint images of the epidermis without contact, and is therefore not affected by deformation that occurs during contact, unlike epidermal fingerprint imaging, which involves contacting a fingertip with a glass plate or the like to acquire a fingerprint image. OCT imaging allows for the acquisition of fingerprint images of the dermis. In other words, fingerprint images can be acquired without being affected by the state of the epidermis, eliminating the difficulty of reading epidermal fingerprints. In addition, since the fingertip does not come into contact with a glass plate or the like, it is hygienic. Furthermore, it is also suitable for detecting alterations to epidermal fingerprints. [2-3: Image processing operation performed by image processing device 2]

[0039] The flow of the image generating operation performed by the image processing device 2 in the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the image processing operation performed by the image processing device 2 in the second embodiment. This operation may be performed, for example, when the optical coherence tomography imaging apparatus 100 generates new three-dimensional intensity data. Alternatively, the three-dimensional intensity data may be acquired in advance from the optical coherence tomography imaging apparatus 100 and may be read out and executed from a storage medium such as the storage device 22. [Acquisition operation performed by acquisition unit 211]

[0040] The acquisition unit 211 acquires three-dimensional luminance data of the skin (step S20). The acquisition unit 211 may acquire three-dimensional luminance data generated by the optical coherence tomography imaging apparatus 100, which indicates the backscattering intensity of the object light at Nz × Nx × Ny points. For example, Nx=300, Ny=300, and Nz=256 may be used. In this case, the signal processing control unit 140 may analyze the interference light spectrum of the object light and the reference light to acquire luminance data resolved into 256 locations in the Z direction. Furthermore, the irradiation optical system 132 may scan so as to irradiate the object light beam at 300 locations in the fast axis direction of scanning (X direction) and 300 locations in the slow axis direction of scanning (Y direction). The three-dimensional luminance data can be regarded as a set of Ny (=300) B-scan tomographic images of Nx×Nz (=300×256). The acquisition unit 211 may acquire three-dimensional luminance data indicating a three-dimensional shape as shown in FIG. 4(c). [Pattern Image Extraction Operation Performed by First Extraction Unit 212]

[0041] The first extraction unit 212 extracts a skin pattern image from the three-dimensional luminance data of the skin (step S21). The first extraction unit 212 may extract at least one of an epidermal fingerprint image and a dermal fingerprint image. The first extraction unit 212 may extract a pattern image such as the one shown in FIG. 4(d).

[0042] The first extraction unit 212 may extract at least one of an epidermal fingerprint image and a dermal fingerprint image by orthogonally projecting the curved surface of the finger's epidermis onto a tangent plane of the highest elevation point of the curved surface. FIG. 6(a) is a conceptual diagram of a non-contact pattern image 2Da obtained when the curved surface CS of the finger's epidermis is orthogonally projected onto a tangent plane of the highest elevation point (4) of the curved surface CS. The non-contact pattern image 2Da is a pattern image in which the same position on the XY plane irradiated with the light beam is reflected in the fingerprint image. Therefore, the distance on the non-contact pattern image 2Da differs from the distance on the curved surface of the finger's epidermis. For example, if equally spaced points are set on the non-contact pattern image 2Da, the intervals between the corresponding points on the curved surface of the finger's epidermis are not equal, and the intervals depend on the normal direction of the curved surface of the finger's epidermis. For example, as shown in Figure 6(a), positions (1) to (7) are equally spaced on the non-contact pattern image 2Da. The normal direction of position (4) is approximately the same as the Z-axis direction, and the difference between the normal directions of positions (2) and (6) and the Z-axis direction is greater than the difference between the normal directions of positions (3) and (5) and the Z-axis direction, and the difference between the normal directions of positions (1) and (7) and the Z-axis direction is greater. Furthermore, the interval between points on the curved surface from position (3) to position (2) or from position (5) to position (6) is wider than the interval between points on the curved surface from position (4) to position (3) or from position (5), and the interval between points on the curved surface from position (2) to position (1) or from position (6) to position (7) is wider.

[0043] Fig. 6(b) is a conceptual diagram of capturing a fingerprint image by bringing a finger S into contact with a glass plate or the like G. As shown in Fig. 6(b), when a contact pattern image 2Db is captured by bringing a finger S into contact with a glass plate or the like G, the distance on the contact pattern image 2Db and the distance on the curved surface of the finger's skin S are approximately equal.

[0044] Incidentally, fingerprint images that have been collected in the past and registered in a fingerprint database are often acquired by touching a finger to a glass plate or pressing it against paper, and are often contact pattern images 2Db as shown in Figure 6(b). When non-contact pattern images 2Da and contact pattern images 2Db are compared, they often have different characteristics even if they belong to the same person. When non-contact pattern image 2Da is compared with contact pattern image 2Db, the comparison may fail, resulting in a decrease in comparison accuracy, even if the patterns belong to the same person.

[0045] Enabling highly accurate matching with previously collected fingerprints is an important issue. For example, when matching fingerprints against fingerprints registered in a database as a blacklist at Japanese immigration control, the fingerprints registered in the database are obtained by touching a glass plate or pressing them against paper. It is said that the fingerprints registered in such databases include those of approximately 14,000 people wanted by the International Criminal Police Organization (ICPO) and the Japanese police, and approximately 800,000 people who have been deported from Japan in the past. Therefore, there is a demand for technology to acquire fingerprint images that can be matched with these fingerprints with high accuracy. To address this demand, in the subsequent step, points (1) to (7) are moved to positions (1') to (7') to generate a shifted fingerprint image 2Dc, as shown in Figure 6(c). [Extraction Operation in the Normal Direction Performed by the Second Extraction Unit 213]

[0046] The second extraction unit 213 extracts the normal direction of the skin surface from the three-dimensional luminance data of the skin (step S22). The second extraction unit 213 may analyze the epidermis shape based on the three-dimensional luminance data. The second extraction unit 213 may extract the normal direction of the curved surface of the epidermis based on the three-dimensional coordinates of the epidermis position.

[0047] A point at a given position n (n x ,n y ,n z ), the angle in the X direction and the angle in the Y direction are calculated for the angle formed by the Z axis and the normal direction of the curved surface.

[0048] The angle θ in the X direction between the Z axis and the normal direction of the curved surface at position n x is calculated from the difference Δx in the X direction and the difference Δz in the Z direction between the position n and the neighboring points, x = arctan(Δx / Δz). Similarly, the angle θ in the Y direction between the Z axis and the normal direction of the surface at position n y is calculated from the difference Δy in the Y direction and the difference Δz in the Z direction between the position n and the neighboring points, y = arctan(Δy / Δz). The points in the vicinity of a given position n may include at least one point adjacent to n. [Image generation operation performed by the generation unit 214]

[0049] The generation unit 214 moves the positions of pixels included in the pattern image based on the normal direction to generate a moved pattern image 2Dc (step S23). The generation unit 214 may move the positions of each pixel in the pattern image based on the normal direction of the pixel position included in the pattern image. The generation unit 214 may move the positions of pixels included in the pattern image based on the difference between the normal direction of the center of the pattern image and the normal direction of the pixel position included in the pattern image. The generation unit 214 may generate a moved pattern image 2Dc equivalent to the contact pattern image 2Db based on the non-contact pattern image 2Da extracted by the first extraction unit 212 and the normal direction extracted by the second extraction unit 213 based on the analysis result of the epidermis shape.

[0050] 7 is a conceptual diagram of pixel movement processing. The movement distance s may be calculated using the difference d in height between the highest point and position n and the angle θ of the normal direction at position n with respect to the Z axis. The movement distance s in the X direction x is, s x = d × tanθ x The moving distance in the Y direction s y is, s y = d × tanθ y That is, the pixel (n x ,n y ) is (n x +d×tanθ x ,n y +d×tanθ y ) may be moved to

[0051] Figure 7(a) shows the movement distance s x FIG. 7(b) shows the movement distance s y 7(a) and 7(b) each illustrate a position F that is relatively far from the highest point of elevation and a position N that is relatively close to the highest point of elevation. At the relatively far position F, the difference in height d between the highest point of elevation and position n, the angle θ of the normal direction at position n with respect to the Z axis, and the movement distance s are all larger than at the relatively close position N.

[0052] Fig. 7(c) is a conceptual diagram of the non-contact pattern image 2Da, and Fig. 7(d) is a conceptual diagram of the moved pattern image 2Dc. As shown in Fig. 7(c) and (d), the further away from the center of the moved pattern image 2Dc, the greater the amount of movement of the positions of the pixels included in the pattern image.

[0053] Furthermore, a table associating the angle θ with the movement distance s may be prepared, for example, in the storage device 22. The generation unit 214 may refer to the table to obtain the movement distance corresponding to the angles in the X direction and Y direction, and move the pixel in the X direction and Y direction. [2-4: Actual conversion example]

[0054] Fig. 8 shows an example of an image when a grid image is actually converted by the above-mentioned operation. Fig. 8(a) is an example of the image before conversion, and Fig. 8(b) is an example of the image generated by the generation unit 214. In the example image shown in Fig. 8(b), the grid distortion increases the further away from the center. Furthermore, when a fingerprint image is transformed, the spacing between ridges and the width of the ridges become wider the further away from the center of the image. [2-5: Technical Effects of Image Processing Device 2]

[0055] The image processing device 2 according to the second embodiment moves the position of each pixel in the pattern image based on the normal direction of the pixel's position included in the pattern image, so that each pixel can be moved to an appropriate position. Also, the position of the pixel in the pattern image is moved based on the difference between the normal direction of the center of the pattern image and the normal direction of the pixel's position included in the pattern image, so that the pixel can be moved appropriately. [3: Third embodiment]

[0056] A third embodiment of an image processing device, an image processing method, and a recording medium will be described below. The third embodiment of the image processing device, the image processing method, and the recording medium will be described below using an image processing device 3 to which the third embodiment of the image processing device, the image processing method, and the recording medium is applied.

[0057] The image processing device 3 in the third embodiment is different from the image processing device 2 in the second embodiment in the generation operation by the generation unit 214. Other features of the image processing device 3 may be the same as other features of the image processing device 2. [3-1: Generation operation by image processing device 3]

[0058] The surface of a finger often has minute irregularities such as ridges and valleys. The three-dimensional shape obtained by OCT imaging often includes fine irregularities and is not a simple quadratic curve. Therefore, even for pixels far from the center of the image, the normal direction may be approximately the same as the Z axis. The center of the image may be the highest point in the image. The center of the image may be the most elevated part of the finger pad. Furthermore, the normal direction at the center of the image may be approximately the same as the Z axis.

[0059] When a finger is actually pressed against a glass plate or the like, the position of the fingerprint shifts at positions farther from the center of the image. Therefore, in the third embodiment, the generation unit 214 increases the amount of shift of the positions of the pixels included in the pattern image as the distance from the center of the pattern image increases.

[0060] The generation unit 214 may correct the pixel so that the greater the distance from the center where the pixel closest to 0 in the Z-axis direction is located, the greater the amount of movement of the pixel. [3-2: Technical Effects of Image Processing Device 3]

[0061] Correction is performed so that the greater the distance from the center of the image, the greater the amount of movement, so even if the three-dimensional shape includes fine irregularities, an appropriate post-movement pattern image can be generated. [4: Fourth embodiment]

[0062] A fourth embodiment of an image processing device, an image processing method, and a recording medium will be described below. The fourth embodiment of the image processing device, the image processing method, and the recording medium will be described below using an image processing device 4 to which the fourth embodiment of the image processing device, the image processing method, and the recording medium is applied.

[0063] The image processing device 4 in the fourth embodiment is different from the image processing device 2 in the second embodiment and the image processing device 3 in the third embodiment in the generation operation by the generation unit 214. Other features of the image processing device 4 may be the same as other features of the image processing device 2 and the image processing device 3. [4-1: Generation operation by image processing device 4]

[0064] The generation unit 214 extracts the normal direction of the pixel position included in the pattern image and corrects the extracted normal direction according to the distance from the center of the pattern image to the pixel. The generation unit 214 may correct the angle of a portion outside a predetermined angle range for each position so that the angle becomes continuous with the surrounding angles.

[0065] For example, as shown in Fig. 9, ranges A, B, C, and D may be set in a B-scan tomographic image according to proximity to the highest elevation point. In the case shown in Fig. 10, range A is set to a range where X is equal to or greater than 90 and less than 210. range B is set to a range where X is equal to or greater than 50 and less than 90, and a range where X is equal to or greater than 210 and less than 250. range C is set to a range where X is equal to or greater than 20 and less than 50, and a range where X is equal to or greater than 250 and less than 280. range D is set to a range where X is equal to or greater than 0 and less than 20, and a range where X is equal to or greater than 260 and less than 300.

[0066] For example, in range A, the generation unit 214 may correct the normal direction so that the angle of the normal with respect to the Z axis is equal to or greater than 0° and less than 5°. Furthermore, in range B, the generation unit 214 may correct the normal direction so that the angle of the normal with respect to the Z axis is equal to or greater than 5° and less than 15°. Furthermore, in range C, the generation unit 214 may correct the normal direction so that the angle of the normal with respect to the Z axis is equal to or greater than 15° and less than 25°. Furthermore, in range D, the generation unit 214 may correct the normal direction so that the angle of the normal with respect to the Z axis is equal to or greater than 25° and less than 35°.

[0067] In such a case, if the normal direction at position X=150 (within range A) is 1°, this is within the predetermined angle range (0° or more and less than 5°), so the generation unit 214 moves the pixel according to the extracted normal direction. On the other hand, if the normal direction at position X=230 (within range B) is 1°, this is outside the predetermined angle range (5° or more and less than 15°), so the generation unit 214 moves the pixel according to the corrected normal direction. As an example of correction, for example, the generation unit 214 may use the average value of the normal directions from position X=220 to position X=240 in the vicinity of X=230 as the corrected normal direction for the position X=230. Alternatively, the generation unit 214 may use the average value of the normal direction at position X=220 and the normal direction at position X=240 in the vicinity of X=230 as the corrected normal direction for the position X=230. The neighboring positions are not limited to positions up to 10 pixels away from the corresponding position, but may include positions up to 20 pixels away from the corresponding position, for example. [4-2: Technical Effects of Image Processing Device 4]

[0068] The extracted normal direction is corrected according to the distance from the center of the pattern image to the corresponding pixel, so that an appropriate post-movement pattern image can be generated even if the three-dimensional shape includes small irregularities. [5: Fifth embodiment]

[0069] A fifth embodiment of an image processing device, an image processing method, and a recording medium will be described below. The fifth embodiment of an image processing device, an image processing method, and a recording medium will be described below using an image processing device 5 to which the fifth embodiment of the image processing device, the image processing method, and the recording medium is applied. [5-1: Configuration of image processing device 5]

[0070] The configuration of the image processing device 5 in the fifth embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the image processing device 5 in the fifth embodiment.

[0071] 10, the image processing device 5 of the fifth embodiment differs from the image processing device 2 of the second embodiment to the image processing device 4 of the fourth embodiment in that the calculation device 21 includes a matching unit 515 and the storage device 22 stores a fingerprint database DB in which registered pattern images are registered. However, the storage device 22 does not necessarily have to store a fingerprint database DB. Other features of the image processing device 5 may be the same as other features of the image processing devices 2 to 4.

[0072] The image processing device 5 may use the three-dimensional luminance data to generate a fingerprint image suitable for fingerprint authentication, register the fingerprint image in advance in the fingerprint database DB, and perform biometric authentication processing by matching the fingerprint image. The matching unit 515 matches the post-movement pattern image 2Dc with a registered pattern image that has been registered in advance. The matching unit 515 may also match the post-movement pattern image 2Dc generated by the generation unit 214 with a fingerprint image registered in the registered pattern image. A fingerprint image measured and extracted in a non-contact manner by OCT imaging can be matched with high accuracy with fingerprint images previously collected in a contact-type manner and recorded in a database.

[0073] For example, a high score may not be obtained when matching the non-contact pattern image 2Da shown in FIG. 6(a) with the contact pattern image 2Db shown in FIG. 6(b), but a high score can be obtained when matching the post-movement pattern image 2Dc shown in FIG. 6(c) with the contact pattern image 2Db shown in FIG. 6(b). [5-2: Technical Effects of Image Processing Device 4]

[0074] Since the non-contact pattern image 2Da is converted into a post-movement pattern image 2Dc, accurate matching can be achieved even when the non-contact pattern image 2Da is matched with the contact pattern image 2Db. [6: Learning to generate pattern images after movement]

[0075] It should be noted that a generation engine for generating a post-movement pattern image may be constructed by machine learning using a mechanical mechanism, and the generation unit 214 may generate a post-movement pattern image using the generation engine.

[0076] For example, the generated post-movement pattern image is compared with a registered pattern image that has been registered in advance, and if they match, position information of the matching feature point is obtained. Data that associates the position of the matching feature point, the difference (distance in the X and Y directions) between the registered pattern image and the pattern image at the matching feature point, and the normal direction of the pattern image at the matching feature point may be used as learning data. The difference corresponds to the amount of movement. Machine learning may be performed using the learning data to generate a generation engine.

[0077] The learning mechanism may cause the generation engine to learn how to generate a post-movement pattern image based on the result of matching between the contact pattern image and the post-movement pattern image generated by the generation unit 214.

[0078] The generation engine may output the amount of movement of a pixel when the position of the pixel in the pattern image and the normal direction of that pixel are input.

[0079] The learning data may be data that includes information on the distance from the center of the fingerprint in addition to the position, difference, and normal direction. [7: Fingerprint image labeling]

[0080] Fingerprint images include at least (1) a non-contact fingerprint image 2Da obtained by simply projecting a three-dimensional shape obtained by OCT imaging or the like onto a plane, (2) a contact fingerprint image 2Db obtained by pressing a finger against a glass plate, and (3) a post-movement fingerprint image 2Dc obtained by processing the three-dimensional shape obtained by OCT imaging or the like in the same manner as in (2). Therefore, fingerprint images may be labeled and registered in a fingerprint database according to each acquisition method. When a fingerprint registered in the fingerprint database is used for authentication, the label may be referenced and authentication according to the acquisition method may be performed.

[0081] In the above-described embodiments, the biometric information of the subject of optical coherence tomography imaging has been described using a finger skin pattern (fingerprint) as an example. However, biometric information is not limited to fingerprints. Biometric information other than fingerprints, such as irises, palmprints, and footprints, may also be used and optical coherence tomography imaging of these biometric information may be applied. Because the iris is a muscle fiber, iris features can be acquired from optical coherence tomography images, and iris authentication may be performed using these features. Fingerprints may be patterns of fingers or toes. When optical coherence tomography imaging of hand or footprints, including fingerprints, light that passes through resin or the like may be used. [8: Note]

[0082] The following additional notes are provided regarding the above-described embodiment. [Appendix 1] an acquisition means for acquiring three-dimensional luminance data of the skin of the finger by performing optical coherence tomography imaging by irradiating the skin of the finger with a light beam while scanning it two-dimensionally; a first extraction means for extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin; a second extraction means for extracting a normal direction of the surface of the skin from the three-dimensional luminance data of the skin; a generating means for shifting the positions of pixels included in the pattern image based on the normal direction and generating a shifted pattern image; An image processing device comprising: [Appendix 2] The generating means moves the position of each pixel of the pattern image based on the normal direction of the pixel position included in the pattern image. 2. The image processing device according to claim 1. [Appendix 3] The generating means moves the positions of pixels included in the pattern image based on the difference between the normal direction of the center of the pattern image and the normal direction of the positions of pixels included in the pattern image. 3. The image processing device according to claim 1 or 2. [Appendix 4] The generating means increases the amount of movement of the positions of the pixels included in the pattern image as the pixels are further away from the center of the pattern image. 4. The image processing device according to any one of claims 1 to 3. [Appendix 5] The second extraction means extracts the normal direction of a pixel included in the pattern image, and corrects the extracted normal direction in accordance with the distance from the center of the pattern image to the pixel. 5. The image processing device according to any one of claims 1 to 4. [Appendix 6] The image processing device further includes a matching unit for matching the moved pattern image with a registered pattern image that has been registered in advance. 6. The image processing device according to any one of appendices 1 to 5. [Appendix 7] irradiating the skin of the finger with a light beam while scanning it two-dimensionally to perform optical coherence tomography imaging, thereby obtaining three-dimensional brightness data of the skin; extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin; extracting a normal direction of the surface of the skin from the three-dimensional luminance data of the skin; The positions of the pixels included in the pattern image are shifted based on the normal direction, and a shifted pattern image is generated. Image processing methods. [Appendix 8] On the computer, irradiating the skin of the finger with a light beam while scanning it two-dimensionally to perform optical coherence tomography imaging, thereby obtaining three-dimensional brightness data of the skin; extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin; extracting a normal direction of the surface of the skin from the three-dimensional luminance data of the skin; The positions of the pixels included in the pattern image are shifted based on the normal direction, and a shifted pattern image is generated. A recording medium on which a computer program for executing an image processing method is recorded.

[0083] At least some of the constituent elements of each of the above-described embodiments can be appropriately combined with at least some of the other constituent elements of each of the above-described embodiments. Some of the constituent elements of each of the above-described embodiments may not be used. Furthermore, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure are incorporated by reference as part of the description of this disclosure.

[0084] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Image processing devices, image processing methods, and recording media that incorporate such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0085] 1,2,3,4,5 Image processing device 11,211 Acquisition Department 12,212 1st extraction part 13,213 2nd extraction part 14,214 generator 515 Collation Unit 100 Optical coherence tomography imaging device 110 wavelength swept laser light source 120 Optical interference receiving unit 130 Optical beam scanning unit 140 Signal processing control section O Measurement object 121 Circulator 122 Junction 123 Reference beam mirror 124 Balanced Receiver 131 Fiber collimator 132 Irradiation Optics Department R1 Object Light R2 reference light R3 Object Light R4 reference light R5 interference light R6 interference light

Claims

1. an acquisition means for acquiring three-dimensional luminance data of the skin of the finger by performing optical coherence tomography imaging by irradiating the skin of the finger with a light beam while scanning it two-dimensionally; a first extraction means for extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin; a second extraction means for extracting a normal direction of the surface of the skin from the three-dimensional luminance data of the skin; a generating means for shifting the positions of pixels included in the pattern image based on the normal direction and generating a shifted pattern image; An image processing device comprising:

2. The generating means moves the position of each pixel of the pattern image based on the normal direction of the pixel position included in the pattern image. The image processing device according to claim 1 .

3. The generating means moves the positions of pixels included in the pattern image based on the difference between the normal direction of the center of the pattern image and the normal direction of the positions of pixels included in the pattern image.

3. The image processing device according to claim 1 or 2.

4. The second extraction means extracts the normal direction of a pixel included in the pattern image, and corrects the extracted normal direction in accordance with the distance from the center of the pattern image to the pixel.

3. The image processing device according to claim 1 or 2.

5. The image processing device further includes a matching unit for matching the moved pattern image with a registered pattern image that has been registered in advance.

3. The image processing device according to claim 1 or 2.

6. 1. A computer-implemented image processing method, comprising: irradiating the skin of the finger with a light beam while scanning it two-dimensionally to perform optical coherence tomography imaging, thereby obtaining three-dimensional brightness data of the skin; extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin; extracting a normal direction of the surface of the skin from the three-dimensional luminance data of the skin; The positions of the pixels included in the pattern image are shifted based on the normal direction, and a shifted pattern image is generated. Image processing methods.

7. On the computer, irradiating the skin of the finger with a light beam while scanning it two-dimensionally to perform optical coherence tomography imaging, thereby obtaining three-dimensional brightness data of the skin; extracting a pattern image of the skin pattern from the three-dimensional luminance data of the skin; extracting a normal direction of the surface of the skin from the three-dimensional luminance data of the skin; The positions of the pixels included in the pattern image are shifted based on the normal direction, and a shifted pattern image is generated. A computer program for executing an image processing method.

Images