Detection device, detection method, and detection program

Abstract

This detection device comprises: a color channel selection unit for selecting a color channel on the basis of a separation degree which is an index that is calculated from each of a first color channel image and a second color channel image regarding an original image including a background image and a hand image and that is for determining separation characteristics between the background image and the hand image; and a hand image detection unit for detecting the hand image included in the original image, using the color channel selected by the color channel selection unit.　

Description

[Technical Field]

[0001] The present invention relates to a detection device, a detection method, and a detection program. [Background technology]

[0002] There is a demand for accurate detection of hand images from an original image for palm vein authentication, palm print authentication, etc. (See, for example, Patent Document 1.) Also, there is known a technique for detecting hand images from an image that is binarized using brightness of an original image captured by irradiating it with visible light (See, for example, Patent Document 2.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-026012 [Patent Document 2] Patent No. 6629150 [Non-patent literature]

[0004] [Non-Patent Document 1] T. Simon, H. Joo, I. Matthews, and Y. Sheikh, “Hand keypoint detection in single images using multiview bootstrapping,” in CVPR, 2017. Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is difficult to accurately detect the palm region.

[0006] In one aspect, the present invention aims to provide a detection device, a detection method, and a detection program that can improve the accuracy of detecting hand images. [Means for solving the problem]

[0007] In one aspect, the detection device includes a color channel selection unit that selects a color channel based on a degree of separation calculated from a first color channel image and a second color channel image for an original image including a background image and a hand image, the degree being an index for determining the separability between the background image and the hand image, and a hand image detection unit that detects the hand image included in the original image using the color channel selected by the color channel selection unit. [Effects of the Invention]

[0008] The accuracy of detecting hand images can be improved. [Brief explanation of the drawings]

[0009] [Figure 1] 1A is a block diagram illustrating an example of the overall configuration of a detection device, and FIG. 1B is a block diagram illustrating an example of the hardware configuration of the detection device. [Figure 2] 10 is a flowchart illustrating an example of a hand image detection process executed by the detection device. [Figure 3] 10A is a diagram illustrating an example of key points, and FIG. 10B is a diagram illustrating an example of small regions. [Figure 4] FIG. 3 is a diagram illustrating an example of a flowchart showing the details of step S6 in FIG. 2. [Figure 5] FIG. 3 is a diagram illustrating the details of steps S5 to S7 in FIG. 2. [Figure 6] FIG. 10 is a diagram illustrating a case where the degree of separation is large. [Figure 7] FIG. 10 is a diagram illustrating a case where the degree of separation is large. [Figure 8] FIG. 10 is a diagram illustrating a case where the degree of separation is small. [Figure 9] FIG. 10 is a diagram illustrating a case where the degree of separation is small. [Figure 10] 10 is a flowchart illustrating an example of a registration process. [Figure 11] 10 is a flowchart illustrating an example of a matching process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Biometric authentication is a technology that verifies identity using biometric information such as fingerprints, faces, veins, etc. In biometric authentication, when verification is required, biometric data for verification acquired by a sensor is compared (matched) with pre-registered biometric data, and identity verification is performed by determining whether the similarity is equal to or exceeds a threshold for identity verification.

[0011] In palm vein authentication, palm print authentication, etc., it is important to accurately detect a hand image from an original image. However, when taking a photograph using a general camera such as the one built into a mobile device, it can be difficult to separate the hand image from the background image due to factors such as the optical characteristics of the camera, the effects of reflections and shadows caused by the light source, and the color and pattern of the background.

[0012] Therefore, in the following embodiments, a detection device, a detection method, and a detection program are described that can improve the accuracy of detecting hand images by separating hand images from background images regardless of the shooting conditions. [Example]

[0013] Fig. 1(a) is a block diagram illustrating an example of the overall configuration of a detection device 100. As illustrated in Fig. 1(a), the detection device 100 functions as an acquisition unit 11, a keypoint detection unit 12, an area division unit 13, a determination unit 14, a color channel conversion unit 15, a binarization processing unit 16, a candidate extraction unit 17, a degree of separation calculation unit 18, a color channel selection unit 19, a hand image detection unit 20, a biometric information extraction unit 21, a matching unit 22, a storage unit 23, and the like.

[0014] Fig. 1(b) is a block diagram illustrating an example of the hardware configuration of the detection device 100. As illustrated in Fig. 1(b), the detection device 100 includes a CPU 101, a RAM 102, a storage device 103, an interface 104, a display device 105, an input device 106, a camera 107, and the like.

[0015] The CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, etc. The storage device 103 is a non-volatile storage device. For example, the storage device 103 can be a read-only memory (ROM), a solid-state drive (SSD) such as a flash memory, or a hard disk driven by a hard disk drive. The storage device 103 stores a detection program. The interface 104 is an interface device with an external device. For example, the interface 104 is an interface device with a LAN (Local Area Network).

[0016] The display device 105 is a display device such as an LCD (Liquid Crystal Device), etc. The input device 106 is an input device such as a keyboard, a mouse, etc. The camera 107 is a MOS (Metal Oxide Semiconductor) sensor, a CCD (Charged Coupled Device) sensor, etc.

[0017] Execution of the detection program by CPU 101 realizes acquisition unit 11, keypoint detection unit 12, region division unit 13, determination unit 14, color channel conversion unit 15, binarization processing unit 16, candidate extraction unit 17, separation degree calculation unit 18, color channel selection unit 19, hand image detection unit 20, biometric information extraction unit 21, matching unit 22, and storage unit 23. Note that hardware such as dedicated circuits may be used as acquisition unit 11, keypoint detection unit 12, region division unit 13, determination unit 14, color channel conversion unit 15, binarization processing unit 16, candidate extraction unit 17, separation degree calculation unit 18, color channel selection unit 19, hand image detection unit 20, biometric information extraction unit 21, matching unit 22, and storage unit 23.

[0018] (Hand image detection processing) 2 is a flowchart showing an example of hand image detection processing executed by detection device 100. An example of the operation of detection device 100 will be described with reference to FIG.

[0019] The acquisition unit 11 acquires an original image including a hand image and a background image from the camera 107 (step S1).

[0020] Next, the key point detection unit 12 detects a plurality of key points indicating parts of the hand in the original image acquired by the acquisition unit 11 (step S2). For example, the key point detection unit 12 detects 21 key points P0 to P 20 Key point P0 is a point on the boundary between the palm and the wrist. Key point P1 is a point at the base of the thumb. Key point P2 is a point at the second joint of the thumb. Key point P3 is a point at the first joint of the thumb. Key point P4 is a point at the tip of the thumb. Key point P5 is a point at the base of the index finger. Key point P6 is a point at the second joint of the index finger. Key point P7 is a point at the first joint of the index finger. Key point P8 is a point at the tip of the index finger. Key point P9 is a point at the base of the middle finger. Key points P 10 is a point on the second joint of the middle finger. Key point P 11 is a point on the first joint of the middle finger. Key point P 12 is a point at the tip of the middle finger. Key point P 13 is a point at the base of the ring finger. Key point P 14 is a point on the second joint of the ring finger. Key point P 15 is a point on the first joint of the ring finger. Key point P 16 is a point at the tip of the ring finger. Key point P 17 is a point at the base of the little finger. Key point P 18 is a point on the second joint of the little finger. Key point P 19 is a point on the first joint of the little finger. Key point P 20 is a point on the tip of the little finger.

[0021] These key points P0 to P20 can be detected from an RGB image. For example, key points P0 to P 20 is obtained by detecting a bounding box that indicates the hand area, and then detecting each part of the hand. For example, key points P0 to P 20 can be detected using the method described in Non-Patent Document 1, "T. Simon, H. Joo, I. Matthews, and Y. Sheikh, 'Hand keypoint detection in single images using multiview bootstrapping,' in CVPR, 2017." Keypoints P0 to P 20 An example of this is shown in FIG. 3(a).

[0022] Next, the region dividing unit 13 obtains a plurality of small regions A to D from the original image using the key points (step S3). The small regions are smaller than the original image. FIG. 3(b) is a diagram illustrating the small regions A to D. The black circles drawn with black lines represent key points P0, P6, and P7. 18 The area D is a circle passing through key points P0, P5, and P 17 The small area B is a triangle with vertices P6 and P5. ... 13 and the line connecting the key point P 13 and key point P 14 The area is the area obtained by the line connecting small area B and small area D. Small area A is the area to the left of small area B and small area D in the black circle. Small area C is the area to the right of small area B and small area D in the black circle.

[0023] Next, the determination unit 14 determines whether each small region is contained within the hand when viewed in a plan view (step S4). Small region D is entirely contained within the hand, so it is determined as "Yes." Small regions A to C include the background, so they are determined as "No."

[0024] If the determination in step S4 is "Yes", the determination unit 14 determines that the small region for which the determination in step S4 is "Yes" is a hand region (step S5).

[0025] If the result of step S4 is "No," the color channel conversion unit 15, the binarization processing unit 16, the candidate extraction unit 17, the separability calculation unit 18, and the color channel selection unit 19 separate the small region for which the result of step S4 is "No" into a hand region and a background region (step S6). Details will be described later.

[0026] Next, hand image detection unit 20 combines the hand areas of the small areas obtained in step S5 and step S6 to detect a hand image (step S7).

[0027] Fig. 4 is a diagram illustrating an example of a flowchart showing the details of step S6. The process of Fig. 4 is executed for each small area determined as "No" in step S4. In this embodiment, the process of Fig. 4 is executed for each of small areas A to C.

[0028] First, the color channel conversion unit 15 converts the target small region into multiple color channel images (step S11). The multiple color channel images include at least two types of color channels: a first color channel image and a second color channel image. The color channels are, for example, R (Red), G (Green), B (Blue), H (Hue), S (Saturation), V (Value / Brightness), Y (Luminance), U (Color Difference), and V (Color Difference). For example, the color channel conversion unit 15 obtains an image of each color channel (hereinafter referred to as a color channel image) for the target small region. Note that the multiple color channel images may include color channel images of three or more types of color channels.

[0029] Next, the binarization processor 16 performs binarization processing on each color channel image using a predetermined threshold value to obtain hand candidates and background candidates (step S12). For example, hand candidates are displayed in white, and background candidates are displayed in black.

[0030] Next, the separability calculation unit 18 creates a color histogram (a frequency distribution of pixel values) of hand candidates and a color histogram of background candidates for each color channel image before binarization, and calculates a first separability (step S13). For example, binarization is performed on the R image to obtain hand candidates and background candidates. Next, a color histogram of the hand candidates and a color histogram of the background candidates are created for the R image before binarization. The first separability of the R image is calculated by overlaying these histograms.

[0031] Next, the separability calculation unit 18 performs morphology processing on at least one of the hand candidates and the background candidates for each color channel image after the binarization processing. The morphology processing is a process of filling in gaps in the area. The separability calculation unit 18 calculates a second separability from the area difference before and after the morphology processing (step S14).

[0032] Next, the color channel selection unit 19 selects the color channel that maximizes the sum of the first and second separation degrees, and obtains the separation result into the palm and the background (step S15). For example, by normalizing the first and second separation degrees, the first and second separation degrees can be quantified on the same scale. For example, by expressing the first and second separation degrees as numbers between 0 and 100, the sum of the first and second separation degrees can be compared for each color channel using the same criteria.

[0033] FIG. 5 is a diagram illustrating the details of steps S5 to S7 in FIG. 2. As illustrated in FIG. 5, multiple color channel images are obtained for each of small regions A to C. For each of small regions A to C, a color channel image that maximizes the sum of the first and second separabilities is selected. In the example of FIG. 5, the R image is selected for small region A, the G image is selected for small region B, and the R image is selected for small region C. A palm region is then synthesized from the image after binarization of the selected color channel images. For small region D, binarization is performed without selecting a color channel. The synthesized palm region is shaped by morphological processing. By extracting from the original image an area corresponding to the area shaped by morphological processing, the accuracy of hand image detection can be improved. Note that, for example, the hand image is an image showing the palm region.

[0034] 6 to 9 are diagrams illustrating the details of calculation of the first degree of separation and the second degree of separation. Fig. 6 and Fig. 7 are diagrams illustrating the case where the degree of separation is large. Fig. 8 and Fig. 9 are diagrams illustrating the case where the degree of separation is small.

[0035] FIG. 6 is a diagram illustrating a color histogram when the first degree of separation is large. In FIG. 6, the color histogram of the hand candidate in the color channel image and the color histogram of the background candidate in the color channel image are superimposed. When the first degree of separation is large, as illustrated in FIG. 6, the separation between the background candidate and the hand region is high across the binarization threshold. For example, the first degree of separation can be defined as the size of the interval between the pixel value at the peak position of the color histogram of the background candidate and the pixel value at the peak position of the color histogram of the hand candidate. The larger the interval, the larger the value of the first degree of separation, and the smaller the interval, the smaller the value of the first degree of separation. Alternatively, the first degree of separation may be the low frequency of the bottom between two peaks. Alternatively, the first degree of separation may be a value that reflects both the above-mentioned interval and the above-mentioned low frequency.

[0036] FIG. 7 is a diagram illustrating the details of a case where the second degree of separation is large. The upper diagram in FIG. 7 is a binarized image. The lower diagram in FIG. 7 is an image obtained by performing morphological processing on the binarized image. When the second degree of separation is large, the difference in area between before and after morphological processing becomes small, as illustrated in FIG. 7.

[0037] Fig. 8 is a diagram illustrating a color histogram when the first degree of separation is small. In Fig. 8, the color histogram of the hand candidate in the color channel image and the color histogram of the background candidate in the color channel image are superimposed. When the first degree of separation is small, as illustrated in Fig. 8, the separability between the background region and the palm region in the color histogram becomes low on both sides of the binarization threshold.

[0038] Fig. 9 is a diagram illustrating the details when the second degree of separation is small. The upper diagram in Fig. 9 is a binarized image. The lower diagram in Fig. 9 is an image obtained by performing morphological processing on the binarized image. When the second degree of separation is small, the difference in area between before and after morphological processing becomes large, as illustrated in Fig. 9.

[0039] The biometric authentication process using the detected palm area will be described below.

[0040] (Registration process) FIG. 10 is a flowchart illustrating an example of the registration process executed by the detection device 100.

[0041] 10, the acquisition unit 11 acquires the original image captured by the camera 107 and the ID (identification information) of the registrant (step S21). The ID is input by the registrant or the like using the input device 106, for example.

[0042] Next, the hand image detection unit 20 detects a hand image by the hand image detection process described above (step S22).

[0043] Next, the biometric information extraction unit 21 extracts biometric information from the hand image detected in step S22 (step S23). The biometric information may be veins, palm prints, etc., but is not particularly limited as long as it is biometric information obtainable from a hand image.

[0044] The storage unit 23 associates the biometric information extracted in step S23 with the ID acquired in step S21 and stores it as registered biometric data (step S24). The above process completes the registration process. For example, by repeating this registration process multiple times, the biometric information of multiple registrants can be registered.

[0045] (Matching process) FIG. 11 is a flowchart illustrating an example of the matching process executed by the detection device 100.

[0046] As illustrated in FIG. 11, the acquisition unit 11 acquires an original image captured by the camera 107 (step S31).

[0047] Next, the hand image detection unit 20 detects a hand image by the hand image detection process described above (step S32).

[0048] Next, the biometric information extraction unit 21 extracts biometric information from the hand image detected in step S32 as biometric data for matching (step S33).

[0049] The matching unit 22 performs authentication by matching the matching biometric data extracted in step S33 with each registered biometric data stored in the storage unit 23 (step S34). Specifically, it determines whether the similarity between each registered biometric data stored in the storage unit 23 and the matching biometric data extracted in step S33 is equal to or greater than a threshold. If there is registered biometric data whose similarity is equal to or greater than the threshold, it determines that the matching biometric data is biometric information of the person with the ID of the registered biometric data. The matching unit 22 outputs the determination result.

[0050] According to this embodiment, a degree of separation, which is an index for determining the separability between a background image and a hand image, is calculated from each of a first color channel image and a second color channel image for an original image including a background image and a hand image. A color channel is selected based on the calculated degree of separation. The selected color channel is used to detect the hand image included in the original image. Because the hand image is detected using a color channel with a high degree of separation, the hand image can be accurately separated from the background image regardless of the shooting conditions. This improves the accuracy of hand image detection.

[0051] The degree of separation is not particularly limited as long as it is an index for determining the degree of separation between the background image and the hand image.

[0052] However, the accuracy of detecting the separability of the background candidate and the hand candidate is improved by using the first separability calculated from the color histograms calculated for each of the background candidate and the hand candidate.Furthermore, the accuracy of detecting the separability of the background candidate and the hand candidate is improved by using the second separability according to the area difference before and after morphological processing performed on at least one of the background candidate and the hand candidate.

[0053] Furthermore, by determining the separability of background candidates and hand candidates for each of multiple small regions obtained from the original image, it becomes possible to select a color channel for each small region. This further improves the accuracy of detecting hand images from the original image. By obtaining small regions using key points related to the hand parts, the influence of individual differences in the palm can be reduced. Therefore, the influence of individual differences such as differences in palm size and shape can be reduced. By not selecting a color channel for small regions determined to be located inside the hand, the amount of processing required to detect hand images can be reduced.

[0054] In each of the above examples, the color channel selection unit 19 is an example of a color channel selection unit that selects a color channel based on a separability, which is calculated from each of a first color channel image and a second color channel image of an original image including a background image and a hand image, and is an index for determining the separability between the background image and the hand image. The hand image detection unit 20 is an example of a hand image detection unit that detects a hand image included in the original image using the color channel selected by the color channel selection unit. The binarization processing unit 16 is an example of a binarization processing unit that binarizes each of the first color channel image and the second color channel image. The candidate extraction unit 17 is an example of a candidate extraction unit that extracts background candidates and hand candidates from each of the binarized first color channel image and the second color channel image. The separability calculation unit 18 is an example of a frequency distribution calculation unit that calculates a frequency distribution of pixel values ​​for each of the background candidates and the hand candidates. The separability calculation unit 18 is also an example of a morphology processing unit that performs morphology processing on at least one of the background candidates and the hand candidates. The region dividing unit 13 is an example of a region dividing unit that divides each of the first color channel image and the second color channel image into a plurality of regions. The determination unit 14 is an example of a determination unit that uses key points to determine whether the plurality of regions include a background.

[0055] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as defined in the claims. [Explanation of symbols]

[0056] 11 Acquisition Department 12 Keypoint detection unit 13 Area division part 14 Judgment section 15 Color channel conversion section 16 Binarization processing section 17 Candidate Extraction Unit 18 Separation degree calculation section 19 Color channel selection section 20 Hand image detection unit 21 Biometric information extraction unit 22 Matching unit 23 Storage area 100 Detection device 105 Display device 107 Camera

Claims

1. a color channel selection unit that selects a color channel based on a degree of separation that is calculated from a first color channel image and a second color channel image of an original image including a background image and a hand image, the degree of separation being an index for determining the degree of separation between the background image and the hand image; a hand image detection unit that detects the hand image included in the original image by using the color channel selected by the color channel selection unit.

2. a binarization processing unit that binarizes the first color channel image and the second color channel image, respectively; a candidate extraction unit that extracts background candidates and hand candidates from the binarized first color channel image and the binarized second color channel image, respectively; a frequency distribution calculation unit that calculates a frequency distribution of pixel values ​​for each of the background candidate and the hand candidate, The detection device according to claim 1 , wherein the degree of separation includes a first degree of separation calculated from a frequency distribution of pixel values.

3. 3. The detection device according to claim 2, wherein the first degree of separation is a degree of separation corresponding to a pixel value difference between two frequency peaks in the frequency distribution of the pixel values.

4. a binarization processing unit that binarizes the first color channel image and the second color channel image, respectively; a candidate extraction unit that extracts background candidates and hand candidates from the binarized first color channel image and the binarized second color channel image, respectively; a morphology processing unit that performs morphology processing on at least one of the background candidate and the hand candidate, The detection device according to claim 1 , wherein the degree of separation includes a second degree of separation according to a difference between regions before and after the morphological processing.

5. a region dividing unit that divides each of the first color channel image and the second color channel image into a plurality of regions; 5. The detection device according to claim 1, wherein the color channel selection unit selects a color channel in accordance with the degree of separation of each of the plurality of regions.

6. 6. The detection device according to claim 5, wherein the region dividing unit divides each of the first color channel image and the second color channel image into the plurality of regions according to key points about hand parts detected from the original image.

7. a determination unit that determines whether the plurality of regions includes a background using the key points; 7. The detection device according to claim 6, wherein for an area determined by the determination unit not to contain a background, the color channel selection unit does not select a color channel, and the hand image detection unit detects it as the hand image.

8. selecting a color channel based on a degree of separation, which is calculated from a first color channel image and a second color channel image of an original image including a background image and a hand image, and is an index for determining the degree of separation between the background image and the hand image; The method further comprises: detecting the hand image included in the original image using a selected color channel, the method comprising: a computer executing a process;

9. On the computer, a process of selecting a color channel based on a degree of separation, which is calculated from each of a first color channel image and a second color channel image of an original image including a background image and a hand image, and is an index for determining the degree of separation between the background image and the hand image; and detecting the hand image included in the original image using the selected color channel.

Images