Detection device, detection method, and detection program

Abstract

This detection device comprises: a color channel selection unit for selecting a color channel on the basis of a separation degree which is an index that is calculated from each of a first color channel image and a second color channel image regarding an original image including a background image and a hand image and that is for determining separation characteristics between the background image and the hand image; and a hand image detection unit for detecting the hand image included in the original image, using the color channel selected by the color channel selection unit.　

Description

Detection device, detection method, and detection program

[0001] The present invention relates to a detection device, a detection method, and a detection program.

[0002] There is a demand for accurate detection of a hand image from an original image for palm vein authentication, palm print authentication, etc. (See, for example, Patent Document 1.) Also, there is known a technique for detecting a hand image from an image binarized using brightness of an original image captured by irradiating it with visible light (See, for example, Patent Document 2.).

[0003] JP 2018-026012 A Japanese Patent No. 6629150 A

[0004] T. Simon, H. Joo, I. Matthews, and Y. Sheikh, “Hand keypoint detection in single images using multiview bootstrapping,” in CVPR, 2017.

[0005] However, it is difficult to accurately detect the palm region.

[0006] In one aspect, the present invention aims to provide a detection device, a detection method, and a detection program that can improve the accuracy of detecting hand images.

[0007] In one aspect, the detection device includes a color channel selection unit that selects a color channel based on a degree of separation, which is calculated from a first color channel image and a second color channel image for an original image including a background image and a hand image, and is an index for determining the separability between the background image and the hand image, and a hand image detection unit that detects the hand image included in the original image using the color channel selected by the color channel selection unit.

[0008] The accuracy of detecting hand images can be improved.

[0009] 1A is a block diagram illustrating an example of the overall configuration of a detection device, and FIG. 1B is a block diagram illustrating an example of the hardware configuration of the detection device. FIG. 1B is a flowchart illustrating an example of hand image detection processing executed by the detection device. FIG. 1A is a diagram illustrating an example of key points, and FIG. 1B is a diagram illustrating an example of small regions. FIG. 1B is a diagram illustrating an example of a flowchart showing details of step S6 in FIG. 2. FIG. 1C is a diagram illustrating details of steps S5 to S7 in FIG. 2. FIG. 1D is a diagram illustrating a case where the degree of separation is large. FIG. 1E is a diagram illustrating a case where the degree of separation is large. FIG. 1F is a diagram illustrating a case where the degree of separation is small. FIG. 1F is a flowchart illustrating an example of a registration process. FIG. 1G is a flowchart illustrating an example of a matching process.

[0010] Biometric authentication is a technology that verifies identity using biometric information such as fingerprints, faces, veins, etc. In biometric authentication, when verification is required, biometric data for verification acquired by a sensor is compared (matched) with pre-registered biometric data, and identity verification is performed by determining whether the similarity is equal to or exceeds a threshold for identity verification.

[0011] In palm vein authentication, palm print authentication, etc., it is important to accurately detect a hand image from an original image. However, when taking a photograph using a general camera such as the one built into a mobile device, it can be difficult to separate the hand image from the background image due to factors such as the optical characteristics of the camera, the effects of reflections and shadows caused by the light source, and the color and pattern of the background.

[0012] Therefore, in the following embodiments, a detection device, a detection method, and a detection program are described that can improve the accuracy of detecting hand images by separating hand images from background images regardless of the shooting conditions.

[0013] Fig. 1A is a block diagram illustrating an example of the overall configuration of a detection device 100. As illustrated in Fig. 1A, the detection device 100 functions as an acquisition unit 11, a keypoint detection unit 12, an area division unit 13, a determination unit 14, a color channel conversion unit 15, a binarization processing unit 16, a candidate extraction unit 17, a degree of separation calculation unit 18, a color channel selection unit 19, a hand image detection unit 20, a biometric information extraction unit 21, a matching unit 22, a storage unit 23, and the like.

[0014] Fig. 1(b) is a block diagram illustrating an example of the hardware configuration of the detection device 100. As illustrated in Fig. 1(b), the detection device 100 includes a CPU 101, a RAM 102, a storage device 103, an interface 104, a display device 105, an input device 106, a camera 107, and the like.

[0015] The CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. The RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a non-volatile storage device. For example, the storage device 103 may be a read-only memory (ROM), a solid-state drive (SSD) such as a flash memory, or a hard disk driven by a hard disk drive. The storage device 103 stores a detection program. The interface 104 is an interface device with external devices. For example, the interface 104 is an interface device with a local area network (LAN).

[0016] The display device 105 is a display device such as an LCD (Liquid Crystal Device), the input device 106 is an input device such as a keyboard or a mouse, and the camera 107 is a MOS (Metal Oxide Semiconductor) sensor, a CCD (Charged Coupled Device) sensor, or the like.

[0017] Execution of the detection program by the CPU 101 realizes an acquisition unit 11, a keypoint detection unit 12, an area division unit 13, a determination unit 14, a color channel conversion unit 15, a binarization processing unit 16, a candidate extraction unit 17, a degree of separation calculation unit 18, a color channel selection unit 19, a hand image detection unit 20, a biometric information extraction unit 21, a matching unit 22, and a storage unit 23. Note that hardware such as a dedicated circuit may be used as the acquisition unit 11, the keypoint detection unit 12, the area division unit 13, the determination unit 14, a color channel conversion unit 15, a binarization processing unit 16, a candidate extraction unit 17, a degree of separation calculation unit 18, a color channel selection unit 19, a hand image detection unit 20, a biometric information extraction unit 21, a matching unit 22, and a storage unit 23.

[0018] (Hand Image Detection Processing) Fig. 2 is a flowchart showing an example of hand image detection processing executed by detection device 100. An example of the operation of detection device 100 will be described with reference to Fig. 2 .

[0019] The acquisition unit 11 acquires an original image including a hand image and a background image from the camera 107 (step S1).

[0020] Next, the key point detection unit 12 detects a plurality of key points indicating hand parts in the original image acquired by the acquisition unit 11 (step S2). For example, the key point detection unit 12 detects 21 key points P ０ ~P ２０ Detect the key point P ０ is a point on the boundary between the palm and the wrist. １ is a point at the base of the thumb. Key point P ２ is a point on the second joint of the thumb. ３ is a point on the first joint of the thumb. ４ is a point on the tip of the thumb. ５ is a point at the base of the index finger. ６ is a point on the second joint of the index finger. ７ is a point on the first joint of the index finger. ８ is a point at the tip of the index finger. ９ is a point at the base of the middle finger.１０ is a point on the second joint of the middle finger. １１ is a point on the first joint of the middle finger. １２ is a point at the tip of the middle finger. １３ is a point at the base of the ring finger. １４ is a point on the second joint of the ring finger. １５ is a point on the first joint of the ring finger. １６ is a point on the tip of the ring finger. １７ is a point at the base of the little finger. １８ is a point on the second joint of the little finger. １９ is a point on the first joint of the little finger. ２０ is a point on the tip of the little finger.

[0021] These key points ０ ~P ２０ can be detected from an RGB image. For example, keypoint P ０ ~P ２０ is obtained by detecting a bounding box that indicates the hand region, and then detecting each part of the hand. For example, keypoint P ０ ~P ２０ can be detected using the method described in Non-Patent Document 1, "T. Simon, H. Joo, I. Matthews, and Y. Sheikh, 'Hand keypoint detection in single images using multiview bootstrapping,' in CVPR, 2017." ０ ~P ２０ is illustrated in FIG.

[0022] Next, the region dividing unit 13 obtains a plurality of small regions A to D from the original image using the key points (step S3). The small regions are smaller than the original image. FIG. 3(b) is a diagram illustrating the small regions A to D. The black circles drawn with black lines represent the key points P ０ , Key Point P ６ , and key point P１８ The area D is a circle passing through the key point P ０ , Key Point P ５ , Key Point P １７ The small area B is a triangle with the black circle and the key point P ６ and key point P ５ and the line connecting the key point P ５ and key point P １３ and the line connecting the key point P １３ and key point P １４ The area is the area obtained by the line connecting the small areas B and D. Small area A is the area to the left of small areas B and D in the black circle. Small area C is the area to the right of small areas B and D in the black circle.

[0023] Next, the determination unit 14 determines whether each small region is contained within the hand when viewed in a planar view (step S4). Small region D is entirely contained within the hand, so it is determined as "Yes." Small regions A to C include the background, so they are determined as "No."

[0024] If the determination in step S4 is "Yes", the determination unit 14 determines that the small region determined as "Yes" in step S4 is a hand region (step S5).

[0025] If the result of the determination in step S4 is "No," the color channel conversion unit 15, the binarization processing unit 16, the candidate extraction unit 17, the separation degree calculation unit 18, and the color channel selection unit 19 separate the small region for which the result of the determination in step S4 is "No" into a hand region and a background region (step S6), as will be described in detail later.

[0026] Next, the hand image detection unit 20 combines the hand areas of the small areas obtained in steps S5 and S6 to detect a hand image (step S7).

[0027] 4 is a flowchart illustrating the details of step S6. The process in FIG. 4 is executed for each small area for which the answer in step S4 is "No." In this embodiment, the process in FIG. 4 is executed for each of small areas A to C.

[0028] First, the color channel conversion unit 15 converts a target small region into multiple color channel images (step S11). The multiple color channel images include at least two types of color channels: a first color channel image and a second color channel image. The color channels may be, for example, R (Red), G (Green), B (Blue), H (Hue), S (Saturation), V (Value / Brightness), Y (Luminance), U (Color Difference), and V (Color Difference). For example, the color channel conversion unit 15 obtains images of each color channel (hereinafter referred to as color channel images) for the target small region. Note that the multiple color channel images may include color channel images of three or more types of color channels.

[0029] Next, the binarization processing unit 16 performs binarization processing on each color channel image using a predetermined threshold value to obtain hand candidates and background candidates (step S12). For example, hand candidates are represented in white, and background candidates are represented in black.

[0030] Next, the separability calculation unit 18 creates a color histogram (a frequency distribution of pixel values) of hand candidates and a color histogram of background candidates for each color channel image before binarization, and calculates a first separability (step S13). For example, binarization is performed on the R image to obtain hand candidates and background candidates. Next, a color histogram of hand candidates and a color histogram of background candidates are created for the R image before binarization. The first separability of the R image is calculated by overlaying these histograms.

[0031] Next, the separability calculation unit 18 performs morphology processing on at least one of the hand candidates and the background candidates for each color channel image after the binarization processing. The morphology processing is a process of filling in gaps in the region. The separability calculation unit 18 calculates a second separability from the region difference before and after the morphology processing (step S14).

[0032] Next, the color channel selector 19 selects the color channel that maximizes the sum of the first and second separation degrees, thereby obtaining the separation result into the palm and the background (step S15). For example, by normalizing the first and second separation degrees, the first and second separation degrees can be quantified on the same scale. For example, by expressing the first and second separation degrees as numerical values ​​between 0 and 100, the sum of the first and second separation degrees can be compared for each color channel using the same criteria.

[0033] FIG. 5 illustrates the details of steps S5 to S7 in FIG. 2. As illustrated in FIG. 5, multiple color channel images are obtained for each of small regions A to C. For each of small regions A to C, the color channel image that maximizes the sum of the first and second separation degrees is selected. In the example of FIG. 5, the R image is selected for small region A, the G image is selected for small region B, and the R image is selected for small region C. A palm region is then synthesized from the image after binarization of the selected color channel images. For small region D, binarization is performed without selecting a color channel. The synthesized palm region is shaped by morphological processing. Extracting from the original image an area corresponding to the area shaped by morphological processing can improve the accuracy of hand image detection. Note that, for example, the hand image is an image showing the palm region.

[0034] 6 to 9 are diagrams illustrating the details of calculation of the first and second degrees of separation. Fig. 6 and Fig. 7 are diagrams illustrating the case where the degree of separation is large. Fig. 8 and Fig. 9 are diagrams illustrating the case where the degree of separation is small.

[0035] FIG. 6 illustrates a color histogram when the first degree of separation is large. In FIG. 6, the color histogram of a hand candidate in a color channel image and the color histogram of a background candidate in a color channel image are superimposed. When the first degree of separation is large, the separation between the background candidate and the hand region across the binarization threshold is high, as illustrated in FIG. 6. For example, the first degree of separation can be defined as the size of the interval between the pixel value at the peak position of the color histogram of the background candidate and the pixel value at the peak position of the color histogram of the hand candidate. The larger the interval, the larger the value of the first degree of separation, and the smaller the interval, the smaller the value of the first degree of separation. Alternatively, the first degree of separation may be the low frequency of the bottom between two peaks. Alternatively, the first degree of separation may be a value that reflects both the interval and the low frequency.

[0036] Fig. 7 is a diagram illustrating the details of a case where the second degree of separation is large. The upper diagram in Fig. 7 is a binarized image. The lower diagram in Fig. 7 is an image obtained by performing morphological processing on the binarized image. When the second degree of separation is large, the difference in area between before and after morphological processing becomes small, as illustrated in Fig. 7.

[0037] 8 is a diagram illustrating a color histogram when the first degree of separation is small. In FIG. 8, the color histogram of the hand candidate in the color channel image and the color histogram of the background candidate in the color channel image are superimposed. When the first degree of separation is small, as illustrated in FIG. 8, the separability between the background region and the palm region in the color histogram becomes low on both sides of the binarization threshold.

[0038] Fig. 9 is a diagram illustrating the details when the second degree of separation is small. The upper diagram in Fig. 9 is a binarized image. The lower diagram in Fig. 9 is an image obtained by performing morphological processing on the binarized image. When the second degree of separation is small, the difference in area between before and after morphological processing becomes large, as illustrated in Fig. 9.

[0039] The biometric authentication process using the detected palm area will be described below.

[0040] (Registration Process) FIG. 10 is a flowchart showing an example of registration process executed by the detection device 100. As shown in FIG.

[0041] 10 , the acquisition unit 11 acquires the original image captured by the camera 107 and the ID (identification information) of the registrant (step S21). The ID is input by the registrant or the like using the input device 106.

[0042] Next, the hand image detection unit 20 detects a hand image by the hand image detection process described above (step S22).

[0043] Next, the biometric information extraction unit 21 extracts biometric information from the hand image detected in step S22 (step S23). The biometric information may be veins, palm prints, etc., but is not limited to such information as long as it is biometric information obtainable from a hand image.

[0044] The storage unit 23 associates the biometric information extracted in step S23 with the ID acquired in step S21 and stores it as registered biometric data (step S24). The above process completes the registration process. For example, by repeating this registration process multiple times, the biometric information of multiple registrants can be registered.

[0045] (Matching Process) FIG. 11 is a flowchart showing an example of the matching process executed by the detection device 100. As shown in FIG.

[0046] As illustrated in FIG. 11, the acquisition unit 11 acquires an original image captured by the camera 107 (step S31).

[0047] Next, the hand image detection unit 20 detects a hand image by the hand image detection process described above (step S32).

[0048] Next, the biometric information extraction unit 21 extracts biometric information from the hand image detected in step S32 as biometric data for matching (step S33).

[0049] The matching unit 22 performs authentication by matching the matching biometric data extracted in step S33 with each registered biometric data stored in the storage unit 23 (step S34). Specifically, it determines whether the similarity between each registered biometric data stored in the storage unit 23 and the matching biometric data extracted in step S33 is equal to or greater than a threshold. If there is registered biometric data whose similarity is equal to or greater than the threshold, it determines that the matching biometric data is biometric information of the person with the ID of the registered biometric data. The matching unit 22 outputs the determination result.

[0050] According to this embodiment, a degree of separation, which is an index for determining the separability between a background image and a hand image, is calculated from each of a first color channel image and a second color channel image for an original image including a background image and a hand image. A color channel is selected based on the calculated degree of separation. The selected color channel is used to detect the hand image included in the original image. Because the hand image is detected using a color channel with a high degree of separation, the hand image can be accurately separated from the background image regardless of the shooting conditions. This improves the accuracy of hand image detection.

[0051] The degree of separation is not particularly limited as long as it is an index for determining the degree of separation between the background image and the hand image.

[0052] However, the accuracy of detecting the separability of the background candidate and the hand candidate is improved by using the first separability calculated from the color histograms calculated for each of the background candidate and the hand candidate, and the accuracy of detecting the separability of the background candidate and the hand candidate is improved by using the second separability according to the area difference before and after morphological processing performed on at least one of the background candidate and the hand candidate.

[0053] Furthermore, by determining the separability of background candidates and hand candidates for each of multiple small regions obtained from the original image, it becomes possible to select a color channel for each small region. This further improves the accuracy of detecting hand images from the original image. By obtaining small regions using key points related to the hand parts, the influence of individual differences in the palm can be reduced. Therefore, the influence of individual differences such as differences in palm size and shape can be reduced. By not selecting a color channel for small regions determined to be located inside the hand, the amount of processing required to detect hand images can be reduced.

[0054] In the above examples, the color channel selection unit 19 is an example of a color channel selection unit that selects a color channel based on a separability, which is calculated from each of a first color channel image and a second color channel image of an original image including a background image and a hand image, and is an index for determining the separability between the background image and the hand image. The hand image detection unit 20 is an example of a hand image detection unit that detects a hand image included in the original image using the color channel selected by the color channel selection unit. The binarization processing unit 16 is an example of a binarization processing unit that binarizes each of the first color channel image and the second color channel image. The candidate extraction unit 17 is an example of a candidate extraction unit that extracts background candidates and hand candidates from each of the binarized first color channel image and the second color channel image. The separability calculation unit 18 is an example of a frequency distribution calculation unit that calculates a frequency distribution of pixel values ​​for each of the background candidates and the hand candidates. The separability calculation unit 18 is also an example of a morphology processing unit that performs morphology processing on at least one of the background candidates and the hand candidates. The region dividing unit 13 is an example of a region dividing unit that divides each of the first color channel image and the second color channel image into a plurality of regions. The determination unit 14 is an example of a determination unit that uses key points to determine whether the plurality of regions include a background.

[0055] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0056] REFERENCE SIGNS LIST 11 Acquisition unit 12 Keypoint detection unit 13 Region division unit 14 Determination unit 15 Color channel conversion unit 16 Binarization processing unit 17 Candidate extraction unit 18 Separability calculation unit 19 Color channel selection unit 20 Hand image detection unit 21 Biometric information extraction unit 22 Matching unit 23 Storage unit 100 Detection device 105 Display device 107 Camera

Claims

1. a color channel selection unit that selects a color channel based on a degree of separation that is an index for determining the separability between the background image and the hand image, the degree being calculated from a first color channel image and a second color channel image for an original image including a background image and a hand image; a hand image detection unit that detects the hand image included in the original image by using the color channel selected by the color channel selection unit.

2. a binarization processing unit that binarizes the first color channel image and the second color channel image, respectively; a candidate extraction unit that extracts background candidates and hand candidates from each of the binarized first color channel image and the binarized second color channel image; a frequency distribution calculation unit that calculates a frequency distribution of pixel values ​​for each of the background candidates and the hand candidates, The detection device according to claim 1 , wherein the degree of separation includes a first degree of separation calculated from a frequency distribution of pixel values.

3. 3. The detection device according to claim 2, wherein the first degree of separation is a degree of separation corresponding to a pixel value difference between two frequency peaks in a frequency distribution of the pixel values.

4. a binarization processing unit that binarizes the first color channel image and the second color channel image, respectively; a candidate extraction unit that extracts background candidates and hand candidates from each of the binarized first color channel image and the binarized second color channel image; a morphology processing unit that performs morphology processing on at least one of the background candidate and the hand candidate, The detection device according to claim 1 , wherein the degree of separation includes a second degree of separation corresponding to a difference in area before and after the morphological processing.

5. a region dividing unit that divides each of the first color channel image and the second color channel image into a plurality of regions; The detection device according to claim 1 , wherein the color channel selection unit selects a color channel in accordance with the degree of separation of each of the plurality of regions.

6. The detection device according to claim 5 , wherein the region division unit divides each of the first color channel image and the second color channel image into the plurality of regions according to key points regarding hand parts detected from the original image.

7. a determination unit that determines whether the plurality of regions includes a background using the key points; 7. The detection device according to claim 6, wherein for an area determined by the determination unit not to include a background, the color channel selection unit does not select a color channel, and the hand image detection unit detects the area as the hand image.

8. selecting a color channel based on a degree of separation, the degree being an index for determining the degree of separation between the background image and the hand image, the degree being calculated from a first color channel image and a second color channel image for an original image including a background image and a hand image; The method further comprises the step of: detecting the hand image in the original image using a selected color channel, the step being performed by a computer.

9. On the computer, A process of selecting a color channel based on a degree of separation, which is calculated from a first color channel image and a second color channel image of an original image including a background image and a hand image, and is an index for determining the degree of separation between the background image and the hand image; and detecting the hand image contained in the original image by using the selected color channel.