An improved method for extracting region of interest (ROI) in palmprint and palm vein recognition

Abstract

The present invention discloses an improved method for extracting a region of interest (ROI) in palm print and palm vein recognition, including: dynamic threshold binarization, morphological operation for boundary extraction, key point determination, frequency domain processing to remove high-frequency noise, palm center of mass determination and ROI region interception; in the step of determining the palm center of mass, density correction is performed on the basis of the center of mass calculated by the average density of the current image, the density of the finger part is reduced, and the center of mass is pulled toward the palm center. The present invention is applicable to a variety of hand shapes and can pre-process palm vein images simply and effectively; the method is simple and can be easily transplanted to the front-end acquisition device; the improved method provides high-quality pre-processed palm vein images for the practical application of the entire palm vein recognition, and improves the accuracy of the recognition algorithm.

Description

Technical Field

[0001] The invention relates to the technical field of palmprint recognition, and in particular to an improved method for extracting a region of interest (ROI) in palmprint and palm vein recognition. Background Art

[0002] Palm vein recognition can be widely used in access control systems, attendance systems, education and finance, and is a biometric recognition technology with broad application prospects. The palm vein recognition process includes four stages: palm vein image acquisition, image preprocessing, feature extraction, and recognition verification.

[0003] In order to make the palm vein image meet the requirements of feature extraction, it is necessary to preprocess the palm vein image in the data set and extract the region of interest (ROI). Palm vein image preprocessing mainly includes key point positioning, ROI extraction, image enhancement and other steps. Among them, the ROI extraction mainly includes the following methods:

[0004] Method 1: Finger gap point interception method

[0005] Step 1: Dynamic Thresholding for Binarization

[0006] By calculating the grayscale histogram of the image, the grayscale value of the image is divided into foreground and background. According to the shape of the grayscale histogram, a threshold T is dynamically selected to binarize the image. The foreground pixel (palm part) is set to 1 and the background pixel is set to 0.

[0007] Step 2: Boundary Extraction Using Morphological Operations

[0008] The morphological erosion operation is used to remove isolated noise points in the binary image to obtain a clearer boundary; the boundary extraction is achieved by calculating the difference between the eroded image and the original image.

[0009] Step 3: Euclidean Distance Calculations and Key Point Determination

[0010] The palm boundary pixels are extracted from the binary image, and then the Euclidean distance from each boundary pixel to the palm center of mass is calculated. By analyzing the extreme points of the distance function, the valley points between the finger tips and fingers are determined, and these valley points are used to locate the ROI.

[0011] Step 4: Frequency Domain Processing to Remove High-Frequency Noise

[0012] The Euclidean distance function is Fourier transformed to decompose it into different frequency components. By setting the energy threshold, the high-frequency noise components are removed, and only the low-frequency components are retained. Then the inverse Fourier transform is performed to reconstruct the distance function, thereby obtaining a smooth distance function and accurately locating the valley point.

[0013] Step 5: Horizontal rotation positioning of valley points

[0014] Since the palm image has high brightness and the background image has low brightness, the pixel values ​​between the two are significantly different. In order to improve the accuracy of positioning, the connecting area of ​​the palm and the background is used as the candidate area of ​​the key point. Generally, the gap between each finger of the palm is selected as the key point reference. Figure 1a As shown, the gap A between the index finger and the middle finger, the gap B between the middle finger and the ring finger, and the gap C between the ring finger and the little finger are selected as the key points of the palm; the image after the key points are located is first rotated vertically using affine transformation with point A as the rotation center, so that the line segment AC is horizontal.

[0015] Step 6: ROI area capture

[0016] Assume the length of the AC line segment is m, L can be set to 20 pixels, and then select a point along the perpendicular line of the AC line segment (L+m / 2) as the center of the square to extract a square with a side length of m. This ensures that the texture-rich palm area is extracted, such as Figure 1b The red square area shown is used as the extracted palm vein ROI area.

[0017] Method 2: Palm interception method

[0018] Method 2 is based on the first four steps of method 1 and introduces the image centroid for ROI interception, that is, the first four steps of method 2 are consistent with the first four steps of method 1.

[0019] Step 5: Determine the center of mass of the palm using geometric moments

[0020] Calculate the geometric moment function of an image or image region (contour). Geometric moment is a basic tool in image analysis and can provide information such as the shape characteristics and center position of the image.

[0021]

[0022] In the formula,

[0023] m 00is the zero-order moment, which represents the area of ​​the image (the total number of white pixels in a binary image);

[0024] m 01 and m 10 are all first-order moments, used to calculate the coordinates of the center of mass O, where:

[0025] m 10 Represents the first-order moment of the image in the horizontal direction, and calculates the x-coordinate of the center of mass O;

[0026] m 01 Represents the first-order moment of the image in the vertical direction, and calculates the y coordinate of the center of mass O;

[0027] Step 6: ROI area capture

[0028] With the centroid O as the center and the length m of the AC line segment as the side length, a square area is intercepted, which is the ROI area, such as Figure 2 shown.

[0029] Method 3: Deep learning model method, such as Figure 3 As shown;

[0030] Step 1: Key point positioning

[0031] Using the improved lightweight HRnet network, four key points (valley points between fingers) are first located in the contactless palm vein image. These key points are accurately located through the multi-resolution fusion characteristics of the HRnet network. By maintaining the high-resolution feature map and performing multiple cross-resolution fusions, the key point positions with high spatial accuracy are finally output.

[0032] Step 2: Construct a local coordinate system and locate the ROI

[0033] According to the four key points located, the local palm coordinate system is constructed. In the constructed local coordinate system, the ROI of the palm vein image is located by geometric method.

[0034] Step 3: Affine transformation to correct ROI

[0035] Using an affine transformation algorithm, the located ROI is corrected to eliminate the effects of translation, rotation, and scaling in the palm vein image.

[0036] The above methods all have different defects in their respective application processes, which are specifically reflected in:

[0037] Method 1: Finger gap point interception method

[0038] The core of this method is to accurately identify three key valley points, namely the gap A between the index finger and the middle finger, the gap B between the middle finger and the ring finger, and the gap C between the ring finger and the little finger as the key points of the palm. Figure 1a shown.

[0039] However, this method is only suitable for determining the valley point when the fingers are open. When the palm is closed, the valley point will move up to the gap between the fingers, such as Figure 4a The three red dots shown in the figure; the recognition error of the palm key points will directly affect the interception of the ROI area, such as Figure 4b The captured ROI area is shown.

[0040] The reason for its failure is that the original method used the maximum depression point algorithm to find the three largest depression points (called valley points) in the palm when identifying valley points; therefore, when the fingers are close together, the algorithm fails to locate the three valley points A, B, and C.

[0041] Method 2: Palm interception method

[0042] This method introduces the image centroid as the center of the finger, and uses the horizontal distance between the two valley points A and C as the side length to intercept a square as the ROI. Similarly, when the fingers are closed, the palm and fingers are integrated, and the image centroid will be significantly higher than the palm center (when the fingers are facing up), such as Figure 5 The blue × dot shown is the determined image centroid. The ROI area intercepted by the center point determined by the original centroid coordinate formula is similar to Figure 4b .

[0043] Therefore, both the finger gap point interception method and the palm interception method are not suitable for intercepting the image region of interest ROI when the fingers are closed.

[0044] Method 3: Deep Learning Model Method

[0045] This method uses a deep neural network, which can identify the valley points in Method 1 and Method 2 by learning and training a variety of hand shape key points. However, the main disadvantages of this method are in two aspects:

[0046] First, the number of palm near-infrared images collected is small. In the currently public data sets, the palm images have relatively standard hand shapes, and there is very little data on fingers closing.

[0047] Second, in the application scenarios of palm print and palm vein recognition, the computing power of the front-end near-infrared camera equipment is very low, and it is impossible to transplant the neural network algorithm package to the front-end device (the commonly used HRNet-W48 model has parameters of about 63.6M). After the palm image is collected, it must be transmitted to the server for centralized processing. Deep neural networks such as HRnet may take up a lot of computing resources to process large amounts of image data in real time under high concurrency and multi-user conditions. Moreover, subsequent palm vein feature extraction, palm vein recognition comparison and verification, etc., also require more computing resources.

[0048] The above two shortcomings make this method difficult to apply in practical scenarios. Summary of the invention

[0049] In view of the deficiencies in the prior art, the present invention provides an improved method for extracting a region of interest (ROI) in palm print and palm vein recognition, which is suitable for extracting palm ROI regions under various hand shapes.

[0050] The invention discloses an improved method for extracting a region of interest (ROI) in palm print and palm vein recognition, comprising: dynamic threshold binarization, morphological operation for boundary extraction, key point determination, frequency domain processing to remove high-frequency noise, palm center determination and ROI region interception;

[0051] In the step of determining the palm center of mass, the center of mass coordinates (X centroid , Y centroid )for:

[0052]

[0053]

[0054] In the formula,

[0055] m 00 is the zero-order moment, representing the area of ​​the image;

[0056] m 10 Represents the first-order moment of the image in the horizontal direction, and calculates the x-coordinate of the center of mass O;

[0057] ρ0 is the density at the center position y0, y is the vertical position, and k is the rate at which the density decreases linearly upward;

[0058] In the ROI region interception step, a square region is intercepted with the determined centroid point as the center and the length of the line segment between the two determined key points as the side length, which is the ROI region.

[0059] As a further improvement of the present invention, in the step of determining key points,

[0060] All the concave points in the maximum contour are clustered using the DBSCAN algorithm to obtain all the concave points in the palm image;

[0061] From all the sunken points obtained, find the three points with the smallest y coordinates, and then select the two points with the largest and smallest x coordinates from these three points as two key points. Compared with the prior art, the beneficial effects of the present invention are:

[0062] The improved method of the present invention is applicable to various hand shapes, can simply and effectively perform preprocessing of palm vein images, and improves the adaptability to complex image acquisition situations in the palm print and palm vein recognition process;

[0063] The improved method of the present invention can be transplanted into the front-end acquisition device to improve the ROI acquisition speed and quality;

[0064] The improved method of the present invention provides high-quality pre-processed palm vein images for the practical application of the entire palm vein recognition, thereby improving the accuracy of the recognition algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1a It is a schematic diagram of the selection of key points in the finger gap point interception method;

[0066] Figure 1b This is a schematic diagram of the ROI area interception in the finger gap point interception method;

[0067] Figure 2 This is a schematic diagram for determining the center of mass O in the palm interception method;

[0068] Figure 3 Schematic diagram of determining ROI area for deep learning model method;

[0069] Figure 4a It is a schematic diagram of key points determined by the finger gap point interception method when the fingers are closed;

[0070] Figure 4b It is a schematic diagram of the ROI area intercepted by the finger gap point interception method when the fingers are closed;

[0071] Figure 5 A schematic diagram of the image centroid determined by the palm interception method when the fingers are closed;

[0072] Figure 6 It is a flow chart of the improved method for extracting region of interest ROI in palm print and palm vein recognition disclosed by the present invention;

[0073] Figure 7a The valley point is determined by the finger gap point interception method in Example 1;

[0074] Figure 7b The valley point is determined by the palm interception method in Example 1;

[0075] Figure 7c The valley point determined by the improved method of the present invention in Example 1;

[0076] Figure 7d The ROI area intercepted by the finger gap point interception method in Example 1;

[0077] Figure 7e The ROI area intercepted by the palm interception method in Example 1;

[0078] Figure 7fThe ROI region captured by the improved method of the present invention in Example 1;

[0079] Figure 8a The valley point is determined by the finger gap point interception method in Example 2;

[0080] Figure 8b The valley point is determined by the palm interception method in Example 2;

[0081] Figure 8c The valley point is determined by the improved method of the present invention in Example 2;

[0082] Figure 8d The ROI area intercepted by the finger gap point interception method in Example 2;

[0083] Figure 8e The ROI area intercepted by the palm interception method in Example 2;

[0084] Figure 8f The ROI region captured by the improved method of the present invention in Example 2;

[0085] Figure 9a The valley point is determined by the finger gap point interception method in Example 3;

[0086] Figure 9b The valley point is determined by the palm interception method in Example 3;

[0087] Fig.9c The valley point is determined by the improved method of the present invention in Example 3;

[0088] Figure 9d The ROI area intercepted by the finger gap point interception method in Example 3;

[0089] Fig.9e The ROI area intercepted by the palm interception method in Example 3;

[0090] Figure 9f The ROI region captured by the improved method of the present invention in Example 3;

[0091] Fig.10a The valley point is determined by the finger gap point interception method in Example 4;

[0092] Fig.10b The valley point is determined by the palm interception method in Example 4;

[0093] Fig.10c The valley point is determined by the improved method of the present invention in Example 4;

[0094] Fig.10d The ROI area is captured by the finger gap point capture method in Example 4;

[0095] Fig.10e The ROI area is captured by the palm interception method in Example 4;

[0096] Fig.10f This is the ROI area captured by the improved method of the present invention in Example 4. DETAILED DESCRIPTION

[0097] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0098] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0099] like Figure 6 As shown, the present invention provides an improved method for extracting a region of interest (ROI) in palm print and palm vein recognition, comprising:

[0100] The first step is dynamic threshold binarization. The specific method is the same as the first step of the finger gap point interception method (method one) and the palm interception method (method two);

[0101] Step 2: Perform boundary extraction using morphological operations. The specific method is the same as the second step of the finger gap point interception method (method 1) and the palm interception method (method 2).

[0102] Step 3: Determine key points based on clustering:

[0103] Different from the existing method of using the maximum concave point algorithm to determine the valley point, in the third step, the present invention clusters all the concave points in the maximum contour using the DBSCAN algorithm to obtain all the concave points in the palm image; from all the obtained concave points, the three points with the smallest y coordinates are found, and then the two points with the largest and smallest x coordinates are selected from the three points as two key points, which are used as the reference distance between the two fingers.

[0104] The present invention no longer uses the acquired valley point as the ROI positioning coordinate point, but uses the lateral distance between the two determined key points as the side length of the ROI interception. The principle is that although the longitudinal coordinates of these key points will change with the closing of the fingers, the lateral coordinates remain basically consistent, so the side length intercepted as the ROI area is relatively stable. For example, when the method is used for four fingers closed or partially closed, or even six fingers, the gap A between the index finger and the middle finger, the gap B between the middle finger and the ring finger, and the gap C between the ring finger and the little finger cannot be accurately determined as the key points of the palm, and the method of intercepting the ROI with wrong key point coordinates cannot be applied.

[0105] Step 4: Frequency domain processing to remove high-frequency noise. The specific method is the same as the fourth step of the finger gap point interception method (method one) and the palm interception method (method two);

[0106] Step 5: Determine the center of mass of the palm based on density correction;

[0107] Method 2 assumes that the captured image does not include the fingers when determining the palm center of mass, so the method of using average density to calculate the center of mass is effective. However, in actual applications, when capturing palm images, it is common for the fingers to be captured into the image, and the proportion of fingers in the image varies greatly. Some may capture part of the fingers, while others may capture all of the fingers.

[0108] In order to eliminate the influence of the finger part on the calculation of the center of mass of the palm part, the improved method of the present invention is to make density correction based on the center of mass calculated by the average density of the current image, and pull the center of mass toward the palm. When the finger is pointing upward, the part above the longitudinal coordinate of the center of mass calculated by the average density is mainly composed of fingers, and the part below the longitudinal coordinate is all palm. For this reason, a method for calculating the center of mass under variable density of palm images is introduced. The specific method is as follows:

[0109] If the density of the object (palm) is the largest in the center, the density decreases linearly upwards, and the density downwards is the same as the center density, then a specific correction is required to the calculation of the center of mass. Assume that the density of the object in the vertical direction y can be described by the following function:

[0110]

[0111] Where ρ0 is the density at the center position y0, k is the rate at which the density decreases linearly upward, and y is the vertical position;

[0112] Corrected centroid calculation formula

[0113] a) Center of mass in the X direction

[0114] In the X direction, the formula for calculating the center of mass remains the same:

[0115] b) Center of mass in the Y direction

[0116] For the calculation of the center of mass in the Y direction, the change in density needs to be taken into account:

[0117]

[0118] Among them, ρ(y) has two cases according to the position of y, and the above formula is transformed into:

[0119]

[0120] In order to simplify this calculation, the above integral can be decomposed and calculated separately:

[0121] Upper part (y ≥ y0):

[0122]

[0123] The lower part (y <y0):

[0124]

[0125] Correct the centroid calculation, and the final centroid calculation formula is:

[0126]

[0127] Therefore, the centroid coordinates (X centroid , Y centroid )for:

[0128]

[0129] In the formula,

[0130] m 00 is the zero-order moment, representing the area of ​​the image;

[0131] m 10 Represents the first-order moment of the image in the horizontal direction, and calculates the x-coordinate of the center of mass O;

[0132] ρ0 is the density at the center position y0, y is the vertical position, and k is the rate at which the density decreases linearly upward.

[0133] Step 6: ROI area interception;

[0134] The ROI area is intercepted by taking the determined centroid point as the center and the length of the line segment between the two determined key points as the side length.

[0135] Embodiment 1, fingers close together:

[0136] The valley points located by method 1 (finger gap point interception method) are as follows: Figure 7a As shown;

[0137] The valley points located by method 2 (palm interception method) are as follows: Figure 7b As shown;

[0138] The valley points located by the improved method of the present invention are as follows: Figure 7c shown.

[0139] It can be seen from this that methods 1, 2 and the improved method all use the same valley point positioning algorithm, and none of them can accurately mark the valley points of the palm.

[0140] The ROI area cut by method 1 is as follows Figure 7d As shown;

[0141] The ROI area intercepted by method 2 is as follows Figure 7e As shown;

[0142] The ROI area intercepted by the improved method of the present invention is as follows: Figure 7f shown.

[0143] From this we can see that:

[0144] Method 1, since it relies on accurate positioning of valley points for ROI interception, one-third of the ROI interception includes the fingers, and the palm part is missing for verification;

[0145] Method 2: Mass point positioning was performed. The center of mass was obviously not at the center of the palm, so the ROI interception also included one-third of the finger.

[0146] The improved method of the present invention significantly improves the particle positioning, which is close to the palm position, and the ROI interception can cover the main area of ​​interest of the palm.

[0147] Example 2, fingers are closed, fingers are only half:

[0148] The valley points located by method 1 are as follows: Figure 8a As shown;

[0149] The valley points located by method 2 are as follows: Figure 8b As shown;

[0150] The valley points located by the improved method of the present invention are as follows: Figure 8c shown.

[0151] From this we can see that: none of the three methods can accurately determine the valley point; but method three, finding the three concave points with the smallest y coordinates in the palm, can obtain the reference size of the palm width.

[0152] The ROI area cut by method 1 is as follows Figure 8d As shown;

[0153] The ROI area intercepted by method 2 is as follows Figure 8e As shown;

[0154] The ROI area intercepted by the improved method of the present invention is as follows: Figure 8f shown.

[0155] From this we can see that:

[0156] Method 1: The valley point positioning is inaccurate. The algorithm rotates the image horizontally according to the normal valley point positioning, and then performs ROI interception. The intercepted part exceeds the palm part.

[0157] Method 2: The particle positioning is obviously higher than the palm. After ROI interception, the finger part occupies more than one-third, and the palm area of ​​interest is seriously lost;

[0158] In the improved method of the present invention, the particle positioning is close to the palm, and the ROI interception covers the palm ROI area of ​​interest.

[0159] Example 3: Fingers separated, fingers and palms filled with images

[0160] The valley points located by method 1 are as follows: Figure 9a As shown;

[0161] The valley points located by method 2 are as follows: Figure 9b As shown;

[0162] The valley points located by the improved method of the present invention are as follows: Fig.9c shown.

[0163] It can be seen from this that the valley point positioning methods 1 and 2 lack point C between the ring finger and the little finger, and the improved method of the present invention can prepare to mark several key valley points.

[0164] The ROI area cut by method 1 is as follows Figure 9d As shown;

[0165] The ROI area intercepted by method 2 is as follows Fig.9e As shown;

[0166] The ROI area intercepted by the improved method of the present invention is as follows: Figure 9f shown.

[0167] From this we can see that:

[0168] Method 1: Since the valley point C is missing, the horizontal rotation direction is reversed, and the ROI is intercepted to the edge of the palm, which loses a lot of areas of interest;

[0169] Method 2: The centroid is located slightly above the palm, and the ROI interception area is also in the upper part of the palm, so the area of ​​interest is not fully covered;

[0170] The improved method of the present invention locates the centroid close to the palm and effectively intercepts the ROI area.

[0171] Example 4, fingers separated, more wrists

[0172] The valley points located by method 1 are as follows: Fig.10a As shown;

[0173] The valley points located by method 2 are as follows: Fig.10b As shown;

[0174] The valley points located by the improved method of the present invention are as follows: Fig.10c shown.

[0175] It can be seen from this that the valley point positioning methods 1 and 2 also lack point C between the ring finger and the little finger. The improved method of the present invention can prepare to mark several key valley points.

[0176] The ROI area cut by method 1 is as follows Fig.10d As shown;

[0177] The ROI area intercepted by method 2 is as follows Fig.10e As shown;

[0178] The ROI area intercepted by the improved method of the present invention is as follows: Fig.10f shown.

[0179] From this we can see that:

[0180] Method 1: The ROI interception result is similar to that of Example 3, and a small part of the ROI part exceeds the palm area;

[0181] Method 2: The mass point is positioned close to the palm (the wrist part at the bottom offsets the weight of the finger part, and the center of mass can be close to the palm). Due to the lack of point C, the ROI interception has a short side length and a small ROI area.

[0182] In the improved method of the present invention, the centroid positioning is substantially consistent with the palm center, and the ROI interception effectively covers the region of interest;

[0183] In summary, the improved method of the present invention can better locate the palm for the above-mentioned major scenes of palm vein collection images, can determine a more appropriate interception side length, and the ROI interception result can better cover the region of interest.

[0184] The advantages of the present invention are:

[0185] The improved method of the present invention can be applied to the changes of hand shape in palm image acquisition in practical application links, such as partial or complete closing of fingers, six fingers, etc.;

[0186] The improved method of the present invention can be applied to practical applications, where the palm image acquisition includes some fingers or all fingers, and the fingers are wearing decorations, etc.

[0187] The improved method of the present invention requires less computing resources, can be used for image preprocessing at the server end, and can also be transplanted to a mobile image acquisition device.

[0188] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An improved method for extracting a region of interest (ROI) in palm print and palm vein recognition, comprising: Dynamic threshold binarization, morphological operation for boundary extraction, key point determination, frequency domain processing to remove high-frequency noise, palm center of mass determination and ROI area interception; its characteristics are: In the step of determining the key points, all the concave points in the maximum contour are clustered using the DBSCAN algorithm to obtain all the concave points in the palm image; from all the obtained concave points, three points with the smallest y coordinates are found, and then two points with the largest and smallest x coordinates are selected from these three points as two key points; In the step of determining the palm center of mass, the center of mass coordinates (X centroid , Y centroid )for: In the formula, m 00 is the zero-order moment, representing the area of ​​the image; m 10 Represents the first-order moment of the image in the horizontal direction, and calculates the x-coordinate of the center of mass O; ρ0 is the density at the center position y0, y is the vertical position, and k is the rate at which the density decreases linearly upward; In the ROI region interception step, a square region is intercepted with the determined centroid point as the center and the length of the line segment between the two determined key points as the side length, which is the ROI region.

Images