A Method for Extracting ROI of Palmprint of Palm at Any Angle and Related Devices

Abstract

The present invention discloses a method and related device for extracting the ROI of palm prints at any angle. The method includes: acquiring a palm image, performing palm segmentation on the palm image through preprocessing to obtain a palm contour image; determining a palm center positioning point and a family of finger detection circles according to the palm contour line in the palm contour image, determining finger positioning points according to the family of finger detection circles, determining the palm direction according to the palm center positioning point and the finger positioning points, and correcting the palm to a unified direction; determining the area where the fingers are located, and determining the final finger valley key points according to the curvature distribution of the contour line in the area; and intercepting a square area at the center position of the palm as the ROI image according to the finger valley key points. The present invention can efficiently and accurately extract the ROI image of the palm area, has the characteristic of high operating efficiency, and can extract the ROI image of the palm in real time on a hardware platform with low computing power and small storage.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method, a system, a terminal and a computer-readable storage medium for extracting the ROI of palmprint of a palm at any angle. Background Art

[0002] With the rapid development of information technology, biometric-based identity recognition technologies have been widely used due to their characteristics such as difficult-to-forge features, convenience and ease of use, including face, fingerprint, iris, palmprint, voiceprint, gait, etc. These technologies have been widely applied in scenarios such as mobile phone unlocking, mobile payment, access control and attendance. Compared with other biometric recognition technologies, palmprint recognition has the characteristics of high accuracy and high anti-counterfeiting, so it has received more and more attention. In addition, non-contact palmprint recognition has become the focus of research due to its significant advantage of low risk of pathogen transmission. In addition, the non-contact and non-invasive sensing method of this technology also improves user friendliness, with little user resistance and easy to promote.

[0003] In a palmprint recognition system, extracting the ROI (Region of Interest) of the palmprint can effectively avoid background interference. At the same time, the position of the ROI is located based on the valley points of the palm, which helps to reduce the intra-class differences of palm images placed in different directions, distances, etc. Therefore, the positioning accuracy of the ROI directly affects the subsequent feature extraction and matching accuracy. However, in a non-contact palmprint recognition system, palm images are collected in an open space, and there are various influencing factors, such as the posture, position of the palm and environmental illumination, etc., which makes the precise positioning of the palm ROI very challenging.

[0004] However, in the prior art, during the ROI extraction process, the palm area segmentation is inaccurate and there are many restrictions on the palm posture, and it is impossible to efficiently and accurately extract the ROI image of the palm area.

[0005] Therefore, the prior art still needs to be improved and developed. Summary of the Invention

[0006] The main purpose of the present invention is to provide a method, a system, a terminal and a computer-readable storage medium for extracting the ROI of palmprint of a palm at any angle, aiming to solve the problem that in the prior art, during the ROI extraction process, the palm area segmentation is inaccurate and there are many restrictions on the palm posture, and it is impossible to efficiently and accurately extract the ROI image of the palm area.

[0007] To achieve the above purpose, the present invention provides a method for extracting the ROI of palmprint of a palm at any angle, and the method for extracting the ROI of palmprint of a palm at any angle includes the following steps:

[0008] Obtain a palm image, and perform palm segmentation on the palm image through preprocessing to obtain a palm contour image;

[0009] Determine the palm center positioning point and the finger detection circle family based on the palm contour line in the palm contour image, determine the finger positioning point according to the finger detection circle family, determine the palm direction according to the palm center positioning point and the finger positioning point, and correct the palm to a unified direction;

[0010] Determine the area where the finger is located, and determine the final finger valley key points according to the curvature distribution of the contour line in the area;

[0011] According to the finger valley key points, intercept a square area at the center position of the palm as the ROI image.

[0012] The method for extracting the palmprint ROI of a palm at any angle, wherein, for the acquisition of the palm image, the palm image is segmented by preprocessing to obtain a palm contour image, which specifically includes:

[0013] Obtain the palm image collected by the image sensor, and perform smoothing processing on the palm image based on a Gaussian filter to obtain a smoothed palm image;

[0014] Detect the edge of the palm in the smoothed palm image based on the XDOG algorithm to obtain a robust palm edge detection image;

[0015] Perform binarization on the smoothed palm image based on the OSTU adaptive threshold segmentation algorithm, perform morphological erosion and dilation operations on the binarized palm image, and calculate the non-zero pixel connected domain of the binarized palm image based on the Seed-Filling method. Retain the region with the largest area in the non-zero pixel connected domain, and fill all other regions with 0 to obtain a binary segmentation map of the palm;

[0016] Multiply the palm edge detection image and the binary segmentation map in matrix form, and calculate the largest connected domain in the binary segmentation map after matrix multiplication again based on the Seed-Filling method. Fill all regions except the largest area with 0 to obtain the final binary segmentation map of the palm;

[0017] Obtain all continuous contour points of the palm region in the final binary segmentation map of the palm based on the contour detection algorithm to obtain a palm contour image.

[0018] The method for extracting the palmprint ROI of a palm at any angle, wherein, for determining the palm center positioning point and the finger detection circle family based on the palm contour line in the palm contour image, determining the finger positioning point according to the finger detection circle family, determining the palm direction according to the palm center positioning point and the finger positioning point, and correcting the palm to a unified direction, specifically includes:

[0019] According to the palm contour line in the palm contour image, use the maximum inscribed circle algorithm of an arbitrary polygon to calculate the maximum inscribed circle within the palm, and set the center of the circle as the palm center positioning point;

[0020] Take the palm reference point as the center of all circles in the finger detection circle family, perform convex hull detection on the palm based on the palm contour line, detect the convex points and the points farthest from the convex points, take the distance between the convex points and the points farthest from the convex points as the length of the finger, and determine the minimum radius d of the finger detection circle family according to the convex points and the points farthest from the convex points 1 and the maximum radius d 2 , select the circles within the range of the minimum radius d 1 and the maximum radius d 2 to form the finger detection circle family;

[0021] Select the detection circle with a radius of in the finger detection circle family, perform matrix multiplication on the binary image corresponding to the detection circle and the binary segmentation map of the palm to obtain the intersection area of the detection circle and the palm area, obtain the center, length and positional relationship of the intersection area based on the connected component analysis method, and record all the fan-shaped areas composed of adjacent three areas and the center of the circle. If the number of all fan-shaped areas is greater than 2 and the radian of the group with the smallest radian is less than the first preset angle, the center position of the group with the smallest radian is the final finger positioning point; otherwise, select the detection circle with a radius of in the finger detection circle family for processing until the finger positioning point is detected. Among them, for the nth detection, the radius of the selected detection circle is

[0022] If the number of all fan-shaped areas is less than two and the corresponding radian is greater than the second preset angle, it is considered that the finger is not opened, and a corresponding prompt is given to re-collect the palm;

[0023] Connect the palm center positioning point and the finger positioning point to determine the palm direction, and correct the palm to a unified direction based on geometric transformation.

[0024] The method for extracting the palmprint ROI of a palm at an arbitrary angle, wherein, to determine the area where the finger is located, and determine the final finger valley key point according to the curvature distribution of the contour line within the area, specifically including:

[0025] After the palm is oriented uniformly, confirm the area where the finger is located in combination with the position of the palm center reference point, obtain the contour line within the area based on the contour detection algorithm, calculate the curvature values of adjacent areas on the contour line based on the RTPDA method, and obtain the curvature distribution curve of the finger area contour;

[0026] Select the local comparison length as the first preset value. If the difference in curvature values between the minimum and maximum points within the local comparison length is greater than the second preset value and the curvature value of the maximum point is greater than the third preset value, retain the maximum point as the candidate extreme point, and determine the final finger valley key point according to the relative positions and distances between the candidate extreme points.

[0027] The method for extracting the ROI of the palmprint of an arbitrarily angled palm, wherein, according to the finger valley key points, a square area at the center position of the palm is intercepted as the ROI image, which specifically includes:

[0028] Establish a coordinate system according to the finger valley key points. Specifically, connect two finger valley key points to establish the y-axis of the coordinate axis, and establish the x-axis through the perpendicular bisector of the line connecting the two finger valley key points. The direction pointing to the palm is the positive direction of the x-axis;

[0029] Calculate the Euclidean distance between two finger valley key points and denote it as d. Then the center point of the square is The side length is And two sides of the square are parallel to the y-axis. Locate the four points of the square, and finally intercept the square area as the ROI image.

[0030] The method for extracting the ROI of the palmprint of an arbitrarily angled palm, wherein the kernel size of the Gaussian filter is 3*3 and the standard deviation of the Gaussian function is 3;

[0031] In the palm edge detection image, the pixel value of the edge area is 0 and the pixel value of the background area is 1;

[0032] In the binarized palm image, the pixel value of the palmprint area is 1 and the pixel value of the background area is 0.

[0033] The method for extracting the ROI of the palmprint of an arbitrarily angled palm, wherein the first preset angle is 45° and the second preset angle is 120°;

[0034] The first preset value is 50, the second preset value is 25, and the third preset value is 30.

[0035] In addition, to achieve the above object, the present invention also provides a system for extracting the ROI of the palmprint of an arbitrarily angled palm, wherein the system for extracting the ROI of the palmprint of an arbitrarily angled palm includes:

[0036] A palm image segmentation module, configured to obtain a palm image, perform palm segmentation on the palm image through preprocessing, and obtain a palm contour image;

[0037] A direction positioning and correction module, configured to determine a palm center positioning point and a finger detection circle family according to the palm contour line in the palm contour image, determine finger positioning points according to the finger detection circle family, determine the palm direction according to the palm center positioning point and the finger positioning points, and correct the palm to a unified direction;

[0038] A key point positioning module, configured to determine the area where the finger is located, and determine the final finger valley key points according to the curvature distribution of the contour line in the area;

[0039] An ROI image cropping module, configured to crop a square area at the palm center position as the ROI image according to the finger valley key points.

[0040] In addition, to achieve the above object, the present invention further provides a terminal, where the terminal includes: a memory, a processor, and a palmprint ROI extraction program for any-angle palm stored on the memory and executable on the processor. When the palmprint ROI extraction program for any-angle palm is executed by the processor, the steps of the palmprint ROI extraction method for any-angle palm as described above are implemented.

[0041] In addition, to achieve the above object, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a palmprint ROI extraction program for any-angle palm. When the palmprint ROI extraction program for any-angle palm is executed by a processor, the steps of the palmprint ROI extraction method for any-angle palm as described above are implemented.

[0042] In the present invention, a palm image is obtained, and the palm image is segmented through preprocessing to obtain a palm contour image; a palm center positioning point and a finger detection circle family are determined according to the palm contour line in the palm contour image, finger positioning points are determined according to the finger detection circle family, the palm direction is determined according to the palm center positioning point and the finger positioning points, and the palm is corrected to a unified direction; the area where the finger is located is determined, and the final finger valley key points are determined according to the curvature distribution of the contour line in the area; a square area at the palm center position is cropped as the ROI image according to the finger valley key points. The present invention can efficiently and accurately extract the ROI image of the palm area, and can be achieved even in the case of blurred palm boundaries, unclear finger gaps, arbitrary rotation angles in the horizontal plane, and incomplete fingers perceived in a non-contact palmprint recognition system. It has the characteristics of high operating efficiency and can extract the ROI image of the palm in real time on a hardware platform with low computing power and small storage. Description of the Drawings

[0043] Figure 1 is a flowchart of a preferred embodiment of the palmprint ROI extraction method for any-angle palm of the present invention;

[0044] Figure 2 It is a schematic diagram of the palm segmentation image transformation process of the palm image through preprocessing in a preferred embodiment of the palmprint ROI extraction method for palms at any angle of the present invention;

[0045] Figure 3 It is a schematic diagram of determining a family of finger detection circles based on the palm in a preferred embodiment of the palmprint ROI extraction method for palms at any angle of the present invention;

[0046] Figure 4 It is a schematic diagram of the intersection points of the respective intersection regions between the finger detection circles and the binary palm image, and the arc lengths of the fan-shaped regions formed by three adjacent intersection points and the center of the circle in a preferred embodiment of the palmprint ROI extraction method for palms at any angle of the present invention;

[0047] Figure 5 It is a recognition effect diagram of one finger being closed and all fingers being closed in a preferred embodiment of the palmprint ROI extraction method for palms at any angle of the present invention;

[0048] Figure 6 It is a palm orientation positioning diagram in a preferred embodiment of the palmprint ROI extraction method for palms at any angle of the present invention;

[0049] Figure 7 It is a schematic diagram of ROI image interception in a preferred embodiment of the palmprint ROI extraction method for palms at any angle of the present invention;

[0050] Figure 8 It is a schematic diagram of the principle of a preferred embodiment of the palmprint ROI extraction system for palms at any angle of the present invention;

[0051] Figure 9 It is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Detailed implementation manners

[0052] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further elaborates on the present invention by way of examples with reference to the accompanying drawings. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] Palmprints contain rich epidermal texture and subcutaneous vein features, and have high discriminability and anti-counterfeiting capabilities. The main processes of a palmprint biometric recognition system are as follows: 1. Image acquisition: Use an image sensor to acquire an image containing a palm; 2. Palm ROI (region of interest) preprocessing: Based on the palm contour information, locate the key points of the finger valleys of the palm, and then calculate and crop the ROI image of the central region of the palm; 3. Feature extraction: Based on the ROI image, extract the palmprint feature vector; 4. Feature matching: Calculate the similarity between the palmprint feature vectors for identity verification or recognition.

[0054] In a palmprint recognition system, extracting the Region of Interest (ROI) of the palmprint can effectively avoid background interference. At the same time, the position of the ROI is located based on the finger valleys of the palm, which helps to reduce the intra-class differences of palm images placed in different directions, distances, etc. Therefore, the positioning accuracy of the ROI directly affects the subsequent feature extraction and matching accuracy. Then, in a non-contact palmprint recognition system, palm images are collected in an open space, and there are various influencing factors, such as the posture, position of the palm, and environmental illumination, etc., which makes the precise positioning of the palm ROI very challenging.

[0055] In recent years, palmprint recognition has been widely studied, and significant progress has also been made in the ROI extraction algorithms. Currently, most of the proposed ROI extraction algorithms are based on the key points of the finger valleys of the palm. By accurately positioning the key points, accurate ROI extraction can be achieved.

[0056] For example, the existing methods for ROI extraction are as follows:

[0057] (1) Straight line cluster scanning method: This method first performs Gaussian filtering on the palm image, and then binarizes it using a threshold segmentation method with a fixed threshold. Then, column scanning is performed on the image from the finger side. When the number of non-zero pixel points in a certain column reaches a certain value, these intersection points are recorded in sequence according to the up-down order or left-right order. According to the relationship between the intersection points, the boundary of the finger valley curve is determined, and the region growing algorithm is used to find the contour of the finger valley curve, and its coordinate extreme points are the final key points. This method requires the palm to be at the same scale and position and not to rotate, and the background needs to be clean. Therefore, it is not applicable to non-contact palmprint recognition systems.

[0058] (2) Convex hull detection method: After filtering and binarizing the palm image, the convex hull detection algorithm is used to calculate the convex polygon that completely encloses the palm and the corresponding concave points. Then, according to the relative distribution of the concave points and the relationship between the concave points and the corner points of the convex polygon, the final finger valley key points are determined. This method is easily affected by the local convex and concave points of the palm contour, resulting in too many candidate valley points and it is difficult to screen accurate finger valley key points.

[0059] (3) Distance distribution method: This method first performs filtering and binarization processing on the palm image, and then obtains the coordinates of all points on the contour line of the palm. Then, a reference point is set on the contour line near the wrist position in the image, and the distances from all points on the contour line to the reference point are calculated along a direction. The maximum value point of the distance on the contour line is the fingertip, and the minimum value point is the finger valley point. According to the interval size and relative relationship between each extreme value point, the position of the final finger valley key point is determined. This method needs to know the direction of the palm in advance when selecting the reference point near the wrist position, and is easily affected by the noise (such as sleeves) at the wrist position in the image, resulting in abnormal distance distribution and thus incorrect positioning.

[0060] (4) Deep learning method: Adopt a deep convolutional network model, combine the positioning and segmentation of the palm, and use a large number of images marked with the positions of the finger valleys to train the model. After training, the network can accurately locate the finger valleys in the palm image. This method requires high-performance hardware support, specifically including large storage and high computing power. On general devices, the running speed is very slow and cannot meet the real-time requirements.

[0061] Therefore, aiming at the problems of inaccurate palm area segmentation and many palm pose restrictions in the ROI extraction process, the present invention proposes a fast ROI extraction method for palms at any angle combined with edge detection.

[0062] The palmprint ROI extraction method for palms at any angle according to the preferred embodiment of the present invention is as Figure 1 shown. The palmprint ROI extraction method for palms at any angle includes the following steps:

[0063] Step S10: Obtain a palm image, and perform palm segmentation on the palm image through preprocessing to obtain a palm contour image.

[0064] Specifically, as Figure 2 shown, from left to right are the original palm image, the binary image after threshold segmentation, the edge detection image, and the binary image combined with edge detection and segmentation. First, obtain the palm image collected by the image sensor, and then preprocess the image for palm segmentation to obtain the contour of the palm for subsequent direction and key point positioning. The process of palm image segmentation is as follows:

[0065] (1) Gaussian filtering: Based on a Gaussian filter (using a Gaussian filter with a kernel size of 3*3 and a standard deviation of 3 for the Gaussian function), smooth the palm image to eliminate noise generated due to environmental reasons such as illumination, and obtain a smoothed palm image.

[0066] (2) Edge detection: For the smoothed palm image, detect the edges of the palm in the smoothed palm image based on the XDOG algorithm, and adjust the parameters of the XDOG algorithm to sigma = 0.65, k = 3.2, gamma = 0.96, epsilon = -1, phi = 200, so that it can also have a high response value for palm edges with blurred boundaries and unclear finger gaps; thereby obtaining a robust palm edge detection image, where the pixel values of the edge region in the edge detection image are 0 and the pixel values of the background region are 1.

[0067] (3) Threshold segmentation: The smoothed palm image is binarized based on the OSTU adaptive threshold segmentation algorithm. Among them, the pixels in the palmprint area of the binarized palm image are 1, and the pixels in the background area are 0. Subsequently, morphological erosion and dilation operations are performed on the binarized palm image to eliminate white noise in the image. Finally, based on the Seed-Filling method, the connected regions of non-zero pixels in the binarized palm image are calculated, and the region with the largest area among the non-zero pixel connected regions is retained, and the remaining regions are all filled with 0 to obtain the binary segmentation map of the palm.

[0068] (4) Combining edge and segmentation: The palm edge detection image and the binary segmentation map are multiplied matrix-wise. The resulting binary image removes a large amount of background noise in the edge matrix on the one hand, and retains the fine edges of the palm on the other hand. Subsequently, based on the Seed-Filling method again, the largest connected region in the binary segmentation map after matrix multiplication is calculated, and the regions other than the largest area are all filled with 0 to obtain the final binary segmentation map of the palm.

[0069] (5) Finding the palm contour: Based on the contour detection algorithm, all continuous contour points in the palm region of the final binary segmentation map of the palm are obtained to get the palm contour image. Step S20: Determine the palm center positioning point and the finger detection circle family according to the palm contour line in the palm contour image, determine the finger positioning point according to the finger detection circle family, and determine the palm direction according to the palm center positioning point and the finger positioning point, and correct the palm to the same direction.

[0070] Specifically, for a palm placed in any direction on the horizontal plane, it is first necessary to locate the palm direction according to the relationship between the fingers and the palm center, and then unify the palm to the same orientation. The specific steps are as follows:

[0071] (1) Selecting the palm center positioning point: According to the palm contour line in the palm contour image, the largest inscribed circle algorithm of an arbitrary polygon is used to calculate the largest inscribed circle in the palm, and the center of the circle is set as the palm center positioning point; this largest inscribed circle determines the position of the palm center on the one hand, and provides a reference center for the finger detection circle family used to confirm the finger positions on the other hand.

[0072] (2) Determining the finger detection circle family: As Figure 3 shown, first take the palm reference point as the center of all circles in the finger detection circle family ( Figure 3 the midpoint o is the center of all points in the circle family, and the distance d is the difference between the largest radius and the smallest radius in the circle family), and then perform convex hull detection on the palm based on the palm contour line, detect the convex points and the points farthest from the convex points, take the distance between the convex points and the points farthest from the convex points as the length of the finger, and determine the smallest radius d of the finger detection circle family according to the convex points and the points farthest from the convex points. 1and the maximum radius d 2 , and then select the minimum radius d 1 and the maximum radius d 2 The radius values ​​of the circles within the range are determined based on the dichotomy method, and these circles constitute the finger detection circle family.

[0073] (3) Determine the finger positioning point: Figure 4 As shown, the radius of the finger detection circle family is selected as The detection circle, the binary image corresponding to the detection circle (the edge of the circle in the image

[0074] The edge pixels are 1, and the rest are 0) and the binary segmentation map of the palm are matrix multiplied to obtain the intersection area of ​​the detection circle and the palm area. Then, based on the connected domain analysis method, the center, length and position relationship of the intersection area are obtained, and the fan-shaped area composed of all three adjacent areas and the center of the circle is recorded. If the number of all fan-shaped areas is greater than 2 and the arc of the group with the smallest arc is less than the first preset angle (for example, 45°), the center position of the group with the smallest arc is the final hand-pointing point. Otherwise, the radius of the finger detection circle family is selected. The above operation is performed until the finger positioning point is detected. For the nth detection, the radius of the detection circle is

[0075] (4) Determine the finger open and closed state: Figure 5 As shown, if the number of all the sector areas is less than two and the corresponding arc is greater than a second preset angle (eg, 120°), it is considered that the fingers are not open, and a corresponding prompt is given to recapture the palm.

[0076] (5) Correct the palm direction: Figure 6 As shown, the palm positioning point and the finger positioning point are connected to determine the palm direction, and then the palm is corrected to a unified direction based on geometric transformation.

[0077] Step S30: determine the area where the finger is located, and determine the final finger valley key point according to the curvature distribution of the contour line in the area.

[0078] Specifically, after correcting the palm to a uniform orientation, the approximate area where the fingers are located can be obtained, and then the final finger valley key points are determined based on the curvature distribution of the contour lines in the area. The specific steps are as follows:

[0079] (1) Calculate the curvature distribution of the finger area: When the palm is facing the same direction, the area where the finger is located is confirmed based on the position of the palm reference point. The contour line in the area is obtained based on the contour detection algorithm. Then, the curvature value of the adjacent area on the contour line is calculated based on the RTPDA method to obtain the curvature distribution curve of the finger area contour.

[0080] (2) Select the key points of the finger valley: Set the local comparison length as the first preset value (e.g., 50). If the difference in curvature values between the minimum and maximum points within the local comparison length is greater than the second preset value (e.g., 25) and the curvature value of the maximum point is greater than the third preset value (e.g., 30), retain the maximum point as a candidate extreme point, and then determine the final key points of the finger valley based on the relative positions and distances between the candidate extreme points.

[0081] Step S40: According to the key points of the finger valley, intercept a square area at the center position of the palm as the ROI image.

[0082] Specifically, as Figure 7 shown, establish a coordinate system based on the key points of the finger valley. Among them, connect the two key points of the finger valley to establish the y-axis of the coordinate axis, and establish the x-axis through the perpendicular bisector of the line connecting the two key points of the finger valley. The direction pointing to the palm is the positive direction of the x-axis; calculate the Euclidean distance between the two key points of the finger valley and denote it as d. Then the center point of the square is The side length is and two sides of the square are parallel to the y-axis. Locate the four points of the square, and finally intercept the square area as the ROI image.

[0083] A fast ROI extraction method for palms at any angle combined with edge detection proposed by the present invention mainly includes four aspects: (1) precise segmentation of palm images based on edge enhancement; (2) fast positioning and correction of palm directions; (3) key point positioning; (4) ROI interception. This method can efficiently and accurately extract the ROI image of the palm area, and can be realized even when the palm boundary is blurred, the finger gaps are unclear, the palm rotates at any angle in the horizontal plane, and the fingers are incomplete in a non-contact palmprint recognition system. In addition, this method has the characteristic of high operating efficiency and can extract the ROI image of the palm in real time on a hardware platform with low computing power and small storage.

[0084] The method of the present invention has real-time performance and robustness, and is applicable to palm images collected in a non-contact palmprint recognition system. Regardless of the palm being at any angle in the horizontal direction, the palm part in the picture is missing, the palm edge is blurred, the finger gaps are unclear, etc., it can quickly and accurately locate the key points and extract the ROI. Among the 8680 non-contact palm images with arbitrary angle rotation processing added, the success rate of ROI positioning and extraction exceeds 97.6%. The algorithm of the present invention is written in C++. For example, the calculation time for a single picture on a personal laptop (CPU i7-12700) is only 3.6 milliseconds. In addition, the present invention has also been tested in a non-contact palmprint recognition system that uses a USB camera to collect images, and realizes the real-time positioning and extraction of ROI key points. The test results show that the method of the present invention has good time performance and accuracy in practical applications.

[0085] The precise palm image segmentation method based on edge enhancement in the present invention: The image is processed through Gaussian filtering, morphological transformation, the largest connected component, and palm contour detection, and combined with palm threshold segmentation and edge detection, so as to obtain a precise palm edge image, which can be achieved even for images with blurred edges.

[0086] The palm orientation positioning and correction method based on the detection circle family in the present invention: The palm center positioning point is determined according to the largest inscribed circle of the palm contour, and then a finger detection circle family is established based on the convex hull detection of the palm. The final finger positioning points are determined according to the distribution of the intersections of the detection circles and the fingers, so as to obtain the final palm direction and correct it to a unified orientation, as well as judge the palm opening and closing state.

[0087] The key point detection method based on curvature distribution in the present invention: After determining the finger region, the curvature distribution of the finger contour within the region is calculated, and the final finger valley key points are determined according to the distances and relative relationships of the extreme points on the distribution.

[0088] Furthermore, as Figure 8 shown, based on the palmprint ROI extraction method for palms at any angle described above, the present invention also correspondingly provides a palmprint ROI extraction system for palms at any angle. Among them, the palmprint ROI extraction system for palms at any angle includes:

[0089] A palm image segmentation module 51, configured to obtain a palm image, perform palm segmentation on the palm image through preprocessing, and obtain a palm contour image;

[0090] A direction positioning and correction module 52, configured to determine a palm center positioning point and a finger detection circle family according to the palm contour line in the palm contour image, determine finger positioning points according to the finger detection circle family, determine the palm direction according to the palm center positioning point and the finger positioning points, and correct the palm to a unified direction;

[0091] A key point positioning module 53, configured to determine the region where the fingers are located, and determine the final finger valley key points according to the curvature distribution of the contour line within the region;

[0092] An ROI image cropping module 54, configured to crop a square region at the center of the palm as an ROI image according to the finger valley key points.

[0093] Furthermore, as Figure 9 shown, based on the palmprint ROI extraction method and system for palms at any angle described above, the present invention also correspondingly provides a terminal, and the terminal includes a processor 10, a memory 20, and a display 30. Figure 9Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively.

[0094] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as the hard disk or memory of the terminal. In some other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 may also include both the internal storage unit of the terminal and the external storage device. The memory 20 is used to store the application software installed on the terminal and various types of data, such as the program code for installing the terminal. The memory 20 may also be used to temporarily store the data that has been output or will be output. In one embodiment, a palmprint ROI extraction program 40 for a palm at any angle is stored on the memory 20, and the palmprint ROI extraction program 40 for a palm at any angle can be executed by the processor 10, so as to implement the palmprint ROI extraction method for a palm at any angle in the present application.

[0095] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chips, and is used to run the program code stored in the memory 20 or process data, such as executing the palmprint ROI extraction method for a palm at any angle, etc.

[0096] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) toucher, etc. The display 30 is used to display the information in the terminal and to display a visual user interface. The components 10 - 30 of the terminal communicate with each other through a system bus.

[0097] In one embodiment, when the processor 10 executes the palmprint ROI extraction program 40 for a palm at any angle in the memory 20, the following steps are implemented:

[0098] Obtain a palm image, perform palm segmentation on the palm image through preprocessing to obtain a palm contour image;

[0099] Determine a palm center positioning point and a family of finger detection circles according to the palm contour line in the palm contour image, determine finger positioning points according to the family of finger detection circles, determine the palm direction according to the palm center positioning point and the finger positioning points, and correct the palm to a unified direction;

[0100] Determine the area where the finger is located, and determine the final finger valley key points according to the curvature distribution of the contour line within the area;

[0101] According to the finger valley key points, intercept a square area at the center position of the palm as the ROI image.

[0102] Among them, for the acquisition of the palm image, the palm image is segmented through preprocessing to obtain a palm contour image, which specifically includes:

[0103] Obtain the palm image collected by the image sensor, and perform smoothing processing on the palm image based on a Gaussian filter to obtain a smoothed palm image;

[0104] Detect the edge of the palm in the smoothed palm image based on the XDOG algorithm to obtain a robust palm edge detection image;

[0105] Perform binarization on the smoothed palm image based on the OSTU adaptive threshold segmentation algorithm, perform morphological erosion and dilation operations on the binarized palm image, and calculate the non-zero pixel connected domain of the binarized palm image based on the Seed-Filling method. Retain the region with the largest area in the non-zero pixel connected domain, and fill the remaining regions with 0 to obtain a binary segmentation map of the palm;

[0106] Multiply the palm edge detection image and the binary segmentation map in matrix form, and again calculate the largest connected domain in the binary segmentation map after matrix multiplication based on the Seed-Filling method. Fill all regions except the largest area with 0 to obtain the final binary segmentation map of the palm;

[0107] Based on the contour detection algorithm, obtain all continuous contour points of the palm region in the final binary segmentation map of the palm to obtain a palm contour image.

[0108] Among them, for determining the palm center positioning point and the finger detection circle family according to the palm contour line in the palm contour image, determining the finger positioning point according to the finger detection circle family, and determining the palm direction according to the palm center positioning point and the finger positioning point, and correcting the palm to a unified direction, it specifically includes:

[0109] According to the palm contour line in the palm contour image, use the maximum inscribed circle algorithm of an arbitrary polygon to calculate the maximum inscribed circle within the palm, and set the center of the circle as the palm center positioning point;

[0110] Taking the palm reference point as the center of all circles in the finger detection circle family, performing convex hull detection on the palm based on the palm contour line, detecting the convex points and the points farthest from the convex points, taking the distance between the convex points and the points farthest from the convex points as the length of the finger, and determining the minimum radius d of the finger detection circle family according to the convex points and the points farthest from the convex points 1 and the maximum radius d 2 , selecting the circles within the range of the minimum radius d 1 and the maximum radius d 2 to form the finger detection circle family;

[0111] Selecting the detection circle with a radius of in the finger detection circle family, performing matrix multiplication on the binary image corresponding to the detection circle and the binary segmentation map of the palm to obtain the intersection region of the detection circle and the palm region, obtaining the center, length, and positional relationship of the intersection region based on the connected component analysis method, and recording all the fan-shaped regions composed of adjacent three regions and the center of the circle. If the number of all fan-shaped regions is greater than 2 and the radian of the group with the smallest radian is less than the first preset angle, the center position of the group with the smallest radian is the final finger positioning point; otherwise, selecting the detection circle with a radius of in the finger detection circle family for processing until the finger positioning point is detected. Among them, for the nth detection, the selected detection circle radius is

[0112] If the number of all fan-shaped regions is less than two and the corresponding radian is greater than the second preset angle, it is considered that the finger is not open, and a corresponding prompt is given to re-collect the palm;

[0113] Connecting the palm center positioning point and the finger positioning point to determine the palm direction, and correcting the palm to a unified direction based on geometric transformation.

[0114] Among them, to determine the region where the finger is located, the final finger valley key points are determined according to the curvature distribution of the inner contour line of the region, specifically including:

[0115] After the palm is oriented uniformly, combining the position of the palm center reference point to confirm the region where the finger is located, obtaining the contour line within the region based on the contour detection algorithm, calculating the curvature values of adjacent regions on the contour line based on the RTPDA method, and obtaining the curvature distribution curve of the finger region contour;

[0116] Selecting the local comparison length as the first preset value. If the difference between the curvature values of the minimum value point and the maximum value point within the local comparison length is greater than the second preset value and the curvature value of the maximum value point is greater than the third preset value, retaining the maximum value point as the candidate extreme point, and determining the final finger valley key points according to the relative position and distance between the candidate extreme points.

[0117] Among them, a square region at the center of the palm is intercepted as the ROI image according to the finger valley key points, which specifically includes:

[0118] A coordinate system is established according to the finger valley key points. Among them, the y-axis of the coordinate axis is established by connecting two finger valley key points, the x-axis is established by the perpendicular bisector of the line connecting the two finger valley key points, and the direction pointing to the palm is the positive direction of the x-axis;

[0119] The Euclidean distance between two finger valley key points is calculated and denoted as d. Then the center point of the square is The side length is And two sides of the square are parallel to the y-axis. The four points of the square are located, and finally the square region is intercepted as the ROI image.

[0120] Among them, the kernel size of the Gaussian filter is 3*3, and the standard deviation of the Gaussian function is 3;

[0121] In the palm edge detection image, the pixels in the edge region are 0, and the pixels in the background region are 1;

[0122] In the binary palm image, the pixels in the palm print region are 1, and the pixels in the background region are 0.

[0123] Among them, the first preset angle is 45°, and the second preset angle is 120°;

[0124] The first preset value is 50, the second preset value is 25, and the third preset value is 30.

[0125] The present invention also provides a computer-readable storage medium. Among them, the computer-readable storage medium stores a palm print ROI extraction program for palms at any angle. When the palm print ROI extraction program for palms at any angle is executed by a processor, the steps of the palm print ROI extraction method for palms at any angle as described above are implemented.

[0126] In summary, the present invention provides a method for extracting the ROI of palmprint of a palm at any angle and related devices. The method includes: obtaining a palm image, performing palm segmentation on the palm image through preprocessing to obtain a palm contour image; determining a palm center positioning point and a family of finger detection circles according to the palm contour line in the palm contour image, determining finger positioning points according to the family of finger detection circles, determining the palm direction according to the palm center positioning point and the finger positioning points, and correcting the palm to a unified direction; determining the area where the fingers are located, and determining the final finger valley key points according to the curvature distribution of the contour line in the area; and intercepting a square area at the center position of the palm as the ROI image according to the finger valley key points. The present invention can efficiently and accurately extract the ROI image of the palm area, and can be realized even when the palm boundary is blurred, the finger gaps are unclear, the palm rotates at any angle in the horizontal plane, and the fingers are incomplete in a non-contact palmprint recognition system. It has the characteristics of high operating efficiency and can extract the ROI image of the palm in real time on a hardware platform with low computing power and small storage.

[0127] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or terminal including the element.

[0128] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium readable by a computer, and when the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0129] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for extracting the ROI of palmprint of a palm at any angle, characterized in that, the method for extracting the ROI of palmprint of a palm at any angle includes: Obtain a palm image, and perform palm segmentation on the palm image through preprocessing to obtain a palm contour image; Determine the palm center positioning point and the finger detection circle family according to the palm contour line in the palm contour image, determine the finger positioning point according to the finger detection circle family, determine the palm direction according to the palm center positioning point and the finger positioning point, and correct the palm to a unified direction; The step of determining the palm center positioning point and the finger detection circle family according to the palm contour line in the palm contour image, determining the finger positioning point according to the finger detection circle family, determining the palm direction according to the palm center positioning point and the finger positioning point, and correcting the palm to a unified direction specifically includes: According to the palm contour line in the palm contour image, use the maximum inscribed circle algorithm of an arbitrary polygon to calculate the maximum inscribed circle in the palm, and set the center of the circle as the palm center positioning point; Taking the palm reference point as the center of all circles in the finger detection circle family, performing convex hull detection of the palm based on the palm contour line, detecting the convex points and the points farthest from the convex points, taking the distance between the convex points and the points farthest from the convex points as the length of the finger, and determining the minimum radius d of the finger detection circle family according to the convex points and the points farthest from the convex points 1 and the maximum radius d 2 , selecting the circles within the range of the minimum radius d 1 and the maximum radius d 2 to form the finger detection circle family; Select a detection circle with a radius in the finger detection circle family , perform matrix multiplication on the binary image corresponding to the detection circle and the binary segmentation map of the palm to obtain the intersection region of the detection circle and the palm region. Based on the connected component analysis method, obtain the center, length, and positional relationship of the intersection region, and record all the sector regions formed by three adjacent regions and the center of the circle. If the number of all sector regions is greater than 2 and the radian of the group with the smallest radian is less than the first preset angle, the center position of the group with the smallest radian is the final finger positioning point; otherwise, select a detection circle with a radius in the finger detection circle family for processing until the finger positioning point is detected. Among them, for the nth detection, the radius of the selected detection circle is If the number of all fan-shaped regions is less than two and the corresponding radian is greater than the second preset angle, it is considered that the fingers are not open, and a corresponding prompt is given to re-acquire the palm; Connect the palm center positioning point and the finger positioning point to determine the palm direction, and correct the palm to a unified direction based on geometric transformation; Determine the region where the fingers are located, and determine the final finger valley key points according to the curvature distribution of the contour line in the region; The step of determining the region where the fingers are located and determining the final finger valley key points according to the curvature distribution of the contour line in the region specifically includes: After the palm orientation is unified, confirm the region where the fingers are located in combination with the position of the palm center reference point, obtain the contour line in the region based on the contour detection algorithm, calculate the curvature values of adjacent regions on the contour line based on the RTPDA method, and obtain the curvature distribution curve of the finger region contour; Select the local comparison length as the first preset value. If the difference between the curvature values of the minimum value point and the maximum value point within the local comparison length is greater than the second preset value and the curvature value of the maximum value point is greater than the third preset value, retain the maximum value point as the candidate extreme point, and determine the final finger valley key points according to the relative position and distance between the candidate extreme points; According to the finger valley key points, intercept a square region at the center position of the palm as the ROI image.

2. The method for extracting the ROI of palmprint of a palm at any angle according to claim 1, characterized in that, the step of obtaining a palm image and performing palm segmentation on the palm image through preprocessing to obtain a palm contour image specifically includes: Obtain the palm image collected by the image sensor, perform smoothing processing on the palm image based on a Gaussian filter to obtain a smoothed palm image; Detect the edge of the palm in the smoothed palm image based on the XDOG algorithm to obtain a robust palm edge detection image; Binarize the smoothed palm image based on the OSTU adaptive threshold segmentation algorithm, perform morphological erosion and dilation operations on the binarized palm image, and calculate the connected domain of non-zero pixels of the binarized palm image based on the Seed-Filling method. Retain the region with the largest area in the non-zero pixel connected domain, and fill all other regions with 0 to obtain a binary segmentation map of the palm; Multiply the palm edge detection image and the binary segmentation map in matrix form, and again calculate the largest connected domain in the binary segmentation map after matrix multiplication based on the Seed-Filling method. Fill all regions except the largest area with 0 to obtain the final binary segmentation map of the palm; Obtain all continuous contour points of the palm region in the final binary segmentation map of the palm based on the contour detection algorithm to obtain a palm contour image.

3. The method for extracting the palmprint ROI of an arbitrarily angled palm according to claim 1, characterized in that, intercept a square region at the center position of the palm as the ROI image according to the key points of the finger valleys, specifically including: Establish a coordinate system according to the key points of the finger valleys. Among them, connect the two key points of the finger valleys to establish the y-axis of the coordinate axis, and establish the x-axis through the perpendicular bisector of the line connecting the two key points of the finger valleys. The direction pointing to the palm is the positive direction of the x-axis; Calculate the Euclidean distance between the key points of the fingertips of two fingers and denote it as d. Then the center point of the square is The side length is And two sides of the square are parallel to the y-axis. Locate the four points of the square, and finally intercept the square area as the ROI image.

4. The method for extracting the palmprint ROI of an arbitrarily angled palm according to claim 2, characterized in that, the kernel size of the Gaussian filter is 3*3, and the standard deviation of the Gaussian function is 3; the pixels in the edge region of the palm edge detection image are 0, and the pixels in the background region are 1; the pixels in the palmprint region of the binarized palm image are 1, and the pixels in the background region are 0.

5. The method for extracting the palmprint ROI of an arbitrarily angled palm according to claim 1, characterized in that, the first preset angle is 45°, and the second preset angle is 120°; the first preset value is 50, the second preset value is 25, and the third preset value is 30.

6. A system for extracting the palmprint ROI of an arbitrarily angled palm, characterized in that, the system for extracting the palmprint ROI of an arbitrarily angled palm includes: a palm image segmentation module for obtaining a palm image and performing palm segmentation on the palm image through preprocessing to obtain a palm contour image; a direction positioning and correction module for determining a palm center positioning point and a finger detection circle family according to the palm contour line in the palm contour image, determining finger positioning points according to the finger detection circle family, determining the palm direction according to the palm center positioning point and the finger positioning points, and correcting the palm to a unified direction; the determining a palm center positioning point and a finger detection circle family according to the palm contour line in the palm contour image, determining finger positioning points according to the finger detection circle family, determining the palm direction according to the palm center positioning point and the finger positioning points, and correcting the palm to a unified direction specifically includes: According to the palm contour line in the palm contour image, use the maximum inscribed circle algorithm of an arbitrary polygon to calculate the maximum inscribed circle in the palm, and set the center of the circle as the palm center positioning point; Taking the palm reference point as the center of all circles in the finger detection circle family, performing convex hull detection on the palm based on the palm contour line, detecting the convex points and the points farthest from the convex points, taking the distance between the convex points and the points farthest from the convex points as the length of the finger, and determining the minimum radius d of the finger detection circle family according to the convex points and the points farthest from the convex points 1 and the maximum radius d 2 , selecting the circles within the range of the minimum radius d 1 and the maximum radius d 2 to form the finger detection circle family; Select the detection circle with a radius of in the finger detection circle family. Perform matrix multiplication on the binary image corresponding to the detection circle and the binary segmentation map of the palm to obtain the intersection region between the detection circle and the palm region. Based on the connected component analysis method, obtain the center, length, and positional relationship of the intersection region, and record all the fan-shaped regions composed of adjacent three regions and the center of the circle. If the number of all fan-shaped regions is greater than 2 and the radian of the group with the smallest radian is less than the first preset angle, the center position of the group with the smallest radian is the final finger positioning point. Otherwise, select the detection circle with a radius of in the finger detection circle family for processing until the finger positioning point is detected. Among them, for the nth detection, the radius of the selected detection circle is If the number of all the fan-shaped areas is less than two and the corresponding arc is greater than the second preset angle, it is considered that the fingers are not open, and a corresponding prompt is given to re-capture the palm; Connect the palm positioning points and finger positioning points to determine the palm direction, and correct the palm to a unified direction based on geometric transformation; The key point positioning module is used to determine the area where the finger is located and determine the final finger valley key point according to the curvature distribution of the contour line in the area; The step of determining the area where the finger is located and determining the final finger valley key point according to the curvature distribution of the contour line in the area specifically includes: When the palms are facing the same direction, the area where the fingers are located is confirmed based on the position of the palm reference point, the contour line in the area is obtained based on the contour detection algorithm, and the curvature values ​​of the adjacent areas on the contour line are calculated based on the RTPDA method to obtain the curvature distribution curve of the finger area contour; The local comparison length is selected as the first preset value. If the difference between the curvature values ​​of the minimum point and the maximum point within the local comparison length is greater than the second preset value and the curvature value of the maximum point is greater than the third preset value, the maximum point is retained as a candidate extreme point, and the final finger valley key point is determined according to the relative position and distance between the candidate extreme points; The ROI image capture module is used to capture a square area at the center of the palm as the ROI image according to the finger valley key points.

7. A terminal, It is characterized in that The terminal includes: a memory, a processor, and a palmprint ROI extraction program for a palm at any angle stored in the memory and executable on the processor. When the palmprint ROI extraction program for a palm at any angle is executed by the processor, the steps of the palmprint ROI extraction method for a palm at any angle as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a palmprint ROI extraction program for a palm at any angle, and when the palmprint ROI extraction program for a palm at any angle is executed by a processor, the steps of the palmprint ROI extraction method for a palm at any angle as claimed in any one of claims 1 to 6 are implemented.

Images