Method for extracting region of interest of closed palm under complex background

Abstract

The present invention discloses a method for extracting a region of interest of a closed palm under a complex background, which belongs to the intersection of bioinformatics and image processing. The method comprises the following steps: measuring the distance of the palm using a binocular camera, extracting specific points for perspective transformation, and correcting the palm image; establishing a coarse segmentation model based on the palm image using a DeepLabv3+ semantic neural network, and combining it with a Gaussian skin color model to improve segmentation accuracy and generate a binary image; performing maximum connected domain analysis on the binary image; after obtaining a complete binary palm image, performing convex hull detection on the closed palm; performing edge detection on the palm image using a Canny operator, obtaining horizontal edges using a horizontal operator, and then obtaining each edge endpoint as a candidate point. The candidate point is screened based on conditions such as the distance from the candidate point to the centroid and the distance from the candidate point to L1 and L2 to obtain the final two key points to extract the palm region of interest.

Description

Technical Field

[0001] The present invention belongs to the intersection of bioinformatics and image processing, and specifically is a method for extracting a region of interest of a closed palm under a complex background. Background Art

[0002] With the continuous development of biometric recognition technology and the need for health and epidemic prevention, biometric recognition technology has begun to gradually transition from contact recognition to contactless recognition. During the transition process, many problems such as palm rotation, tilt, and distortion that lead to recognition errors naturally arise. Compared with contact palmprint recognition, contactless palmprint recognition not only needs to overcome the palm posture problem, but also needs to overcome the interference of complex backgrounds and the influence of lighting.

[0003] Palm rotation can cause inconsistent orientation of the extracted ROI (Region of Interest) feature information, while tilting the hand directly transforms the contours, leading to deviations in the ROI extraction position. Currently, most palm ROI extraction methods are based on a specific hand posture (with open fingers), but it is difficult to extract the palm ROI when the palm is closed. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for extracting a closed palm region of interest under a complex background, so as to solve the problem in the prior art that non-contact palmprint recognition is difficult to extract a closed palm region of interest under a complex background.

[0005] A method for extracting a closed palm region of interest under complex background is provided, including the following steps:

[0006] S1: Obtain biometric information of the palm's circumscribed rectangle. Leveraging the specificity of the rectangle, perform perspective transformation on specific points to correct the tilted palm.

[0007] S2: Use DeepLabv3+ semantic neural network to build a segmentation model based on palm contours to separate the corrected palm from the complex background. Then, combine it with the Gaussian skin color model to improve segmentation accuracy and generate a binary image.

[0008] S3: Perform maximum connected component analysis on the binary image to further eliminate errors in the palm image segmentation process, and then perform morphological operations to eliminate holes in the palm area;

[0009] S4: After obtaining the complete palm binary image, perform convex hull detection on the closed palm to obtain the concave points between the fingertips. Through morphology and geometry, two lines L1 and L2 passing through the gaps between the index finger and middle finger, and the ring finger and little finger are obtained;

[0010] S5: Use the Canny operator to perform edge detection on the palm image, and use the horizontal operator to obtain the horizontal edge. Then obtain each edge endpoint as a candidate point. Screen the candidate point based on the distance from the candidate point to the center of mass and the distance from the candidate point to L1 and L2, and obtain the final two key points to extract the area of ​​interest of the palm.

[0011] As a further solution of the present invention: In S1, the three-dimensional coordinates of the palms P1, P2, P3, P4, and P5 are obtained by using a binocular camera and skeleton point detection technology. These five discrete points are used to fit a plane. Using P3 as a reference point, a symmetrical point P6 is made with respect to P1 and P3, and the three-dimensional coordinates of P6 are obtained. The point P7 is obtained by the following three conditions:

[0012] A) The distance from P7 to P3 should be equal to the distance from P6 to P3;

[0013] B) The vectors of P7 and P3 should be perpendicular to the vectors of P6 and P3;

[0014] C) P7 should be on the fitting plane;

[0015] After obtaining P7, make the symmetrical point P8 of P7;

[0016] Repeat the above steps, assume and obtain point A, and use symmetry to obtain points B, C, and D respectively to obtain the coordinates of the four key points of the rectangle on the palm plane, and obtain a rectangular area on the palm plane that includes the entire hand; the obtained four points A, B, C, and D are passed as parameters into the perspective transformation function, so that the four points form a standard rectangle in the image coordinate system, completing the correction of the tilted palm.

[0017] The specific equation for P6 is:

[0018] .

[0019] As a further solution of the present invention: using the five discrete points P1, P2, P3, P4 and P5 to fit the plane, it is known that a three-dimensional plane can be Therefore, fitting a plane by discrete points is essentially solving an overdetermined equation to obtain the fitted plane coefficient Ω(a, b, c). The formula of the overdetermined equation is as follows:

[0020] ;

[0021] Among them, x, y, and z are the position coordinates of the pixel in the spatial domain. a 、 b 、 c are the coefficients of the plane equation,

[0022] Assume that the three-dimensional coordinates of P7 are ( X7, Y7 , Z7 ), the three conditional equations for point P7 are:

[0023] ;

[0024] in, X3 、 Y3 、 Z3 、 X6 、 Y6 、 Z6 、 X7 、 Y7 、 Z7 is the three-dimensional coordinate information of the relevant coordinate points, a 、 b 、 c is the plane equation coefficient, and the perspective transformation equation is:

[0025] ;

[0026] in u 、 v 、 w is the original image coordinate, and the corresponding transformed image homogeneous coordinate is x 、 y 、 w , a 11 ~ a 33 is the transformation matrix;

[0027] After obtaining the transformed homogeneous coordinates, the transformed image coordinates are obtained, and the formula is:

[0028] ;

[0029] ;

[0030] in x , y is the transformed image coordinate.

[0031] As a further solution of the present invention: in S2, DeepLabv3+ semantic segmentation is first used to perform the first segmentation of the palm; in order to make the segmentation model more stable and effective, training samples are added for different hand postures, different lighting environments and different backgrounds when training the model; after obtaining the binary image segmented by DeepLabv3+ semantic segmentation, the binary area with the largest connected domain is selected as the palm area, the binary area is moderately expanded, and an "AND" operation is performed with the original palm image, so that most of the background areas except the palm are shielded; the image after the "AND" operation is segmented using a traditional method based on a Gaussian skin color model; after shielding most of the background, when the complex background interference is small, the traditional image segmentation method has better accuracy and stability than the neural network semantic segmentation.

[0032] As a further solution of the present invention: in S3, connected domain analysis is performed on the binary image obtained by segmentation, a median filter is used to smooth the binary contour, and a dilation and corrosion operation in morphology is used to remove holes inside the palm.

[0033] As a further solution of the present invention: in S4, by adjusting parameters to adapt to the palm shape, a convex hull detection technique is used to obtain information of a group of concave points on the closed palm, and an OpenCV function is used to obtain centroid coordinate information of the binary image and palm contour point information;

[0034] Through the information of the concave points and the depth of the concave points, it can be concluded that the point with the largest depth is the concave point of the thumb and index finger, recorded as Q1;

[0035] Compare the distances between the remaining concave points and the centroid one by one, and the point with the farthest distance is the tip of the middle finger T1;

[0036] Connect T1 and the centroid C1 and extend them to intersect the palm outline. The intersection point is recorded as D1. Point D1 is located at the wrist of the palm;

[0037] Traverse the concave point information and calculate the distance from each concave point to point D1, so as to obtain the three concave points with the longest distances: between the index finger and middle finger, between the ring finger and little finger, and the tip of the index finger. Then, based on the distances between these three points, filter out the concave points between the index finger and middle finger, and between the ring finger and little finger.

[0038] By comparing the distance with point Q1, the concave point between the index finger and the middle finger is determined to be Q2, and the concave point between the ring finger and the little finger is determined to be Q3;

[0039] Draw a perpendicular line between Q1 and Q2, and rotate Q1 around Q2 by 14°-16° to obtain V1. The angle is determined by the empirical threshold, so that the direction of the line connecting Q2 and V1 follows the direction of the finger gap. Q2 and Q3 are mapped to the perpendicular line along the direction of the finger gap to obtain candidate points V2 and V3. Here, the line connecting Q2 and V2 (i.e., the gap between the index and middle fingers) is called L1, and the line connecting Q3 and V3 (i.e., the gap between the ring and little fingers) is called L2.

[0040] As a further solution of the present invention: after the palm direction is corrected in S5, the palm is enhanced by using the Laplace operator, the enhanced image is grayed, and then the edge of the image is extracted by using the Canny operator, and a threshold is set to filter the extracted connectivity to retain only the large connectivity; the contour area of ​​the palm is expanded as a whole, and a horizontal edge detection operator is constructed to detect the contour image to obtain the horizontal contour; the extracted horizontal contour is eroded, and then a skeleton line is extracted using these binary areas, and a threshold is re-filtered to delete small edges; the endpoints of the remaining edges are obtained by using the eight-neighborhood scanning algorithm, and the coordinate information of these endpoints is saved; these key points are filtered, and the distance from each candidate point to the center of mass and the distance from each candidate point to the connecting lines L1 and L2 are calculated; first, the candidate points that are not in the three finger pits are filtered out, and the specific constraints are as follows:

[0041] ;

[0042] ;

[0043] ;

[0044] At this time, the selected key points are all in the finger pits. Then, these points are screened to obtain the two points with the longest distance between them, namely the index finger and middle finger pits and the ring finger and little finger pits as the final key points. In the candidate point set, the specific formula for calculating the two points with the largest distance between them as the palm valley points is:

[0045] .

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. To address the palm tilt problem, the binocular camera is used to obtain the spatial coordinate information of the points on the palm. Then, by constructing the palm plane equation coefficients, a rectangle on the plane is obtained. This rectangle is converted into a rectangle in the image coordinate system through perspective transformation, thereby also correcting the tilt of the palm image and obtaining the final image suitable for segmentation.

[0048] 2. Image segmentation is performed by building a DeepLabv3+ network structure. The feature extraction network of the DeepLabV3+ network selects the MobileNetV2 network model to process palm images. The model parameters are determined by training a large number of labeled palm images. It can effectively solve the palm segmentation problem under difficult conditions such as complex backgrounds and uneven lighting. It is then combined with the traditional Gaussian skin color model for re-segmentation to ensure high accuracy and stability of segmentation without being disturbed by complex backgrounds.

[0049] 3. A method using morphology and mathematical geometry is proposed to fully utilize the connections between various points on the palm, successfully extract the feature information of the palm center area, and make it possible to extract the area of ​​interest of the palm when the palm is closed. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0051] Figure 1 This is the overall framework diagram of the closed palm region of interest extraction method under complex background;

[0052] Figure 2 This is the overall flow chart of the method for extracting the region of interest of a closed palm under complex background;

[0053] Figure 3 The tilt correction feature point information map provided by the present invention;

[0054] Figure 4 This is the feature point information map extracted from the region of interest provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. That is, the embodiments described herein are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein may be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention to be protected, but merely represents selected embodiments of the present invention; based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative work shall fall within the scope of protection of the present invention.

[0057] Example 1

[0058] See also Figure 1、 Figure 2 and Figure 3 As shown, this embodiment is directed to the processing of palm image tilt correction, and the processing flow includes the following steps:

[0059] Step 1: Use a binocular camera and the gesture recognition model supported by the camera to mark the palm skeleton point information and obtain the coordinate depth information of the marked points P1, P2, P3, P4, and P5 in the palm area;

[0060] Step 2: Use the depth information of these points to fit a three-dimensional plane, which should correspond to the palm;

[0061] Step 3: On the plane (i.e. the palm position), use P1 and P3 as reference points to construct a rectangle of the plane. However, this rectangle is not a rectangle in the camera coordinate system.

[0062] Step 4: Expand the obtained rectangle to include the entire palm area and obtain the palm circumscribed rectangle to avoid distortion and loss of hand information during correction.

[0063] Step 5: Use the four key points of the final palm circumscribed rectangular area as the coordinate points for perspective transformation, perform perspective transformation, and successfully correct the tilted palm image.

[0064] In step 2, the three-dimensional coordinates of the palm P1, P2, P3, P4, and P5 are obtained by using binocular cameras and skeleton point detection technology. The plane is fitted using these five discrete points. A three-dimensional plane can be used to To express it, so fitting a plane by discrete points is essentially solving an overdetermined equation. The formula of the overdetermined equation is as follows:

[0065] ;

[0066] in, x 、 y 、 z is the position coordinate of the pixel in the spatial domain, a, b, c is the plane equation coefficient, thus obtaining the fitting plane coefficient Ω(a,b,c).

[0067] In step 3, constructing a planar rectangle includes the following steps:

[0068] Using P3 as the reference point, find the symmetrical point P6 of P1 with respect to P3 and obtain the three-dimensional coordinates of P6. The specific equation is:

[0069] ;

[0070] in, x 、 y 、 zis the three-dimensional coordinate information of the point; in order to obtain the coordinates of the four key points of the rectangle on the palm plane, assume that the three-dimensional coordinates of P7 ( X7 , Y7 , Z7 ), then P7 should meet the following three conditions:

[0071] A) The distance from P7 to P3 should be equal to the distance from P6 to P3;

[0072] B) The vectors of P7 and P3 should be perpendicular to the vectors of P6 and P3;

[0073] C) P7 should be on the fitting plane.

[0074] Assume that the three-dimensional coordinates of P3 are ( X3 , Y3 , Z3 ), the three-dimensional coordinates of P6 are ( X6 , Y6 , Z6 ), we get the following system of equations:

[0075]

[0076] in, X3 、 Y3 、 Z3 、 X6 、 Y6 、 Z6 、 X7 、 Y7 、 Z7 is the three-dimensional coordinate information of the relevant coordinate points, a 、 b 、 c are the coefficients of the plane equation; after solving the three-variable equation group, we get the three-dimensional coordinates of P7, and with P3 as the reference point, we make P8, the symmetrical point of P7.

[0077] In step 4, obtaining the circumscribed rectangular area of ​​the palm includes the following steps:

[0078] Repeat step 3 above, assume and obtain point A, and obtain points B, C, and D respectively through symmetry. At this time, a rectangular area on the palm plane has been obtained, and this area has included the entire hand.

[0079] In step 5, the specific formula for perspective transformation is:

[0080] ;

[0081] in u 、 v 、 w is the original image coordinate, and the corresponding transformed image coordinate is x 、 y 、 w , a 11 ~ a 33 is the transformation matrix.

[0082] After obtaining the transformed homogeneous coordinates, the transformed image coordinates are obtained, and the formula is:

[0083] ;

[0084] ;

[0085] in x , y is the transformed image coordinate.

[0086] Example 2

[0087] See also Figure 1 and Figure 2 As shown, this embodiment is directed to the processing of palm segmentation under complex background, which includes the following steps:

[0088] Step 1: Using approximately 6,000 palm training samples with different backgrounds, a reliable palm image coarse segmentation model is trained using the DeepLabv3+ semantic neural network.

[0089] Step 2: After obtaining the palm mask, perform a slight dilation on the mask to make it slightly larger than the palm;

[0090] Step 3: Perform an AND operation on the processed mask and the palm image to obtain an image with a completely black background area and only the palm area. This effectively blocks out most of the complex background and eliminates most of the interference from the complex background of the palm.

[0091] Step 4: Use the Gaussian skin color model to segment the image, which reduces the instability caused by the neural network segmentation and improves the segmentation accuracy.

[0092] In step 2, when separating the background of the palm image, the DeepLabv3+ semantic neural network segmentation is first used to perform the first segmentation of the palm; in order to make the segmentation model more stable and effective, training samples are added for different hand postures, different lighting environments, and different backgrounds during model training.

[0093] In step 4, after obtaining the image in which the palm is separated from the background, a connected domain analysis is performed on the binary image, and the binary contour is smoothed using a median filter to restore the true contour of the palm as much as possible; the dilation and corrosion operation in morphology is used to remove the holes inside the palm.

[0094] Example 3

[0095] See also Figure 1 、 Figure 2 and Figure 4 As shown, this embodiment is directed to the processing of the closed palm region of interest extraction method, and the processing flow is as follows:

[0096] Step 1: First, use the OpenCV function to obtain the centroid information of the palm area, denoted as C1, and use the convex hull detection technology to obtain the various concave points on the palm; because in a closed palm, the depth of the thumb and index finger concave points is the largest, this feature can be used to locate the thumb and index finger concave points, denoted as Q1;

[0097] Step 2: Compare the distance between the remaining concave points obtained by convex hull detection and the centroid C1 to obtain the middle finger tip farthest from the centroid, recorded as T1;

[0098] Step 3: Connect the tip of the middle finger T1 and the center of mass C1, and extend them to intersect at the wrist, which is recorded as D1;

[0099] Step 4: Traverse the concave point information and calculate the distance from each concave point to point D1, so as to obtain the three concave points with the longest distance between the index finger and the middle finger, between the ring finger and the little finger, and the tip of the middle finger. Then, based on the distance between the three points, select the two points with the largest distance between each other, namely the concave points between the index finger and the middle finger, and between the ring finger and the little finger; and by comparing the distance with point Q1, determine that the concave point between the index finger and the middle finger is Q2, and the concave point between the ring finger and the little finger is Q3.

[0100] Step 5: Draw a perpendicular line between the line connecting Q1 and Q2. Rotate Q1 around Q2 by 14° to 16° to obtain V1. The angle is determined by the empirical threshold, so that the direction of the line connecting Q2 and V1 follows the direction of the finger gap. Map Q2 and Q3 along the finger gap to the perpendicular line to obtain candidate points V2 and V3. Here, the line connecting Q2 and V2 (i.e., the gap between the index and middle fingers) is called L1, and the line connecting Q3 and V3 (i.e., the gap between the ring and pinky fingers) is called L2.

[0101] Step 6: Enhance the image, convert it to grayscale, and extract the horizontal edges to obtain the endpoint information of the horizontal edges of the palm. Then, through screening, the final finger pit key points are obtained to delineate the area of ​​interest of the palm.

[0102] In step 5, the obtained finger seam lines L1 and L2 are used to perform a rotation correction on the palm. The left and right hands are determined by comparing the coordinates of Q2 and Q3, so as to rotate in different directions and finally unify the palm horizontally to the right.

[0103] In step 6, the palm image is enhanced using the Laplacian operator, and the edge is extracted using the Canny operator. A threshold is set to filter the extracted connectivity, and only the large connectivity is retained.

[0104] At this point, the entire contour area of ​​the palm has been obtained, and then the entire contour is expanded, and a horizontal edge detection operator is constructed to detect the contour image to obtain the horizontal contour.

[0105] The extracted horizontal contours are eroded, and then a skeleton line is extracted using these binary areas. Threshold filtering is performed again to delete small edges and obtain the endpoints of these skeleton lines.

[0106] The specific method for screening the final finger nest key points is as follows: use the eight-neighborhood scanning algorithm to obtain the endpoints of the remaining edges and save the coordinate information of these endpoints; screen these key points and calculate the distance from each candidate point to the center of mass and the distance from each candidate point to the connecting lines L1 and L2; first filter out candidate points that are not in the three finger nests, and the specific constraints are as follows:

[0107] ;

[0108] ;

[0109] ;

[0110] At this time, the selected key points are all in the finger pits. Then, by screening these points, the two points with the longest distance between them are obtained, namely the index finger and middle finger pits and the ring finger and little finger pits as the final key points; in the candidate point set, the specific formula for calculating the two points with the largest pairwise distance as the palm valley points is:

[0111] .

[0112] The above content is merely an example and explanation of the structure of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the scope of protection of the present invention.

Claims

1. A method for extracting regions of interest from closed palms under complex backgrounds, characterized in that: The steps include: S1: Obtain biometric information of the palm's circumscribed rectangle. Leveraging the specificity of the rectangle, perform perspective transformation on specific points to correct the tilted palm. S2: Use DeepLabv3+ semantic neural network to build a segmentation model based on palm contours to separate the corrected palm from the complex background. Then, combine it with the Gaussian skin color model to improve segmentation accuracy and generate a binary image. S3: Perform maximum connected component analysis on the binary image to further eliminate errors in the palm image segmentation process, and then perform morphological operations to eliminate holes in the palm area; S4: After obtaining the complete palm binary image, perform convex hull detection on the closed palm to obtain the concave points between the fingertips. Through morphology and geometry, two lines L1 and L2 passing through the gaps between the index finger and middle finger, and the ring finger and little finger are obtained; S5: Use the Canny operator to perform edge detection on the palm image, and use the horizontal operator to obtain the horizontal edge. Then obtain each edge endpoint as a candidate point, and filter it by the distance from the candidate point to the center of mass and the distance from the candidate point to L1 and L2 to obtain the final two key points to extract the palm area of ​​interest.

2. The method for extracting the region of interest of a closed palm under complex background according to claim 1, characterized in that: In S1, the three-dimensional coordinates of the palms P1, P2, P3, P4, and P5 are obtained by using a binocular camera and skeleton point detection technology. These five discrete points are used to fit a plane. Using P3 as a reference point, a symmetrical point P6 is made with respect to P1, and the three-dimensional coordinates of P6 are obtained. The point P7 is obtained by the following three conditions: A) The distance from P7 to P3 is equal to the distance from P6 to P3; B) The vectors of P7 and P3 are perpendicular to the vectors of P6 and P3; C) P7 is on the fitting plane; After obtaining P7, make the symmetrical point P8 of P7; Repeat the above steps, assume and obtain point A, and use symmetry to obtain points B, C, and D respectively, to obtain a rectangular area on the palm plane, which already includes the entire hand; the obtained points A, B, C, and D are passed as parameters into the perspective transformation equation, so that the four points form a standard rectangle in the image coordinate system, completing the correction of the tilted palm.

3. The method for extracting the region of interest of a closed palm under complex background according to claim 2, characterized in that: Fitting a plane using the five discrete points P1, P2, P3, P4, and P5 is essentially solving an overdetermined equation to obtain the fitted plane coefficients Ω(a, b, c). The formula for the overdetermined equation is as follows: ； in, x 、 y 、 z is the position coordinate of the pixel in the spatial domain, a 、 b 、 c are the coefficients of the plane equation, Assume that the three-dimensional coordinates of P7 are ( X7 , Y7 , Z7 ), the three conditional equations for point P7 are: ； in, X3 、 Y3 、 Z3 、 X7 、 Y7 、 Z7 is the three-dimensional coordinate information of the relevant coordinate points, a 、 b 、 c is the plane equation coefficient, and the perspective transformation equation is: ； in u 、 v 、 w is the original image coordinate, and the corresponding transformed image homogeneous coordinate is x 、 y 、 w , a 11 ~a 33 is the transformation matrix; After obtaining the transformed homogeneous coordinates, the transformed image coordinates are obtained, and the formula is: ； ； in x , y is the transformed image coordinate.

4. The method for extracting the region of interest of a closed palm under complex background according to claim 1, characterized in that: In S2, DeepLabv3+ semantic segmentation is first used to perform the first segmentation of the palm. After obtaining the binary image segmented by DeepLabv3+ semantic segmentation, the binary area with the largest connected domain is selected as the palm area, the binary area is moderately expanded, and an "AND" operation is performed with the original palm image so that the complex background area except the palm is shielded; the image after the "AND" operation is segmented using a traditional method based on the Gaussian skin color model.

5. The method for extracting the region of interest of a closed palm under complex background according to claim 1, characterized in that: In S3, a connected domain analysis is performed on the binary image obtained by segmentation, a median filter is used to smooth the binary contour, and a dilation and corrosion operation in morphology is used to remove holes inside the palm.

6. The method for extracting the region of interest of a closed palm under complex background according to claim 1, characterized in that: In said S4, the convex hull detection technology is used to obtain information of a group of concave points on the closed palm, and the Opencv function is used to obtain the coordinate information of the centroid point C1 of the binary image and the palm contour point information; Using the depth of the concave point, we can find that the point with the largest depth is the concave point between the thumb and index finger, recorded as Q1; Compare the distances between the remaining concave points and the centroid, and the point with the farthest distance is the tip of the middle finger T1; Connect T1 and the centroid C1 and extend them to intersect the palm contour, the intersection point is recorded as D1; Traverse the concave point information and calculate the distance from each concave point to point D1. The three concave points with the longest distances are between the index finger and the middle finger, between the ring finger and the little finger, and the tip of the middle finger. The concave point between the index finger and the middle finger is determined to be Q2, and the concave point between the ring finger and the little finger is determined to be Q3. Draw a perpendicular line between the line connecting Q1 and Q2, rotate Q1 around Q2 by 14°~16° to obtain V1, so that the direction of the line connecting Q2 and V1 follows the direction of the finger gap, and map Q2 and Q3 along the direction of the finger gap to the perpendicular line to obtain candidate points V2 and V3. The line connecting Q2 and V2 is L1, and the line connecting Q3 and V3 is L2.

7. The method for extracting a closed palm region of interest under a complex background according to claim 6, characterized in that: In S5, the palm image is enhanced using the Laplace operator, the enhanced image is grayscaled, and then the image edge is extracted using the Canny operator. A threshold is set and the extracted connectivity is screened to retain only the large connectivity. The palm contour area is expanded as a whole, and a horizontal edge detection operator is constructed to detect the contour image to obtain the horizontal contour. The extracted horizontal contour is eroded, and then a skeleton line is extracted using these binary areas. Threshold screening is performed again to delete small edges. The eight-neighborhood scanning algorithm is used to obtain the endpoints of the remaining edges, and the coordinate information of these endpoints is saved to form a candidate point set P i ; These candidate points are screened and the distance from each candidate point to the centroid C1 and the distance from each candidate point to the connecting lines L1 and L2 are calculated. First, candidate points that are not in the three finger pits are screened out. The specific constraints are as follows: ； ； ； After screening, the two points with the longest distance between them are obtained, namely the index finger and middle finger nests and the ring finger and little finger nests as the final key points.

Images