Human anatomy-based palm vein recognition method, device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a palm vein recognition method and device based on human anatomy, equipment and a medium, and the method comprises the steps: obtaining a palm vein image sequence of a user; carrying out anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence; according to the anatomical key point coordinates, carrying out geometric correction processing on an ROI region of the palm vein image sequence to obtain a standardized ROI region sequence; performing multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features; and performing cross-modal fusion processing on the vein texture features and the hemodynamic features to obtain palm vein fusion features. According to the invention, the recognition rate and safety of palm vein recognition can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a palm vein recognition method and device based on human anatomy, equipment and medium. BACKGROUND

[0002] Palm vein recognition has wide application value. For example, in the financial field, palm vein recognition can be used for identity authentication of transactions; in the security field, palm vein recognition helps door access management or security protection in public places (such as airports, stations, museums, etc.); in the medical field, palm vein recognition is used for patient identity authentication or secure access and protection of medical data.

[0003] Traditional palm vein recognition schemes mainly rely on the extraction of global texture or local curve features, but these features are easily affected by changes in light and palm posture, and have the defect of insufficient stability of features, thereby causing the recognition rate to decrease, and the global texture or local curve features belong to static features, which are easy to be imitated, and have weak anti-counterfeiting ability, resulting in low security. SUMMARY

[0004] The present application provides a palm vein recognition method and device based on human anatomy, a computer device and a storage medium, aiming to solve the technical problems of low recognition rate and low security of existing palm vein recognition schemes.

[0005] In a first aspect, a palm vein recognition method based on human anatomy is provided, comprising:

[0006] obtaining a palm vein image sequence of a user;

[0007] performing anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence;

[0008] performing geometric correction processing on the ROI region of the palm vein image sequence according to the anatomical key point coordinates to obtain a standardized ROI region sequence;

[0009] performing multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features;

[0010] performing cross-modal fusion processing on the vein texture features and the hemodynamic features to obtain palm vein fusion features.

[0011] In a second aspect, a palm vein recognition device based on human anatomy is provided, comprising:

[0012] an image acquisition module for acquiring a palm vein image sequence of a user;

[0013] A key point positioning module is configured to perform anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence.

[0014] An ROI correction module is configured to perform geometric correction processing on an ROI region of the palm vein image sequence according to the anatomical key point coordinates to obtain a standardized ROI region sequence.

[0015] A feature extraction module is configured to perform multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features.

[0016] A feature fusion module is configured to perform cross-modal fusion processing on the vein texture features and the hemodynamic features to obtain palm vein fusion features.

[0017] In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the palm vein recognition method based on human anatomy when executing the computer program.

[0018] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the palm vein recognition method based on human anatomy when executed by a processor.

[0019] In the solution implemented by the above-mentioned palm vein recognition method, device, computer equipment and storage medium based on human anatomy, a user's palm vein image sequence is obtained; anatomical key point positioning processing is performed on the palm vein image sequence to obtain the anatomical key point coordinates of the palm vein image sequence; based on the anatomical key point coordinates, the ROI area of ​​the palm vein image sequence is geometrically corrected to obtain a standardized ROI area sequence; multimodal feature extraction processing is performed on the standardized ROI area sequence to obtain vein texture features and hemodynamic features; and cross-modal fusion processing is performed on the vein texture features and hemodynamic features to obtain palm vein fusion features. In the present invention, on the one hand, the coordinates of the anatomical key points of the palm vein image sequence are located to provide an accurate and reasonable basis for extracting the standardized ROI region sequence from the palm vein image sequence. On the other hand, geometric correction processing of the ROI region based on the anatomical key point coordinates can effectively solve the problem of palm rotation, improve the accuracy of the standardized ROI region sequence, and help improve the accuracy and stability of palm vein recognition. On the other hand, multimodal feature extraction processing is performed on the standardized ROI region sequence to achieve rapid extraction of vein texture features and hemodynamic features, so that the vein texture features and hemodynamic features complement each other and have strong stability. At the same time, the comprehensiveness and accuracy of feature extraction are improved, and the difficulty of counterfeiting or forging these two types of features is increased, which helps to improve the security of palm vein recognition. Finally, cross-modal fusion processing of vein texture features and hemodynamic features can fully utilize the complementarity of the two, and provide a more comprehensive, richer, and more detailed feature representation for the palm vein image sequence, which can better represent the comprehensive information of the palm vein image sequence, thereby better reflecting the user's biometric characteristics and enhancing the reliability of palm vein recognition. As a result, the recognition efficiency, recognition accuracy, stability, robustness and adaptability of palm vein recognition are improved, thereby improving the recognition rate and security of palm vein recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 1 is a flow chart of a palm vein recognition method based on human anatomy in one embodiment of the present invention;

[0022] Figure 2 yes Figure 1 A schematic flow chart of a specific implementation of step S40;

[0023] Figure 3 is another flow diagram of the palm vein recognition method based on human anatomy in an embodiment of the present application;

[0024] Figure 4 is a structural diagram of the palm vein recognition device based on human anatomy in an embodiment of the present application;

[0025] Figure 5 is a structural diagram of the computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0027] The palm vein recognition method based on human anatomy provided by the embodiment of the present application can be applied to a computer device, and the computer device can acquire a palm vein image sequence of a user; anatomical key point positioning processing is performed on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence; according to the anatomical key point coordinates, geometric correction processing is performed on a ROI region of the palm vein image sequence to obtain a standardized ROI region sequence; multi-modal feature extraction processing is performed on the standardized ROI region sequence to obtain vein texture features and hemodynamic features; and cross-modal fusion processing is performed on the vein texture features and the hemodynamic features to obtain palm vein fusion features. In the present application, on the one hand, the anatomical key point coordinates of the palm vein image sequence are located to provide accurate and reasonable basis for extracting the standardized ROI region sequence of the palm vein image sequence; on the other hand, the ROI region is geometrically corrected based on the anatomical key point coordinates, which can effectively solve the palm rotation problem, improve the accuracy of the standardized ROI region sequence, and help improve the accuracy and stability of palm vein recognition; on the other hand, multi-modal feature extraction processing is performed on the standardized ROI region sequence to quickly extract the vein texture features and the hemodynamic features, so that the vein texture features and the hemodynamic features complement each other, have strong stability, and at the same time improve the comprehensiveness and accuracy of feature extraction, increase the difficulty of imitating or forging the two types of features, and help improve the security of palm vein recognition; finally, cross-modal fusion processing is performed on the vein texture features and the hemodynamic features, which can fully play the complementarity of the two features, provide more comprehensive, richer and more detailed feature representation for the palm vein image sequence, better represent the comprehensive information of the palm vein image sequence, and thus better reflect the biological characteristics of the user, thereby enhancing the reliability of palm vein recognition. Therefore, the recognition efficiency, recognition accuracy, stability, robustness and adaptability of palm vein recognition are improved, thereby improving the recognition rate and security of palm vein recognition.

[0028] The computer device can include a terminal device or a server. The terminal device can include a smartphone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, a wearable device, etc. The server can be a standalone server or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc.

[0029] Please refer to Figure 1 , Figure 1 A flowchart of the palm vein recognition method based on human anatomy provided by the embodiment of the present application is shown in the figure, which includes the following steps:

[0030] S10: Acquire a palm vein image sequence of the user.

[0031] The palm vein recognition method based on human anatomy provided by the present invention combines artificial intelligence technology and anatomical knowledge of the human palm to accurately extract palm vein fusion features. It is mainly divided into three processes: process one is ROI extraction driven by anatomical key points, process two is multimodal feature extraction, and process three is cross-modal feature fusion, so as to improve the recognition rate of palm vein recognition and prevent forgery attacks. It has high security and real-time performance.

[0032] For ease of understanding, the terms involved in the present invention are first explained:

[0033] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0034] The pre-trained HRNet model is an efficient deep learning model that can accurately detect the positions of human joints and other key points. In this embodiment of the present invention, it is used to extract anatomical key points.

[0035] Lightweight Feature Fusion Network: This is a deep learning model designed to efficiently process and fuse different modalities. It has a small number of computational parameters and is suitable for running in resource-constrained environments. It can effectively extract and fuse features from different sources. It consists of an input layer, a backbone network, and an output layer.

[0036] Among them, the input layer supports dual-channel input.

[0037] The backbone network consists of four groups of Depthwise-Res layers, each of which contains three depthwise separable convolution layers. The advantages of depthwise separable convolution are low computational complexity and few parameters. The structure of each group of Depthwise-Res layers is as follows:

[0038] Output=BN(DepthwiseConv(ReLU(BN(Input))))+Input

[0039] Depthwise-Res combines the operations of Batch Normalization (BN), ReLU activation function and depthwise separable convolution, and helps lightweight feature fusion network to better train and converge through residual connection (input added to output). The number of channels of each group of Depthwise-Re gradually increases (such as from 32 to 64, from 64 to 128, and from 128 to 256), and a channel expansion factor (such as 6) is used. This design enables the network to gradually learn more rich features while maintaining low computational complexity.

[0040] The lightweight feature fusion network has a small number of parameters (such as only 2.3M), indicating that the model is relatively lightweight and suitable for application in scenarios with limited computational resources.

[0041] The output layer is a multi-layer perceptron (MLP) that can output a multi-dimensional (such as 1024-dimensional) comprehensive feature vector.

[0042] Based on this, the palm vein recognition method based on human anatomy provided by the present application is described in detail below.

[0043] In step S10 of some embodiments, a raw palm vein image sequence of a user can be captured; and the raw palm vein image sequence is dynamically denoised to obtain a palm vein image sequence.

[0044] In step S10, a raw palm vein image sequence of a user can be first captured by a near-infrared LED light source of a specific wavelength (such as 850nm) and a high-resolution (such as resolution greater than 500dpi) CMOS sensor, ensuring that a clear high-quality raw palm vein image sequence is captured.

[0045] The raw palm vein image sequence is composed of a plurality of consecutive raw palm vein images, which can provide more details about vein texture and blood flow changes.

[0046] The raw palm vein image sequence is then dynamically denoised to obtain a palm vein image sequence, further improving the quality, adaptability and usability of the palm vein image sequence.

[0047] The dynamic denoising process includes aligning consecutive frames using optical flow method, enhancing vein contrast and eliminating motion blur.

[0048] ①Align consecutive frames using optical flow method: Optical flow method is an effective method for calculating pixel motion between consecutive frames. Due to the subtle movement of the hand during capture, there may be some deviation between consecutive raw palm vein images in the raw palm vein image sequence. Therefore, the raw palm vein image sequence can be aligned using the optical flow method to obtain an aligned palm vein image sequence, which can reduce the deviation.

[0049] ②Enhancing vein contrast: the aligned palm vein image sequence can be processed for vein contrast enhancement to enhance the contrast between the veins and the surrounding tissues, obtaining the enhanced palm vein image sequence, so that the shape and distribution of the veins are more obvious, facilitating the subsequent extraction of vein texture features.

[0050] ③Eliminating motion blur: due to the slight motion of the hand during capture, the original palm vein image may also be blurred, therefore, the enhanced palm vein image sequence can be processed for motion blur elimination in the time domain or spatial domain, obtaining the palm vein image sequence, and improving the clarity of the palm vein image sequence.

[0051] S20: anatomical key point positioning processing is performed on the palm vein image sequence to obtain the anatomical key point coordinates of the palm vein image sequence.

[0052] The anatomical key point positioning processing is performed on the palm vein image sequence to obtain the anatomical key point coordinates of the palm vein image sequence, which provides accurate and reasonable basis for extracting the standardized ROI region sequence of the palm vein image sequence.

[0053] In step S20 of some embodiments, the metacarpal root point and the phalangeal and metacarpal connection point of the palm vein image sequence can be positioned based on the skeletal anatomy of the palm by using a pre-trained HRNet model, to obtain the metacarpal root point coordinates and the phalangeal and metacarpal connection point coordinates of the palm vein image sequence, wherein the metacarpal root point coordinates and the phalangeal and metacarpal connection point coordinates constitute the anatomical key point coordinates.

[0054] In step S20, six anatomical key points are positioned based on the skeletal anatomy of the palm by using a pre-trained HRNet model, to obtain six anatomical key point coordinates, wherein the six anatomical key points include: a metacarpal root point p1, and connection points (defined as phalangeal and metacarpal connection points) p2-p6 of the five phalanges of the thumb, index finger, middle finger, ring finger, and little finger with the metacarpal.

[0055] S30: According to the anatomical key point coordinates, the ROI region of the palm vein image sequence is geometrically corrected to obtain a standardized ROI region sequence.

[0056] For step S30, first, the six anatomical key point coordinates are used to extract the Region of Interest (ROI) of the palm vein image sequence to obtain the ROI region of each frame of the palm vein image in the palm vein image sequence.

[0057] Then, according to the coordinates of the metacarpal root point p1 in the anatomical key point coordinates and the coordinates of the connection point p5 of the ring finger and the metacarpal The rotation angle θ of the palm in the ROI region of each frame of palm vein image is calculated by a rotation angle calculation formula, and the rotation angle calculation formula is as follows:

[0058]

[0059] Next, affine transformation is performed on the ROI region of each frame of palm vein image according to the rotation angle θ of the palm, and a standardized ROI region sequence is obtained. Affine transformation is a technique for transforming images by translation, rotation and scaling. By performing affine transformation on the ROI region, that is, by rotating, translating and scaling to correct the position of the ROI region, a standardized ROI region sequence can be obtained.

[0060] Specifically, an affine matrix M is obtained according to the rotation angle θ of the palm, and the affine matrix M is as follows:

[0061]

[0062] where t x , t y represents the translation amount, which can be calculated by aligning the geometric centers of the anatomical key points.

[0063] For the coordinates of each pixel point in the ROI region, the new coordinates of each pixel point can be obtained by transforming the coordinates through the affine matrix M, thereby updating the ROI region and obtaining a standardized ROI region sequence. For example, the ROI region is standardized to a 256x256 pixel region by affine transformation.

[0064] In this way, by performing geometric correction processing on the ROI region, the problem of palm rotation in the ROI region can be effectively solved, and the accuracy and usability of the standardized ROI region sequence can be improved.

[0065] S40: Perform multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features.

[0066] After that, multi-modal feature extraction processing is performed on the standardized ROI region sequence to obtain vein texture features and hemodynamic features. Vein texture features reflect the morphology and structure of veins, and hemodynamic features reflect the speed, direction and other dynamic characteristics of blood flow. These two types of features complement each other and have strong stability, which can better reflect the biological characteristics of the user.

[0067] And at the same time, the difficulty of counterfeiting vein texture features or forging blood flow dynamics features will be greatly increased. Attackers not only need to replicate the external structure of the vein, but also need to simulate the dynamic characteristics of blood flow, which makes the palm vein recognition scheme based on multi-modal feature extraction of the embodiments of the present application have higher security when facing attacks, can greatly improve the recognition accuracy, robustness and security of the palm vein recognition scheme provided by the embodiments of the present application, and at the same time can overcome the limitations of single feature to improve the recognition rate.

[0068] In some embodiments, referring to Figure 2 , step S40 can include but is not limited to the following steps:

[0069] S41: Perform vein skeleton extraction processing on the standardized ROI region sequence to obtain a vein skeleton graph;

[0070] S42: Perform blood flow velocity extraction processing on the standardized ROI region sequence to obtain a blood flow velocity histogram;

[0071] S43: Perform vein texture feature extraction processing on the vein skeleton graph to obtain vein texture features;

[0072] S44: Perform encoding processing on the blood flow velocity histogram to obtain a blood flow graph;

[0073] S45: Perform blood flow dynamics feature extraction processing on the blood flow graph to obtain blood flow dynamics features.

[0074] For step S41, the standardized ROI region sequence can be subjected to vein skeleton extraction processing to obtain a vein skeleton graph. Specifically, the standardized ROI region sequence is first subjected to Gaussian filtering processing to remove noise of the standardized ROI region sequence.

[0075] Then, the vein skeleton is extracted from the standardized ROI region sequence after Gaussian filtering by using the maximum curvature method to obtain the vein skeleton graph, which involves curvature calculation and skeletonization decision.

[0076] ①Curvature calculation:

[0077] Curvature is a quantity used to describe the shape change of an image and is used to identify edges and contours in an image. Based on this, for each pixel point (x, y) in the standardized ROI region sequence after Gaussian filtering, its curvature K(x, y) can be calculated by the curvature calculation formula, and the curvature calculation formula is:

[0078]

[0079] Wherein, L(x, y) represents the standardized ROI region sequence after Gaussian filtering.

[0080] L xrepresents the gradient of the normalized ROI region sequence after Gaussian filtering in the x direction;

[0081] L y represents the gradient of the normalized ROI region sequence after Gaussian filtering in the y direction;

[0082] L xx , L yy , L xy respectively represent the second derivative, which is a matrix obtained by twice differentiating L(x, y).

[0083] ② Skeletonization decision

[0084] According to the curvature K(x, y), a decision rule is used to decide which part of the normalized ROI region sequence after Gaussian filtering belongs to the vein skeleton, and a vein skeleton map is obtained, wherein the formula of the decision rule is as follows:

[0085]

[0086] Wherein S(x, y) represents the vein skeleton map;

[0087] τ high represents the curvature threshold, which is used to filter out obvious vein textures;

[0088] τ grad represents the gradient threshold, which is used to ensure that the selected part is obviously changed.

[0089] In this way, by using Gaussian filtering, maximum curvature method and other technologies to extract the center line of the vein, a binary vein skeleton map S(x, y) can be quickly and accurately obtained, in which the part of the vein skeleton is marked as 1 and the remaining part is marked as 0, for vein texture feature extraction.

[0090] For step S42, blood flow velocity extraction processing can be performed on the normalized ROI region sequence to obtain a blood flow map, which involves dynamic absorption model construction and blood flow velocity estimation.

[0091] ① Dynamic absorption model construction:

[0092] Because the hemoglobin in the blood can absorb light of a specific wavelength, different blood areas in the image absorb light to different degrees. Therefore, by constructing a dynamic absorption model for describing the absorption characteristics of blood in the normalized ROI region sequence, the dynamic absorption model is as follows:

[0093]

[0094] Wherein A(x, y, t) represents the absorption value of each pixel point (x, y) and time t in the normalized ROI region sequence;

[0095] I0 represents the incident light intensity;

[0096] μ a represents the absorption coefficient of hemoglobin, such as at a wavelength of 850 nm, μ a is 0.3 mm-1;

[0097] d(x, y, t) represents the distance (or blood vessel depth) of blood at pixel point (x, y) and time t;

[0098] ε(t) represents a noise term.

[0099] The dynamic absorption model can reflect the blood absorption characteristics of different blood vessel parts in the standardized ROI region sequence.

[0100] ②Blood flow velocity estimation:

[0101] The estimation of blood flow velocity is based on the displacement of blood vessels in the standardized ROI region sequence. The same blood vessel point on the blood vessel is tracked by the optical flow method, and the displacement change between multiple time points is used to estimate the blood flow velocity v (unit: mm / s). The blood flow velocity can be calculated by the blood flow velocity calculation formula, and the calculation formula of blood flow velocity is:

[0102]

[0103] wherein, represents the displacement change of the same blood vessel in the standardized ROI region sequence;

[0104] represents the time interval of the standardized ROI region sequence;

[0105] PixelSize represents the actual physical size represented by each pixel point in the standardized ROI region sequence.

[0106] In this way, the blood flow velocity is calculated by tracking the displacement of the blood vessel point.

[0107] Therefore, the blood flow velocity histogram is generated based on the dynamic absorption model and the blood flow velocity for blood flow dynamics feature extraction.

[0108] For step S43, the vein skeleton map S(x, y) is input to the vein texture branch of the lightweight feature fusion network, so that the vein texture branch performs vein texture feature extraction processing on the vein skeleton map S(x, y), and outputs the vein texture feature F vein .

[0109] For steps S44-S45, the blood flow velocity histogram is input to the blood flow branch of the lightweight feature fusion network, so that the blood flow branch performs encoding processing on the blood flow velocity histogram, and obtains the blood flow map Vflow and the blood flow graph is subjected to blood flow dynamics feature extraction processing to obtain blood flow dynamics features F flow , wherein the blood flow graph V flow has a resolution consistent with the vein skeleton graph S(x, y).

[0110] Thus, multi-modal feature extraction is achieved through the vein texture branch and the blood flow branch of the lightweight feature fusion network, improving the efficiency and accuracy of extracting vein texture features and blood flow dynamics features.

[0111] S50: Cross-modal fusion processing is performed on the vein texture features and the blood flow dynamics features to obtain palm vein fusion features.

[0112] Finally, cross-modal fusion processing is performed on the vein texture features and the blood flow dynamics features through the cross-modal attention mechanism of the lightweight feature fusion network to obtain palm vein fusion features.

[0113] In step S50 of some embodiments, the vein texture features and the blood flow dynamics features can be weighted and fused through the cross-modal attention mechanism to obtain palm vein fusion features.

[0114] In step S50, the vein texture features and the blood flow dynamics features are weighted and fused through the cross-modal attention mechanism to obtain palm vein fusion features F fuse , and the fusion process is completed through the following formula:

[0115]

[0116] wherein W1 and W2 represent learnable weight parameters;

[0117] sigmoid represents an activation function;

[0118] represents element-wise multiplication;

[0119] represents channel concatenation (concatenating the vein texture features and the blood flow dynamics features according to the channel dimension).

[0120] Thus, by performing cross-modal fusion processing on the vein texture features and the blood flow dynamics features, more comprehensive, richer, and more detailed feature representations are provided for the palm vein image sequence, which can better represent the comprehensive information of the palm vein image sequence, thereby better reflecting the biological features of the user.

[0121] After step S44 of some embodiments, the user can also be subjected to a live body verification according to the vein skeleton graph and the blood flow graph.

[0122] The relationship between blood flow velocity and vessel cross-sectional area is inferred using the basic principle of hemodynamics, and a linear regression model is used to determine whether it is a living body. The basic principle is that when blood flow remains constant, blood flow velocity is inversely proportional to the cross-sectional area of the blood vessel, that is, the larger the cross-sectional area of the blood vessel, the slower the blood flow velocity.

[0123] Based on this basic principle, a linear regression model can be constructed to describe the relationship between blood flow velocity and vessel cross-sectional area using the vein skeleton map S(x, y) and the blood flow map V flow .

[0124] Specifically, ① first randomly select n regions from the vein skeleton map S(x, y) and the blood flow map V flow , n is a positive integer greater than or equal to 2, such as n = 10, and the size of each region is, for example, 10x10.

[0125] ② For each selected region, count the number of black pixels (reflecting the cross-sectional area of the blood vessel) and the blood flow velocity (obtained through the blood flow map V flow ) in the region, so that from each selected region, a pair of data (x i , y i ) can be obtained, where x i represents the number of black pixels, and y i represents the blood flow velocity, and ultimately an array {(x1, y1), (x2, y2), …, (x n , y n )} containing n pairs of data points can be obtained.

[0126] ③ According to the array, it can be concluded that there is a linear relationship between blood flow velocity and vessel cross-sectional area:

[0127] y = ax + b

[0128] where a represents the slope;

[0129] b represents the intercept.

[0130] ④ Find the better a and b through the least squares method, that is, according to the least squares method, define the objective function as the sum of the squares of the residuals, and the objective function is as follows:

[0131]

[0132] Find the values of a and b when S(a, b) is minimized.

[0133] V, wherein a is less than 0, indicates that the blood flow velocity and the cross-sectional area are consistent with the characteristics of a living body, i.e., the larger the cross-sectional area of the blood vessel, the slower the blood flow velocity, and this inverse relationship is generally consistent with the characteristics of a living body, and thus it can be determined that the user is a living body, and the user passes the living body verification.

[0134] Thus, based on the vein skeleton map and the blood flow map, the living body verification of the user is implemented, and fraud or forgery can be effectively avoided.

[0135] Referring to Figure 3 After step S50 of some embodiments, the following step can also be included:

[0136] S60: According to the cosine distance between the palm vein fusion feature and the pre-stored legal palm vein feature, the legality of the user is verified.

[0137] The cosine distance between the palm vein fusion feature of the user and the pre-stored legal palm vein feature can be calculated, and the cosine distance can reflect the similarity between the palm vein fusion feature of the user and the pre-stored legal palm vein feature. Therefore, when the cosine distance is less than a preset threshold, it indicates that the palm vein fusion feature of the user is very similar to the pre-stored legal palm vein feature, and the user's identity is legal, and the user passes the legality verification.

[0138] Thus, by calculating the cosine distance, the similarity between the palm vein fusion feature and the pre-stored legal palm vein feature can be quantified, and thus the legality verification of the user can be accurately implemented.

[0139] The palm vein recognition method based on human anatomy provided by an embodiment of the present invention obtains a user's palm vein image sequence; performs anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence; performs geometric correction processing on the ROI area of ​​the palm vein image sequence based on the anatomical key point coordinates to obtain a standardized ROI area sequence; performs multimodal feature extraction processing on the standardized ROI area sequence to obtain vein texture features and hemodynamic features; and performs cross-modal fusion processing on the vein texture features and hemodynamic features to obtain palm vein fusion features. In the present invention, on the one hand, the coordinates of the anatomical key points of the palm vein image sequence are located to provide an accurate and reasonable basis for extracting the standardized ROI region sequence from the palm vein image sequence. On the other hand, geometric correction processing of the ROI region based on the anatomical key point coordinates can effectively solve the problem of palm rotation, improve the accuracy of the standardized ROI region sequence, and help improve the accuracy and stability of palm vein recognition. On the other hand, multimodal feature extraction processing is performed on the standardized ROI region sequence to achieve rapid extraction of vein texture features and hemodynamic features, so that the vein texture features and hemodynamic features complement each other and have strong stability. At the same time, the comprehensiveness and accuracy of feature extraction are improved, and the difficulty of counterfeiting or forging these two types of features is increased, which helps to improve the security of palm vein recognition. Finally, cross-modal fusion processing of vein texture features and hemodynamic features can fully utilize the complementarity of the two, and provide a more comprehensive, richer, and more detailed feature representation for the palm vein image sequence, which can better represent the comprehensive information of the palm vein image sequence, thereby better reflecting the user's biometric characteristics and enhancing the reliability of palm vein recognition. As a result, the recognition efficiency, recognition accuracy, stability, robustness and adaptability of palm vein recognition are improved, thereby improving the recognition rate and security of palm vein recognition.

[0140] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0141] In one embodiment, a palm vein recognition device based on human anatomy is provided, which corresponds one-to-one with the palm vein recognition method based on human anatomy in the above embodiment. Figure 4 As shown, the palm vein recognition device based on human anatomy includes an image acquisition module 101, a key point positioning module 102, a ROI correction module 103, a feature extraction module 104 and a feature fusion module 105. The functional modules are described in detail as follows:

[0142] An image acquisition module 101 is used to acquire a sequence of palm vein images of a user;

[0143] a key point positioning module 102, configured to perform anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence;

[0144] an ROI correction module 103, configured to perform geometric correction processing on an ROI region of the palm vein image sequence according to the anatomical key point coordinates to obtain a standardized ROI region sequence;

[0145] a feature extraction module 104, configured to perform multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features;

[0146] a feature fusion module 105, configured to perform cross-modal fusion processing on the vein texture features and the hemodynamic features to obtain palm vein fusion features.

[0147] In an embodiment, the key point positioning module 102 is specifically configured to:

[0148] perform positioning processing on metacarpal root points and phalangeal and metacarpal connection points of the palm vein image sequence based on a skeletal anatomical structure of a palm through a pre-trained HRNet model to obtain metacarpal root point coordinates and phalangeal and metacarpal connection point coordinates, wherein the metacarpal root point coordinates and the phalangeal and metacarpal connection point coordinates constitute the anatomical key point coordinates.

[0149] In an embodiment, the feature extraction module 104 is specifically configured to:

[0150] perform vein skeleton extraction processing on the standardized ROI region sequence to obtain a vein skeleton graph;

[0151] perform blood flow velocity extraction processing on the standardized ROI region sequence to obtain a blood flow velocity histogram;

[0152] perform vein texture feature extraction processing on the vein skeleton graph to obtain the vein texture features;

[0153] perform encoding processing on the blood flow velocity histogram to obtain a blood flow graph;

[0154] perform hemodynamic feature extraction processing on the blood flow graph to obtain the hemodynamic features.

[0155] In an embodiment, the palm vein recognition device based on human anatomy further includes a living body verification module, and the living body verification module is configured to:

[0156] perform living body verification on the user according to the vein skeleton graph and the blood flow graph.

[0157] In one embodiment, the feature fusion module 105 is specifically configured to:

[0158] The vein texture feature and the hemodynamic feature are weightedly fused through a cross-modal attention mechanism to obtain the palm vein fusion feature.

[0159] In one embodiment, the palm vein recognition device based on human anatomy further includes a legitimacy verification module, which is configured to:

[0160] The legitimacy of the user is verified based on the cosine distance between the palm vein fusion feature and the pre-stored legal palm vein feature.

[0161] In one embodiment, the image acquisition module 101 is specifically configured to:

[0162] capturing a sequence of original palm vein images of the user;

[0163] Dynamic denoising is performed on the original palm vein image sequence to obtain the palm vein image sequence.

[0164] The present invention provides a palm vein recognition device based on human anatomy. On the one hand, by locating the coordinates of anatomical key points of a palm vein image sequence, an accurate and reasonable basis is provided for extracting a standardized ROI region sequence from the palm vein image sequence. On the other hand, geometric correction processing of the ROI region based on the anatomical key point coordinates can effectively solve the problem of palm rotation, improve the accuracy of the standardized ROI region sequence, and help improve the accuracy and stability of palm vein recognition. On the other hand, by performing multimodal feature extraction processing on the standardized ROI region sequence, vein texture features and hemodynamic features can be rapidly extracted, so that the vein texture features and hemodynamic features complement each other and have strong stability. At the same time, the comprehensiveness and accuracy of feature extraction are improved, and the difficulty of counterfeiting or forging these two types of features is increased, which helps to improve the security of palm vein recognition. Finally, by cross-modal fusion processing of vein texture features and hemodynamic features, the complementarity of the two can be fully utilized to provide a more comprehensive, richer, and more detailed feature representation for the palm vein image sequence, which can better represent the comprehensive information of the palm vein image sequence, thereby better reflecting the user's biometric characteristics and enhancing the reliability of palm vein recognition. As a result, the recognition efficiency, recognition accuracy, stability, robustness and adaptability of palm vein recognition are improved, thereby improving the recognition rate and security of palm vein recognition.

[0165] The specific limitations of the palm vein recognition device based on human anatomy can refer to the limitations of the palm vein recognition method based on human anatomy described above, and will not be repeated here. Each module in the palm vein recognition device based on human anatomy described above can be realized by software, hardware and their combination in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in the form of software, so that the processor calls and executes the operations corresponding to each module.

[0166] In one embodiment, a computer device is provided, and its internal structure diagram can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media, internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to implement the functions or steps of the palm vein recognition method based on human anatomy on the computer device side.

[0167] In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the following steps:

[0168] Obtaining a palm vein image sequence of a user;

[0169] Performing anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence;

[0170] According to the anatomical key point coordinates, performing geometric correction processing on the ROI region of the palm vein image sequence to obtain a standardized ROI region sequence;

[0171] Performing multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features;

[0172] Performing cross-modal fusion processing on the vein texture features and the hemodynamic features to obtain palm vein fusion features.

[0173] In one embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the following steps:

[0174] obtain a palm vein image sequence of a user;

[0175] perform anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence;

[0176] perform geometric correction processing on a ROI region of the palm vein image sequence according to the anatomical key point coordinates to obtain a standardized ROI region sequence;

[0177] perform multi-modal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features;

[0178] perform cross-modal fusion processing on the vein texture features and the hemodynamic features to obtain palm vein fusion features.

[0179] It should be noted that the functions or steps described above with respect to the computer readable storage medium or the computer device can correspond to the related description of the computer device in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0180] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in each embodiment of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0181] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0182] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A palm vein recognition method based on human anatomy, characterized in that: include: Obtaining a user's palm vein image sequence; performing anatomical key point positioning processing on the palm vein image sequence to obtain anatomical key point coordinates of the palm vein image sequence; performing geometric correction processing on the ROI region of the palm vein image sequence according to the coordinates of the anatomical key points to obtain a standardized ROI region sequence; Performing multimodal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features; The vein texture feature and the hemodynamic feature are cross-modally fused to obtain a palm vein fusion feature.

2. The palm vein recognition method according to claim 1, wherein: The performing anatomical key point positioning processing on the palm vein image sequence to obtain the anatomical key point coordinates of the palm vein image sequence includes: The palm vein image sequence is positioned using a pre-trained HRNet model based on the skeletal anatomical structure of the palm, and the metacarpal root points and the phalangeal-metacarpal connection points of the palm vein image sequence are obtained. The metacarpal root point coordinates and the phalangeal-metacarpal connection point coordinates constitute the anatomical key point coordinates.

3. The palm vein recognition method according to claim 1, wherein: The multimodal feature extraction process is performed on the standardized ROI region sequence to obtain vein texture features and hemodynamic features, including: Performing vein skeleton extraction processing on the standardized ROI region sequence to obtain a vein skeleton map; performing blood flow velocity extraction processing on the standardized ROI region sequence to obtain a blood flow velocity histogram; performing vein texture feature extraction processing on the vein skeleton image to obtain the vein texture feature; performing encoding processing on the blood flow velocity histogram to obtain a blood flow map; The blood flow map is subjected to hemodynamic feature extraction processing to obtain the hemodynamic feature.

4. The palm vein recognition method based on human anatomy as claimed in claim 3, characterized in that: After encoding the blood flow velocity histogram to obtain a blood flow map, the method further includes: Liveness verification is performed on the user based on the venous skeleton diagram and the blood flow diagram.

5. The palm vein recognition method based on human anatomy according to claim 1, characterized in that: The cross-modal fusion processing of the vein texture feature and the hemodynamic feature to obtain the palm vein fusion feature includes: The vein texture feature and the hemodynamic feature are weightedly fused through a cross-modal attention mechanism to obtain the palm vein fusion feature.

6. The palm vein recognition method based on human anatomy according to claim 1, characterized in that: After performing cross-modal fusion processing on the vein texture feature and the hemodynamic feature to obtain the palm vein fusion feature, the method further includes: The legitimacy of the user is verified based on the cosine distance between the palm vein fusion feature and the pre-stored legal palm vein feature.

7. The method for palm vein recognition based on human anatomy according to any one of claims 1 to 6, wherein obtaining a sequence of palm vein images of a user comprises: capturing a sequence of original palm vein images of the user; Dynamic denoising is performed on the original palm vein image sequence to obtain the palm vein image sequence.

8. A palm vein recognition device based on human anatomy, characterized in that: include: An image acquisition module, used to acquire a user's palm vein image sequence; a key point positioning module, configured to perform anatomical key point positioning processing on the palm vein image sequence to obtain coordinates of the anatomical key points of the palm vein image sequence; An ROI correction module is used to perform geometric correction processing on the ROI region of the palm vein image sequence according to the coordinates of the anatomical key points to obtain a standardized ROI region sequence; A feature extraction module is used to perform multimodal feature extraction processing on the standardized ROI region sequence to obtain vein texture features and hemodynamic features; The feature fusion module is used to perform cross-modal fusion processing on the vein texture feature and the hemodynamic feature to obtain a palm vein fusion feature.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the palm vein recognition method based on human anatomy are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the palm vein recognition method based on human anatomy are implemented.

Images