Binocular finger vein identity recognition method and device

Abstract

The invention discloses a binocular finger vein identity recognition method and device. The method comprises the following steps: calibrating parameters of left and right cameras; two-dimensional left and right finger vein texture images are obtained through calculation; performing three-dimensional correction on the vein texture images of the left and right fingers; obtaining finger vein texture point cloud data with a three-dimensional space topological structure; performing noise reduction processing on the finger vein texture point cloud data; constructing a cross-modal finger vein identity information representation model with two-dimensional image and three-dimensional point cloud cooperation; calculating to obtain a fusion feature corresponding to the maximum correlation and a joint representation vector of binocular finger vein feature attributes; and constructing a finger vein feature database with identity identification attributes. According to the matching relation between the binocular infrared finger vein texture images, the convolutional neural network model is established, the two-dimensional finger vein texture image and the three-dimensional finger vein texture point cloud are converted into the corresponding feature vectors, the joint representation of the cross-modal finger vein feature information is constructed, and the identification capability of identity recognition is improved.

Description

Technical Field

[0001] The invention relates to a binocular finger vein identity recognition method and also to a device for realizing the method, belonging to the technical field of biometric feature recognition. Background Art

[0002] The importance of information security in today's society has become increasingly prominent with the rapid development of computer and information technology, especially in key areas such as finance, security and online transactions. As an important means of protecting information and property security, identity recognition and authentication has changed from traditional methods based on passwords or certificates to biometric recognition technology that uses inherent human features such as face and fingerprints. These biometric features have been widely used in daily life due to their uniqueness, immutability and portability.

[0003] As an emerging biometric identification method, finger vein recognition technology has received extensive attention in research and application in recent years. This technology collects near-infrared images of finger veins inside the finger, extracts vein features from them, and performs pattern matching with the existing feature library to achieve identity recognition. Finger vein information is protected by human skin tissue and is not easily damaged or contaminated by external factors. It is also difficult to steal vein information, so it has an inherent anti-counterfeiting advantage. Compared with biometric identification technologies such as fingerprints and faces, finger vein recognition technology has a higher security level and anti-counterfeiting ability, and has become a research hotspot in the field of biometric identification.

[0004] However, despite the many advantages of finger vein recognition technology, it also faces some challenges. At present, most finger vein recognition research is based on two-dimensional vein images, which ignores the information loss problem that may be caused by the projection of the actual three-dimensional finger vein information of the human body in the plane imaging. This information loss limits the accuracy of finger vein recognition technology in large-scale user identification. In addition, the existing recognition methods based on two-dimensional finger vein texture images are easily affected by factors such as changes in ambient lighting, changes in finger placement, and finger disturbances, resulting in high rejection rates and false recognition rates of finger vein recognition systems. These methods have strict requirements on the placement of the user's fingers and lack environmental adaptability. Therefore, in order to promote and apply finger vein recognition technology, it is particularly important to construct more robust finger vein feature representation and identity recognition methods. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide a method for binocular finger vein identity recognition.

[0006] Another technical problem to be solved by the present invention is to provide a binocular finger vein identity recognition device.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a method for binocular finger vein identity recognition is provided, comprising the following steps:

[0009] S1: Use the plane chessboard calibration method to set the positions of the left and right cameras and calibrate the parameters of the left and right cameras;

[0010] S2: According to the calibration parameters, the two-dimensional left and right finger vein texture images are calculated;

[0011] S3: According to the calibrated parameters, the left and right finger vein texture images are stereo-corrected, and the left and right finger vein texture images are transformed to the same plane whose optical axes are parallel to each other;

[0012] S4: According to the template matching method, a corresponding matching relationship between the left and right finger vein texture images is constructed, and the finger vein texture images are three-dimensionally reconstructed to obtain finger vein texture point cloud data with a three-dimensional spatial topological structure;

[0013] S5: performing noise reduction processing on the finger vein texture point cloud data;

[0014] S6: Based on the left and right finger vein texture information and finger vein texture point cloud data, a cross-modal finger vein identity information representation model with 2D image and 3D point cloud collaboration is constructed;

[0015] S7: Using a deep convolutional neural network as the backbone network, two 2D left and right finger vein texture images of equal size at the same time are combined to form a multi-channel input finger vein texture image;

[0016] S8: according to the multi-channel input finger vein texture image, obtain the feature vector of the corresponding two-dimensional finger vein texture image and the feature vector of the corresponding three-dimensional finger vein point cloud data through a convolutional neural network;

[0017] S9: by using the canonical correlation analysis method, the correlation between the feature vector of the two-dimensional finger vein texture image and the feature vector of the corresponding three-dimensional finger vein point cloud data is calculated to obtain the fusion feature corresponding to the maximum correlation;

[0018] S10: concatenating the two-dimensional finger vein texture feature corresponding to the maximum correlation and the three-dimensional finger vein point cloud data feature corresponding to the maximum correlation to obtain a joint representation vector of binocular finger vein feature attributes;

[0019] S11: Based on the joint representation vector of binocular finger vein feature attributes obtained from different individuals, a finger vein feature database with identity recognition attributes is constructed.

[0020] Preferably, the parameters include: the intrinsic parameters K of the left camera l ; Distortion coefficient D of the left camera l; Intrinsic parameters K of the right camera r ; Distortion coefficient D of the right camera r ; Rotation transformation R between the left and right cameras; Translation transformation T between the left and right cameras; Relative position relationship of the left and right cameras [R, T];

[0021] The intrinsic parameters of the camera include: the coordinates of the principal point of the finger vein texture image and the focal length between the left and right cameras.

[0022] Preferably, the method for calculating and obtaining the two-dimensional left and right finger vein texture information comprises the following sub-steps:

[0023] S21: collecting a left finger vein texture image acquired by a left camera and a right finger vein texture image acquired by a right camera;

[0024] S22: performing distortion correction on the left finger vein texture image acquired by the left camera and the right finger vein texture image acquired by the right camera by using the calibrated intrinsic parameters of the left camera, the distortion coefficient of the left camera, the intrinsic parameters of the right camera, and the distortion coefficient of the right camera;

[0025] S23: preprocessing the corrected left finger vein texture image and the right finger vein texture image by using a nonlinear median filtering method;

[0026] S24: extracting binary texture information from the preprocessed left finger vein texture image and the right finger vein texture image by using a local dynamic threshold segmentation method;

[0027] S25: Obtain the two-dimensional left finger vein texture image L through the skeleton extraction algorithm v (x l ,y l ) and the two-dimensional right finger vein texture image R v (x r ,y r );

[0028] Among them, (x l ,y l ) is the two-dimensional pixel coordinate of the left finger vein texture in the left camera image; (x r ,y r ) is the two-dimensional pixel coordinate of the right finger vein texture in the right camera image.

[0029] Preferably, the stereoscopic correction method comprises the following sub-steps:

[0030] S31: calculating the parallax of the left and right finger vein texture images at the same horizontal line;

[0031] S32: Calculate the three-dimensional space coordinates P corresponding to the left and right finger vein texture points by triangulation method i(X i ,Y i ,Z i );

[0032] S33: transforming the left and right finger vein texture images to the same plane whose optical axes are parallel to each other according to the three-dimensional space coordinates corresponding to the left and right finger vein texture points.

[0033] Preferably, the formula for calculating the parallax of the left and right finger vein texture images at the same horizontal line is:

[0034] d i =x il -x ir

[0035] Among them, x il is the horizontal coordinate value in the left finger vein texture image; x ir is the horizontal coordinate value in the finger vein texture image on the right.

[0036] Preferably, the calculation formula of each coordinate in the space coordinate is:

[0037]

[0038] Among them, T 0 is the baseline distance between the left and right cameras, and its value is the modulus of the calibrated translation exchange between the left and right cameras; (u 0 ,v 0 ) are the principal point coordinates of the left and right finger vein texture images; f is the focal length between the left and right cameras.

[0039] Preferably, the method for performing noise reduction processing on the finger vein texture point cloud data is:

[0040] The average Euclidean distance between each finger vein texture point and its k neighboring points is calculated, and the corresponding average distance and variance value are statistically obtained from a single finger vein texture point cloud data, and the pseudo finger vein texture point cloud judgment threshold is set accordingly;

[0041] If the average distance of a finger vein texture point is greater than the pseudo finger vein texture point cloud judgment threshold, the point is defined as an outlier and deleted from the finger vein texture point cloud data; if the average distance of a finger vein texture point is less than or equal to the pseudo finger vein texture point cloud judgment threshold, the point is retained.

[0042] Preferably, the expression of the model for representing the cross-modal finger vein identity information is:

[0043] T vein =Ω{L v (x l ,y l ),Rv (x r ,y r ),P fv}

[0044] Among them, P fv is the finger vein texture point cloud data; Ω{g} is the cross-modal finger vein identity information expression function.

[0045] Preferably, the maximum correlation is calculated as follows:

[0046]

[0047]

[0048] Where M is the feature vector of the two-dimensional finger vein texture image; N is the feature vector of the three-dimensional finger vein point cloud data; M * is the two-dimensional finger vein texture feature corresponding to the maximum correlation; N * is the 3D finger vein point cloud data feature corresponding to the maximum correlation; S mm is the autocovariance matrix of the eigenvector of the two-dimensional finger vein texture image; S nn is the autocovariance matrix of the eigenvector of the three-dimensional finger vein point cloud data; S mn is the cross-covariance matrix between the feature vector of the two-dimensional finger vein texture image and the feature vector of the three-dimensional finger vein point cloud data; W m is the transfer matrix of the two-dimensional finger vein texture image; W n is the transfer matrix of 3D finger vein point cloud data; W m The transposed matrix of W n The transposed matrix of .

[0049] Preferably, the expression of the joint representation vector of the binocular finger vein feature attributes is:

[0050] Z * =(M * ,N * ).

[0051] According to a second aspect of an embodiment of the present invention, there is provided a device for binocular finger vein identity recognition, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the above method.

[0052] Compared with the prior art, the present invention realizes the three-dimensional spatial representation of finger vein texture information by establishing a matching relationship between binocular infrared finger vein texture images. On this basis, a convolutional neural network model is introduced to convert the two-dimensional finger vein texture image and the three-dimensional finger vein texture point cloud into corresponding feature vectors. Then, the canonical correlation analysis method is used to align the finger vein feature vectors under different modalities, and a joint representation of cross-modal finger vein feature information is constructed to improve its recognition ability and authentication effect in the identity recognition process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flow chart of a method for binocular finger vein identification provided by an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of the camera placement in an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of a binocular finger vein identity recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] First embodiment

[0058] like Figure 1 As shown, a method for binocular finger vein identification provided by the first embodiment of the present invention includes:

[0059] S1: Use the plane chessboard calibration method to set the positions of the left and right cameras and calibrate the parameters of the left and right cameras.

[0060] It should be noted that Planar Chessboard Calibration is a camera calibration method proposed by Zhang Zhengyou. This method mainly calibrates the camera by shooting a planar image containing a chessboard pattern, that is, determining the camera's internal parameters (such as focal length, principal point position, etc.) and external parameters (such as the camera's position and posture relative to the chessboard).

[0061] The specific steps of the plane chessboard calibration method are as follows:

[0062] Prepare a checkerboard pattern: Make or print a checkerboard pattern, usually a black and white grid pattern with each grid of known dimensions.

[0063] Take images: Place the chessboard in different positions and orientations and take multiple images containing the chessboard. These images will be used for subsequent calculations.

[0064] Feature point detection: In each image, a computer vision algorithm is used to detect the corners of the checkerboard. The positions of these corners in the image are known.

[0065] Build a model: Associate the detected corner points with the actual physical positions of the chessboard and build a mathematical model. This model will be used to calculate the camera's intrinsic and extrinsic parameters.

[0066] Parameter optimization: The model parameters are adjusted through an optimization algorithm (such as the least squares method) to minimize the error between the corner point positions predicted by the model and the actual detected corner point positions.

[0067] Calibration results: Finally, the intrinsic and extrinsic parameters of the camera are obtained. These parameters can be used to correct the images taken by the camera or for subsequent computer vision tasks such as 3D reconstruction.

[0068] Through the above-mentioned plane chessboard calibration method, the accuracy of image processing and analysis can be improved, laying the foundation for subsequent image recognition, object tracking and other tasks.

[0069] Figure 2 : is a schematic diagram of the camera arrangement in an embodiment of the present invention. The figure shows the relative position relationship between the two cameras and the finger in the binocular system, which is crucial for understanding the entire identity recognition process.

[0070] exist Figure 2 In the figure, two rectangles can be seen, marked as "1" and "2", which represent the left and right cameras in the system. The two cameras are placed on both sides of the finger to capture the finger vein texture images from different angles. This dual-camera layout is the basis for stereoscopic vision and three-dimensional information acquisition. The oval in the center of the figure, marked as "3", represents the finger. The finger is a key object in the identity recognition process, and the vein texture inside it will be captured by the two cameras and used for identity authentication. The position of the finger is in the common field of view of the two cameras, ensuring that images can be acquired from two different perspectives. In addition, the placement angles and positions of the two cameras are carefully designed to achieve the best image capture effect. This layout allows the system to reconstruct the three-dimensional finger vein structure of the finger using stereoscopic vision technology by comparing the images on both sides, thereby improving the accuracy and robustness of identity recognition.

[0071] Figure 2 The structural relationship shown is the key to achieving high-precision identity recognition, which ensures that the system can effectively capture and process finger vein information.

[0072] In one embodiment of the present invention, the parameters of the left and right cameras include: the intrinsic parameters K of the left camera l , the distortion coefficient D of the left camera l , the intrinsic parameters K of the right camera r, the distortion coefficient D of the right camera r , the rotation transformation R between the left and right cameras, the translation transformation T between the left and right cameras, and the relative posture relationship between the left and right cameras [R,T].

[0073] The intrinsic parameters of the camera include: the coordinates of the principal point of the finger vein texture image and the focal length between the left and right cameras.

[0074] S2: According to the calibrated parameters, the two-dimensional left and right finger vein texture images are calculated.

[0075] The method for calculating and obtaining two-dimensional left and right finger vein texture information comprises the following sub-steps:

[0076] S21: Acquire a left-side finger vein texture image acquired by a left camera and a right-side finger vein texture image acquired by a right camera.

[0077] S22: performing distortion correction on the left finger vein texture image acquired by the left camera and the right finger vein texture image acquired by the right camera using the calibrated intrinsic parameters of the left camera, the distortion coefficient of the left camera, the intrinsic parameters of the right camera, and the distortion coefficient of the right camera.

[0078] S23: Preprocessing the corrected left finger vein texture image and right finger vein texture image by nonlinear median filtering method.

[0079] The role of the preprocessing is to reduce image noise while ensuring vein texture information.

[0080] S24: extracting the binary texture information from the preprocessed left finger vein texture image and the right finger vein texture image through a local dynamic threshold segmentation method.

[0081] S25: Obtain the two-dimensional left finger vein texture image L through the skeleton extraction algorithm v (x l ,y l ) and the two-dimensional right finger vein texture image R v (x r ,y r ).

[0082] Among them, (x l ,y l ) is the 2D pixel coordinate of the left finger vein texture in the left camera image. (x r ,y r ) is the two-dimensional pixel coordinate of the right finger vein texture in the right camera image.

[0083] S3: According to the calibrated parameters, the left and right finger vein texture images are stereo-rectified, and the left and right finger vein texture images are transformed to the same plane whose optical axes are parallel to each other.

[0084] The stereoscopic correction method comprises the following sub-steps:

[0085] S31: Calculate the parallax of the left and right finger vein texture images on the same horizontal line.

[0086] The formula for calculating the parallax of the left and right finger vein texture images at the same horizontal line is:

[0087] d i =x il -x ir

[0088] Among them, x il is the horizontal coordinate value in the left finger vein texture image. ir is the horizontal coordinate value in the finger vein texture image on the right.

[0089] S32: Calculate the three-dimensional space coordinates P corresponding to the left and right finger vein texture points by triangulation method i (X i ,Y i ,Z i ).

[0090] Among them, the calculation formula of each coordinate in the space coordinate is:

[0091]

[0092] Among them, T 0 is the baseline distance between the left and right cameras, and its value is the modulus of the calibrated translation exchange between the left and right cameras. (u 0 ,v 0 ) is the coordinate of the principal point of the left and right finger vein texture images. f is the focal length between the left and right cameras.

[0093] S33: transforming the left and right finger vein texture images to the same plane whose optical axes are parallel to each other according to the three-dimensional space coordinates corresponding to the left and right finger vein texture points.

[0094] S4: According to the template matching method, a corresponding matching relationship between the left and right finger vein texture images is constructed, and three-dimensional reconstruction of the finger vein texture images is performed to obtain finger vein texture point cloud data with a three-dimensional spatial topological structure.

[0095] S5: Perform noise reduction processing on the finger vein texture point cloud data.

[0096] The average Euclidean distance between each finger vein texture point and its k neighboring points is calculated, and the corresponding average distance and variance value are statistically obtained from a single finger vein texture point cloud data, and the pseudo finger vein texture point cloud judgment threshold is set accordingly.

[0097] If the average distance of a finger vein texture point is greater than the pseudo finger vein texture point cloud judgment threshold, the point is defined as an outlier and deleted from the finger vein texture point cloud data. If the average distance of a finger vein texture point is less than or equal to the pseudo finger vein texture point cloud judgment threshold, the point is retained.

[0098] Taking into account that the binary finger vein texture image is prone to vein burrs or broken pseudo veins, which in turn leads to the appearance of noise or pseudo vein point clouds in the finger vein texture point cloud data obtained by three-dimensional reconstruction, the present invention constructs a method for three-dimensional filtering of finger vein texture point clouds based on a sliding window based on the spatial continuity assumption of an ideal finger vein texture.

[0099] S6: Based on the left and right finger vein texture information and finger vein texture point cloud data, a cross-modal finger vein identity information representation model with 2D image and 3D point cloud collaboration is constructed. vein .

[0100] The expression of the model for representing the cross-modal finger vein identity information is:

[0101] T vein =Ω{L v (x l ,y l ),R v (x r ,y r ),P fv}

[0102] Among them, P fv is the finger vein texture point cloud data. Ω{g} is the cross-modal finger vein identity information expression function.

[0103] S7: Using the deep convolutional neural network (Residual Network, ResNet) as the backbone network, two two-dimensional left and right finger vein texture images of equal size at the same time are combined to form a multi-channel input finger vein texture image.

[0104] S8: Based on the multi-channel input finger vein texture image, the feature vector of the corresponding two-dimensional finger vein texture image and the feature vector of the corresponding three-dimensional finger vein point cloud data are obtained through a convolutional neural network (CNN).

[0105] S9: By using the canonical correlation analysis (CCA) method, the correlation between the feature vector of the two-dimensional finger vein texture image and the feature vector of the corresponding three-dimensional finger vein point cloud data is calculated to obtain the fusion feature corresponding to the maximum correlation.

[0106] The maximum correlation is calculated as follows:

[0107]

[0108] Where M is the feature vector of the two-dimensional finger vein texture image. N is the feature vector of the three-dimensional finger vein point cloud data. * is the two-dimensional finger vein texture feature corresponding to the maximum correlation. N * is the 3D finger vein point cloud data feature corresponding to the maximum correlation. mm is the autocovariance matrix of the eigenvector of the two-dimensional finger vein texture image. nn is the autocovariance matrix of the eigenvector of the three-dimensional finger vein point cloud data. mn is the cross-covariance matrix between the feature vector of the two-dimensional finger vein texture image and the feature vector of the three-dimensional finger vein point cloud data. m is the transfer matrix of the two-dimensional finger vein texture image. n is the transfer matrix of 3D finger vein point cloud data. W m The transposed matrix of . W n The transposed matrix of .

[0109] Based on the properties of two-dimensional finger vein texture images, this model has the characteristics of rotation invariance of three-dimensional finger vein texture images, and has better adaptability to different finger postures and orientations during the recognition process. However, due to the large distribution difference between the left and right finger vein texture information and the finger vein texture point cloud data in the physical space scale in this model, a reliable alignment method is required in the specific recognition and matching to map the finger vein feature information in different dimensions to the same state vector.

[0110] S10: The two-dimensional finger vein texture feature corresponding to the maximum correlation and the three-dimensional finger vein point cloud data feature corresponding to the maximum correlation are connected in series to obtain a joint representation vector of binocular finger vein feature attributes.

[0111] The expression of the joint representation vector of the binocular finger vein feature attributes is:

[0112] Z * =(M * ,N * )

[0113] S11: Based on the joint representation vector of binocular finger vein feature attributes obtained from different individuals, a finger vein feature database with identity recognition attributes is constructed.

[0114] When an individual is identified and authenticated, the method provided by the embodiment of the present invention can obtain the corresponding multimodal joint representation attribute feature vector and match it with the prior finger vein feature database to achieve accurate identification of the target identity.

[0115] Second embodiment

[0116] Based on the above method, the second embodiment of the present invention further provides a device for binocular finger vein identification. Figure 3 As shown, the device 400 includes one or more processors 41 and a memory 42. The memory 42 is used to store one or more programs, and when the one or more programs are executed by the one or more processors 41, the method in the above embodiment is implemented.

[0117] Wherein, the processor is used to control the overall operation of the device to complete all or part of the steps of the above method. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support the operation of the device, and these data may include, for example, instructions for any application or method for operating on the device, and application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, etc.

[0118] In an exemplary embodiment, the device can be implemented by a computer or a microprocessor entity, or by a product with a certain function, for executing the above method and achieving the same technical effect as the above method. Specifically, the computer can be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0119] In another exemplary embodiment, the present invention further provides a computer-readable storage medium including program instructions, which, when executed by a processor, implements the steps of the method in any of the above embodiments. For example, the computer-readable storage medium may be the above-mentioned memory including program instructions, which may be executed by a processor to complete the above-mentioned method and achieve the same technical effect as the above-mentioned method.

[0120] It should be noted that the above embodiments are only examples, and the technical solutions of the various embodiments can be combined, all within the protection scope of the present invention.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0122] The above is a detailed description of the binocular finger vein identification method and device provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims

1. A method for binocular finger vein identification, characterized in that include: S1: Use the plane chessboard calibration method to set the positions of the left and right cameras and calibrate the parameters of the left and right cameras; S2: According to the calibration parameters, the two-dimensional left and right finger vein texture images are calculated; S3: According to the calibrated parameters, the left and right finger vein texture images are stereo-corrected, and the left and right finger vein texture images are transformed to the same plane whose optical axes are parallel to each other; S4: According to the template matching method, a corresponding matching relationship between the left and right finger vein texture images is constructed, and the finger vein texture images are three-dimensionally reconstructed to obtain finger vein texture point cloud data with a three-dimensional spatial topological structure; S5: performing noise reduction processing on the finger vein texture point cloud data; S6: Based on the left and right finger vein texture information and finger vein texture point cloud data, a cross-modal finger vein identity information representation model with 2D image and 3D point cloud collaboration is constructed; S7: Using a deep convolutional neural network as the backbone network, two 2D left and right finger vein texture images of equal size at the same time are combined to form a multi-channel input finger vein texture image; S8: according to the multi-channel input finger vein texture image, obtain the feature vector of the corresponding two-dimensional finger vein texture image and the feature vector of the corresponding three-dimensional finger vein point cloud data through a convolutional neural network; S9: by using the canonical correlation analysis method, the correlation between the feature vector of the two-dimensional finger vein texture image and the feature vector of the corresponding three-dimensional finger vein point cloud data is calculated to obtain the fusion feature corresponding to the maximum correlation; S10: concatenating the two-dimensional finger vein texture feature corresponding to the maximum correlation and the three-dimensional finger vein point cloud data feature corresponding to the maximum correlation to obtain a joint representation vector of binocular finger vein feature attributes; S11: Based on the joint representation vector of binocular finger vein feature attributes obtained from different individuals, a finger vein feature database with identity recognition attributes is constructed.

2. The method according to claim 1, characterized in that The parameters include: the intrinsic parameters K of the left camera l ; Distortion coefficient D of the left camera l ; Intrinsic parameters K of the right camera r ; Distortion coefficient D of the right camera r ; Rotation transformation R between the left and right cameras; Translation transformation T between the left and right cameras; Relative position relationship of the left and right cameras [R, T]; The intrinsic parameters of the camera include: the coordinates of the principal point of the finger vein texture image and the focal length between the left and right cameras.

3. The method according to claim 1, characterized in that The method for calculating and obtaining two-dimensional left and right finger vein texture information includes the following sub-steps: S21: collecting a left finger vein texture image acquired by a left camera and a right finger vein texture image acquired by a right camera; S22: performing distortion correction on the left finger vein texture image acquired by the left camera and the right finger vein texture image acquired by the right camera by using the calibrated intrinsic parameters of the left camera, the distortion coefficient of the left camera, the intrinsic parameters of the right camera, and the distortion coefficient of the right camera; S23: preprocessing the corrected left finger vein texture image and the right finger vein texture image by using a nonlinear median filtering method; S24: extracting binary texture information from the preprocessed left finger vein texture image and the right finger vein texture image by using a local dynamic threshold segmentation method; S25: Obtain the two-dimensional left finger vein texture image L through the skeleton extraction algorithm v (x l ,y l ) and the two-dimensional right finger vein texture image R v (x r ,y r ); Among them, (x l ,y l ) is the two-dimensional pixel coordinate of the left finger vein texture in the left camera image; (x r ,y r ) is the two-dimensional pixel coordinate of the right finger vein texture in the right camera image.

4. The method according to claim 1, characterized in that The stereoscopic correction method comprises the following sub-steps: S31: calculating the parallax of the left and right finger vein texture images at the same horizontal line; S32: Calculate the three-dimensional space coordinates P corresponding to the left and right finger vein texture points by triangulation method i (X i ,Y i ,Z i ); S33: transforming the left and right finger vein texture images to the same plane whose optical axes are parallel to each other according to the three-dimensional space coordinates corresponding to the left and right finger vein texture points.

5. The method according to claim 4, characterized in that The formula for calculating the parallax of the left and right finger vein texture images at the same horizontal line is: d i =x il -x ir Among them, x il is the horizontal coordinate value in the left finger vein texture image; x ir is the horizontal coordinate value in the finger vein texture image on the right.

6. The method according to claim 4, characterized in that The calculation formula of each coordinate in the space coordinate is: Among them, T0 is the baseline distance between the left and right cameras, and its value is the modulus of the calibrated translation exchange between the left and right cameras; (u0, v0) is the coordinate of the principal point of the left and right finger vein texture images; f is the focal length between the left and right cameras.

7. The method according to claim 1, characterized in that The method for performing noise reduction processing on finger vein texture point cloud data comprises the following sub-steps: The average Euclidean distance between each finger vein texture point and its k neighboring points is calculated, and the corresponding average distance and variance value are statistically obtained from a single finger vein texture point cloud data, and the pseudo finger vein texture point cloud judgment threshold is set accordingly; If the average distance of a finger vein texture point is greater than the pseudo finger vein texture point cloud judgment threshold, the point is defined as an outlier and deleted from the finger vein texture point cloud data; if the average distance of a finger vein texture point is less than or equal to the pseudo finger vein texture point cloud judgment threshold, the point is retained.

8. The method according to claim 1, characterized in that The expression of the model for representing the cross-modal finger vein identity information is: T vein =Ω{L v (x l ,y l ),R v (x r ,y r ),P fv } Among them, P fv is the finger vein texture point cloud data; Ω{g} is the cross-modal finger vein identity information expression function.

9. The method according to claim 1, characterized in that The maximum correlation is calculated as follows: Where M is the feature vector of the two-dimensional finger vein texture image; N is the feature vector of the three-dimensional finger vein point cloud data; M * is the two-dimensional finger vein texture feature corresponding to the maximum correlation; N * is the 3D finger vein point cloud data feature corresponding to the maximum correlation; S mm is the autocovariance matrix of the eigenvector of the two-dimensional finger vein texture image; S nn is the autocovariance matrix of the eigenvector of the three-dimensional finger vein point cloud data; S mn is the cross-covariance matrix between the feature vector of the two-dimensional finger vein texture image and the feature vector of the three-dimensional finger vein point cloud data; W m is the transfer matrix of the two-dimensional finger vein texture image; W n is the transfer matrix of 3D finger vein point cloud data; W m The transposed matrix of W n The transposed matrix of .

10. A binocular finger vein identification device, characterized in that It comprises a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the method described in any one of claims 1 to 9.

Images