User identification method and device in combination with biological characteristics

Abstract

The invention relates to a user identification method and device combined with biological characteristics, and relates to the field of user identification data processing, and the method comprises the steps: identifying a palmprint angle and an image angle, analyzing the similarity between the palmprint angle and a plurality of palmprint angles of a plurality of historical palmprint angle sequences, and determining the similarity between the palmprint angles and a plurality of historical palmprint angle sequences; analyzing a plurality of image angle similarities between the image angle and the plurality of historical image angle sequences; and performing analysis to obtain a plurality of amount fluctuation similarities, performing compensation adjustment on a plurality of palmprint probabilities and a plurality of image probabilities in combination with a plurality of palmprint angle similarities and a plurality of image angle similarities to obtain a plurality of user probabilities, and outputting an undetermined user with the maximum user probability as a user identification result. The technical problems that a traditional user identification method is low in adaptability in a complex environment, the user identity cannot be effectively identified, and identification accuracy and reliability are insufficient can be solved; through multi-modal information fusion, the precision, accuracy and reliability of user identity authentication in a complex environment can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of user identification data processing, and in particular to a user identification method and device combining biological features. Background Art

[0002] With the widespread application of smart devices and self-service systems, the application of user identity authentication technology in payment, security, smart device control and other fields has become increasingly important. In these application scenarios, the accuracy and reliability of identity authentication are directly related to user experience and system security.

[0003] Traditional user authentication methods are mainly based on single biometric recognition, such as palm print recognition, fingerprint recognition, and face recognition. Although these methods can provide good recognition results under specific environments and conditions, in complex environments, background noise, messy visual information, or irregular physical environments (such as crowded places, dynamic backgrounds) may interfere with the recognition process, resulting in misidentification or missed recognition problems, which greatly limits the accuracy and reliability of recognition. Summary of the invention

[0004] The present invention aims to solve the technical problems that the traditional user identification method based on a single biometric feature has low adaptability in complex environments, cannot effectively identify the user identity, and has insufficient identification accuracy and reliability. A user identification method and device combining biometric features are provided to solve the problem.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: In a first aspect, the present invention provides a user identification method combined with biometric features, comprising: collecting palm print information of a user through a palm print collection device, performing user palm print recognition, and obtaining multiple pending users and multiple palm print probabilities; when the maximum palm print probability is not greater than a preset probability threshold, collecting image information of the user through an image collection device, performing user image recognition, and obtaining multiple image probabilities; identifying the palm print angle of the palm print information and the image angle of the image information, obtaining multiple historical palm print angle sequences and multiple historical image angle sequences of the multiple pending users, analyzing the palm print angle and multiple palm print angle similarities of the multiple historical palm print angle sequences, and analyzing the image angle and multiple image angle similarities of the multiple historical image angle sequences; obtaining the current charge amount, and obtaining multiple historical amount sequences of the multiple pending users, respectively analyzing to obtain multiple amount fluctuation similarities, combining the multiple palm print angle similarities and multiple image angle similarities, compensating and adjusting the multiple palm print probabilities and multiple image probabilities to obtain multiple user probabilities, and outputting the pending user with the largest user probability as the user identification result.

[0006] In a second aspect, the present invention provides a user authentication device in combination with biometric features, including: a user palm print recognition module, used to collect the user's palm print information through a palm print collection device, perform user palm print recognition, and obtain multiple pending users and multiple palm print probabilities; a user image recognition module, used to collect the user's image information through an image collection device when the maximum palm print probability is not greater than a preset probability threshold, perform user image recognition, and obtain multiple image probabilities; an angle similarity analysis module, used to identify the palm print angle of the palm print information and the image angle of the image information, and obtain multiple historical palm print angle sequences and multiple image probabilities of the multiple pending users. a historical image angle sequence, analyzing the palm print angle and multiple palm print angle similarities of the multiple historical palm print angle sequences, and analyzing the image angle and multiple image angle similarities of the multiple historical image angle sequences; a user identification result output module, used to obtain the current charging amount, and obtain multiple historical amount sequences of the multiple pending users, respectively analyze to obtain multiple amount fluctuation similarities, combine the multiple palm print angle similarities and the multiple image angle similarities, compensate and adjust the multiple palm print probabilities and the multiple image probabilities, obtain multiple user probabilities, and output the pending user with the largest user probability as the user identification result.

[0007] The beneficial effects of the present invention are as follows: by collecting the palm print information of the user to perform the palm print recognition of the user, a plurality of pending users and a plurality of palm print probabilities are obtained; then when the maximum palm print probability is not greater than a preset probability threshold, the image information of the user is collected to perform the user image recognition to obtain a plurality of image probabilities; then the palm print angle of the palm print information and the image angle of the image information are identified to obtain a plurality of historical palm print angle sequences and a plurality of historical image angle sequences of the plurality of pending users; further the palm print angle and the plurality of palm print angle similarities of the plurality of historical palm print angle sequences are analyzed, and the plurality of image angle similarities of the image angle and the plurality of historical image angle sequences are analyzed. ; On the other hand, the current charging amount is obtained, and multiple historical amount sequences of the multiple pending users are obtained, and multiple amount fluctuation similarities are obtained by analysis respectively; finally, according to the multiple amount fluctuation similarities, multiple palm print angle similarities and multiple image angle similarities, the multiple palm print probabilities and multiple image probabilities are compensated and adjusted to obtain multiple user probabilities, and the pending user with the largest user probability is output as the user identification result; that is to say, through multimodal information fusion, the user's recognition probability can be compensated and optimized in multiple dimensions, avoiding the possible misidentification or missed identification of a single feature, thereby significantly improving the precision, accuracy and reliability of user identity authentication in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of a flow chart of a user identification method combined with biometric features provided by the present invention; Figure 2 A schematic diagram of the structure of a user authentication device combined with biometric features provided by the present invention.

[0009] In the accompanying drawings, the components represented by the reference numerals are described as follows: User palm print recognition module 01, user image recognition module 02, angle similarity analysis module 03, user identification result output module 04. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0011] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0012] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0013] Embodiment 1, as Figure 1 As shown, the embodiment of the present invention provides a user identification method combining biometric features, which specifically includes the following steps: S100: Collecting palm print information of a user through a palm print collection device, performing palm print recognition on the user, and obtaining a plurality of pending users and a plurality of palm print probabilities.

[0014] Furthermore, step S100 of the present invention further includes: S110: collecting palmprint information of a user through a palmprint collection device; S120: inputting the palmprint information into a pre-constructed palmprint recognition channel, identifying similarities with multiple sample palmprint information in the palmprint recognition channel, outputting multiple maximum palmprint similarities and corresponding users, and obtaining multiple pending users, wherein the palmprint recognition channel is obtained by training based on a sample palmprint information set and a sample palmprint similarity set; S130: outputting the palmprint similarities corresponding to the multiple pending users as multiple palmprint probabilities.

[0015] Specifically, during the data collection process, all user information (such as palm prints, facial images, etc.) is collected legally and with the user's explicit authorization. When the user uses the relevant service for the first time, the system will inform the user of the purpose, scope, and usage of the data collection through the user agreement or privacy policy. The user needs to explicitly agree to the data collection. At the same time, the user can withdraw the consent at any time, and the system will stop collecting and processing the user information. On the other hand, the collected biometric data (such as palm print images, facial images) will undergo strict security protection measures (such as encrypted storage and transmission) to ensure that user information is not abused, leaked or accessed by unauthorized third parties.

[0016] In payment scenarios of desktop toll machines such as convenience stores, using a palmprint collection device to collect the user's palmprint information as a form of identity authentication can improve the security and convenience of payment; first, when the user operates the toll machine, the user places the palm on the sensor area of ​​the palmprint collection device. Usually the device is equipped with optical, infrared or capacitive sensors that can capture the detailed features of the palmprint, and then the palmprint collection device is used to collect the user's palmprint information, including biometric information such as the direction of the palm prints.

[0017] Convolutional neural network (CNN) is a feedforward neural network that is particularly good at processing multi-dimensional data, such as two-dimensional images. Through the combination of multiple convolutional layers, pooling layers, fully connected layers and other modules, it automatically extracts the features of input data and performs tasks such as classification or regression on the data. A palmprint recognition channel is constructed based on a convolutional neural network, which includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer, wherein the input data of the input layer is palmprint information, and the output data of the output layer is palmprint similarity; the convolutional layer is used to extract local features, such as texture, shape, etc.; the pooling layer is used to reduce the data dimension and enhance the robustness of the model; the fully connected layer is used to convert local features into global features and perform classification.

[0018] Then, based on the historical palmprint detection log, a sample palmprint information set and a sample palmprint similarity set are collected, and the sample palmprint information (palm pattern direction, etc.) is used as input, and the sample palmprint similarity is used as output. The sample palmprint information set and the sample palmprint similarity set are used to supervise the palmprint recognition channel. First, during training, a set of palmprint image data is input each time. These data enter the network through the input layer of CNN. CNN automatically extracts local features of the image through the convolution layer and the pooling layer, and fuses these features through the fully connected layer; then, CNN outputs the similarity prediction value between the palmprints, and uses the loss function (such as mean square error) to measure the difference between the prediction result and the true label; then, by calculating the gradient of the loss function, the back propagation algorithm is used to feed back the error to each layer in the network, and the weight and bias of each layer will be adjusted according to the gradient descent algorithm to reduce the prediction error. Further, the optimization algorithm (such as Adam optimizer, SGD, etc.) is used to update the weight and bias of the network to gradually minimize the loss function. Iterative training is performed using sample data until the loss function converges, and the trained palmprint recognition channel is output.

[0019] Collect multiple sample palmprint information and embed them into the palmprint recognition channel, wherein the multiple sample palmprint information is the palmprint information registered by the user or the palmprint information shared by other payment platforms, and the sample palmprint information has a unique user identifier; then input the palmprint information into the palmprint recognition channel, perform similarity traversal identification with the multiple sample palmprint information in the palmprint recognition channel, and output the largest multiple (such as the first 10) palmprint similarities, wherein the output quantity can be set according to the actual scenario, such as when there is less interference, a relatively small number of palmprint similarities can be output, such as 5; when there is more interference, a relatively large number of palmprint similarities can be output, such as 10; and obtain the user identifier corresponding to the largest multiple palmprint similarities, and obtain multiple pending users (users with higher palmprint similarities). Then set the palmprint similarity as the palmprint probability, and output the multiple palmprint probabilities corresponding to the multiple pending users. Exemplarily, the multiple palmprint probabilities are 94%, 92%, 90%, 89%, and 85% respectively.

[0020] By embedding multiple sample palmprint information (including user registration information and other platform shared data) into the palmprint recognition channel, the similarity between real-time palmprint information and multiple sample information can be accurately identified. Based on the similarity, the system can output multiple most similar pending users and provide a basis for subsequent identity verification. This method can not only improve recognition accuracy, but also flexibly adjust the number of recognition results according to the interference situation in the actual scene, thereby enhancing the reliability and adaptability of the system.

[0021] S200: When the maximum palm print probability is not greater than a preset probability threshold, the image information of the user is collected by an image collection device, and user image recognition is performed to obtain a plurality of image probabilities.

[0022] Furthermore, step S200 of the present invention further includes: S210: Determine whether the maximum palm print probability is greater than a preset probability threshold. If so, output the palm print user corresponding to the maximum palm print probability as the user identification result; S220: If not, collect the image information of the user through the image acquisition device; S230: Input the image information into a pre-constructed image recognition channel, identify the similarity with the sample image information of the multiple pending users, and obtain multiple image similarities, wherein the image recognition channel is trained based on a sample image information set and a sample image similarity set; S240: Output the multiple image similarities as multiple image probabilities.

[0023] Specifically, a preset probability threshold is obtained. The preset probability threshold is used to measure the accuracy of recognition and can be set according to the recognition accuracy requirements (such as 95%). If the maximum palm print probability exceeds this threshold, it means that the current palm print recognition is accurate enough to effectively confirm the user's identity. Next, it is determined whether the maximum palm print probability is greater than the preset probability threshold. If the maximum palm print probability is greater than the preset probability threshold, it means that the current recognition result is credible, and the palm print user corresponding to the maximum palm print probability is output as the user identification result. If the maximum palm print probability is less than or equal to the preset probability threshold, the image information (face image) of the user is collected through an image acquisition device (such as a camera). At this time, the image information is used as a supplementary identity feature to improve the accuracy of recognition.

[0024] An image recognition channel is constructed based on a convolutional neural network, and the image recognition channel includes an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer, wherein the input data of the input layer is image information (face image), and the output data of the output layer is image similarity. Then, a sample image information set and a sample image similarity set are collected, and the sample image information is used as input and the sample image similarity is used as output. The image recognition channel is supervised and trained using the sample image information set and the sample image similarity set. First, the sample image information is used as input, the sample image similarity is used as an output label, and the image similarity value is used as a label to indicate the similarity between different images (usually calculated by a metric such as Euclidean distance or cosine similarity); then, the error between the network output and the true label (i.e., the actual similarity) is calculated by a loss function (such as a mean square error loss function or a cross entropy loss function), and the goal of the network is to minimize the loss function so that the image recognition result is as close as possible to the actual similarity value; then, the gradient of each layer is calculated by a back-propagation algorithm, the weights in the network are updated, and the performance of the network is optimized. An optimization algorithm such as stochastic gradient descent (SGD) or an Adam optimizer is usually used to update the parameters of the model. Finally, through multiple iterative training processes, the network weight parameters are continuously adjusted to optimize the model's image recognition ability, so that the network can recognize and distinguish the image features of different users until the loss function converges and a trained image recognition channel is obtained.

[0025] Further obtain sample image information of the multiple pending users (which can be obtained by data screening in the user information database); then combine the image information and the sample image information of the multiple pending users respectively and input them into the image recognition channel to output multiple image similarities; and set the image similarities as image probabilities to output multiple image probabilities. Through supervised learning, the collected sample images and similarity information are used as training data, and a convolutional neural network is used to extract image features and learn the similarity relationship between user identities. After multiple trainings, the network can efficiently identify the user identity based on the input image information, and combine it with the palm print recognition results to achieve accurate user identification.

[0026] S300: Identify the palm print angle of the palm print information and the image angle of the image information, obtain multiple historical palm print angle sequences and multiple historical image angle sequences of the multiple pending users, analyze the similarities between the palm print angle and multiple palm print angles of the multiple historical palm print angle sequences, and analyze the similarities between the image angle and multiple image angles of the multiple historical image angle sequences.

[0027] Furthermore, step S300 of the present invention further includes: S310: According to the user identification data in the historical time, a sample palmprint information set and a sample image information set are collected, and the palmprint angle and the image angle of each sample palmprint information and each sample image information are marked to obtain a sample palmprint angle set and a sample image angle set; S320: The sample palmprint information set and the sample palmprint angle set are used as supervised training data to train a palmprint angle recognition branch; S330: The sample image information set and the sample image angle set are used as supervised training data to train an image angle recognition branch; S340: The palmprint angle recognition branch and the image angle recognition branch are combined to obtain a biometric angle recognition channel; S350: The palmprint information and the image information are input into the palmprint angle recognition branch and the image angle recognition branch in the biometric angle recognition channel, and the palmprint angle and the image angle are obtained by recognition output.

[0028] Specifically, user identification data within a historical period (such as within the last month) is obtained, and a sample palm print information set and a sample image information set are collected based on the user identification data; then, the palm print angle and image angle of each sample palm print information and each sample image information are annotated, wherein the palm print angle refers to the angle of the user's palm in front of the collection device, and this angle affects the accuracy of palm print recognition. The purpose of angle annotation is to describe the user's palm print information in different postures. The palm angle can be extracted through image analysis methods, and the angle of each palm print image can be annotated. Exemplarily, the palm print angle may include the horizontal angle and pitch angle of the palm print are both 0 degrees, that is, the user's palm is at a horizontal angle and is perpendicular to the user's body plane, and may also include the horizontal angle relative to the user's direction deviating 15 degrees to the left, the pitch angle deviating 15 degrees upward on the left, and the right side deviating 15 degrees downward, or the horizontal angle deviating 15 degrees to the right, etc.

[0029] Similarly, the user's facial image will also be affected by the angle when it is collected. The image angle refers to the angle of the facial image in front of the camera, which may affect the effect of facial recognition. Through image processing technology or deep learning models (such as face recognition algorithms), facial features in the image are extracted and the angle of the image is calculated. For example, by identifying the relative positions of the eyes, nose and mouth, the angle between the device is calculated; a set of sample palm print angles and a set of sample image angles are obtained. Exemplarily, the image angle of the facial image can be 0 degrees, that is, facing the camera, or it can be 15 degrees rotated to the left or 15 degrees rotated to the right along the Y axis relative to the user's direction, and the Y axis is the coordinate axis perpendicular to the horizontal plane.

[0030] A palmprint angle recognition branch is constructed based on a convolutional neural network. The input of the palmprint angle recognition branch is palmprint information, and the output data is palmprint angle. Then, the sample palmprint information is used as input, the sample palmprint angle is used as supervision, and the sample palmprint information set and the sample palmprint angle set are used as supervised training data to perform supervised training on the palmprint angle recognition branch. First, a loss function suitable for the regression task is selected, such as the mean square error (MSE) loss function, which measures the difference between the predicted angle and the true angle. Then, by using the sample palmprint information set and the sample palmprint angle set as training data, each input palmprint image has a corresponding angle label, and the weight in the network is adjusted by comparing the difference between the predicted angle and the true angle (i.e., loss). Further, the back propagation algorithm is used to calculate the gradient of the loss function relative to each weight, and the weight in the network is adjusted by the gradient descent method to gradually reduce the error between the predicted angle and the true angle. The network is updated through multiple iterations, and the parameters of the network are continuously optimized until the loss function converges or reaches the preset stop condition, and the trained palmprint angle recognition branch is obtained.

[0031] On the other hand, an image angle recognition branch is constructed based on a convolutional neural network, wherein the input of the image angle recognition branch is image information, and the output is image angle; then, sample image information is used as input, and sample image angle is used as supervision, and the sample image information set and the sample image angle set are used as supervised training data to train the image angle recognition branch until convergence, thereby obtaining a trained image angle recognition branch, wherein the training method of the image angle recognition branch is the same as the training method of the palmprint angle recognition branch mentioned above, and will not be elaborated here.

[0032] Next, the palm print angle recognition branch and the image angle recognition branch are combined to obtain a biometric angle recognition channel; then the palm print information and the image information are respectively input into the palm print angle recognition branch and the image angle recognition branch in the biometric angle recognition channel for angle recognition, and the palm print angle and the image angle are output.

[0033] Furthermore, step S300 of the present invention further includes: S360: Collect multiple historical palm print angle sequences and multiple historical image angle sequences based on the historical identification data records of the multiple pending users; S370: Calculate the first user average palm print angle of the first historical palm print angle sequence of the first pending user; S380: Calculate the palm print angle deviation ratio between the palm print angle and the first user average palm print angle, and subtract the palm print angle deviation ratio from 1 to obtain the first palm print angle similarity; S390: Continue to calculate to obtain multiple palm print angle similarities between the palm print angle and the multiple historical palm print angle sequences, and calculate to obtain multiple image angle similarities between the image angle and the multiple historical image angle sequences.

[0034] Specifically, according to the historical identification data records of the plurality of pending users, a plurality of historical palm print angle sequences (user's palm print angle information in the past period of time) and a plurality of historical image angle sequences (user's face image angle information in the past period of time) are collected within a historical time period (such as the latest month); then, a first pending user is randomly selected from the plurality of pending users, the first historical palm print angle sequence of the first pending user is obtained, and the mean of the plurality of first historical palm print angles in the first historical palm print angle sequence is calculated to obtain the first user's average palm print angle. The palm print angle deviation ratio between the palm print angle and the first user's average palm print angle is further calculated, wherein the palm print angle deviation ratio is the ratio of the first angle difference between the palm print angle and the first user's average palm print angle to the first user's average palm print angle; then, 1 is subtracted from the palm print angle deviation ratio, and the difference between the two is taken as the first palm print angle similarity, wherein the greater the palm print angle similarity, the higher the degree of agreement between the palm print angle of the current user and the historical record. Then, the same method is used to continue calculating to obtain the plurality of palm print angle similarities between the palm print angle and the plurality of historical palm print angle sequences.

[0035] On the other hand, a plurality of average image angles of the plurality of historical image angle sequences are calculated, and a plurality of image angle similarities are calculated based on the image angles and the plurality of average image angles.

[0036] S400: Obtain the current charge amount, and obtain multiple historical amount sequences of the multiple pending users, respectively analyze and obtain multiple amount fluctuation similarities, combine the multiple palm print angle similarities and the multiple image angle similarities, compensate and adjust the multiple palm print probabilities and the multiple image probabilities, obtain multiple user probabilities, and output the pending user with the largest user probability as the user identification result.

[0037] Furthermore, step S400 of the present invention further includes: S410: Obtain the current charge amount; S420: Collect multiple historical amount sequences based on the historical consumption data records of the multiple pending users; S430: Calculate the average amounts of multiple users in the multiple historical amount sequences; S440: Calculate multiple amount deviation ratios between the charge amount and the average amounts of the multiple users, and respectively subtract the multiple amount deviation ratios from 1 to obtain multiple amount fluctuation similarities.

[0038] Specifically, the current charging amount is obtained, such as the consumption amount recorded in real time when the user purchases goods at a desktop charging machine such as a convenience store; then, based on the historical consumption data records of the multiple pending users, multiple historical amount sequences within the historical time period (such as the last month) are collected. The multiple historical amount sequences are respectively averaged to obtain multiple user average amounts. Then, multiple amount deviation ratios of the charging amount and the multiple user average amounts are calculated respectively, where the amount deviation ratio is the ratio of the absolute value of the difference between the charging amount and the user average amount to the user average amount. For example, assuming that the charging amount is 60 yuan and the user average amount is 50 yuan, the deviation ratio is (60-50) / 50=0.2. Further, 1 is subtracted from the multiple amount deviation ratios to obtain multiple amount fluctuation similarities.

[0039] By combining the current charge amount with the historical consumption amounts of multiple pending users, calculating the deviation ratio and amount fluctuation similarity, the accuracy of user identity authentication can be effectively improved. In complex scenarios, the amount fluctuation similarity provides another effective matching dimension, and together with other biometric information (such as palm print and facial angle similarity), it can provide more accurate and reliable user identity authentication.

[0040] Furthermore, step S400 of the present invention further includes: S450: Use the multiple palm print angle similarities, multiple image angle similarities and multiple amount fluctuation similarities to calculate and obtain multiple compensation adjustment coefficients respectively; S460: Use the multiple compensation adjustment coefficients to multiply the palm print probability and image probability of the corresponding pending users respectively to obtain multiple compensated palm print probabilities and multiple compensated image probabilities; S470: Perform weighted calculation on the multiple compensated palm print probabilities and multiple compensated image probabilities to obtain multiple user probabilities, and output the pending user with the largest user probability as the user identification result.

[0041] Specifically, the multiple palm print angle similarities, multiple image angle similarities and multiple amount fluctuation similarities are integrated, and the palm print angle similarity, image angle similarity and amount fluctuation similarity of the first pending user are randomly selected; weight ratios are set for the palm print angle, image angle and amount fluctuation indicators respectively, wherein the sum of the three weights is 1, and the weights are set according to the degree of influence of the indicator on the user identification accuracy, and the greater the influence, the greater the corresponding weight; according to the weight ratio, the palm print angle similarity, image angle similarity and amount fluctuation similarity are weightedly calculated to obtain a first compensation adjustment coefficient of the first pending user. For example, assuming that the palm print angle similarity, image angle similarity and amount fluctuation similarity of the first user are 0.9, 0.8 and 0.8 respectively, and the weight ratios of the palm print angle, image angle and amount fluctuation indicators are 0.5, 0.4 and 0.1 respectively, then the compensation adjustment coefficient of the first pending user is 0.9*0.5+0.8*0.4+0.8*0.1=0.85, and multiple compensation adjustment coefficients of multiple pending users are calculated.

[0042] Then, the multiple compensation adjustment coefficients are used to multiply the palm print probability and image probability of the corresponding pending user, respectively, to obtain multiple compensated palm print probabilities and multiple compensated image probabilities. For example, assuming that the compensation adjustment coefficient of the first pending user is 0.85, the palm print probability and the image probability are 0.95 and 0.9 respectively, then the compensated palm print probability of the first pending user is 0.85*0.95=0.8275, and the compensated image probability is 0.85*0.9=0.765. Trust weights are set for palm print recognition and image recognition (face recognition) respectively, where the sum of the weights of the two is 1, which can be set according to the historical recognition accuracy of palm print recognition and image recognition. The higher the accuracy, the larger the corresponding trust weight, and the palm print recognition weight and the image recognition weight are obtained. Finally, the multiple compensated palm print probabilities and the multiple compensated image probabilities are weightedly calculated according to the palm print recognition weight and the image recognition weight to obtain multiple user probabilities; and the pending user with the largest user probability among the multiple user probabilities is output as the user identification result. Optionally, the average of the compensated palm print probabilities and compensated image probabilities of multiple pending users may also be calculated as the user probability.

[0043] By calculating the compensation adjustment coefficients for the palmprint angle similarity, image angle similarity and amount fluctuation similarity respectively, and adjusting the palmprint probability, image probability and amount probability according to these compensation adjustment coefficients, and finally obtaining the final user probability through weighted calculation, this method of comprehensively considering multi-dimensional information can effectively improve the accuracy and robustness of user identity authentication, especially in the accurate identification of user identities in complex environments.

[0044] The user identification method combined with biometric features provided by the embodiment of the present invention has at least the following technical effects: The method comprises collecting the palm print information of the user to perform palm print recognition, obtaining a plurality of pending users and a plurality of palm print probabilities; then, when the maximum palm print probability is not greater than a preset probability threshold, collecting the image information of the user, performing user image recognition, and obtaining a plurality of image probabilities; then identifying the palm print angle of the palm print information and the image angle of the image information, and obtaining a plurality of historical palm print angle sequences and a plurality of historical image angle sequences of the plurality of pending users; further analyzing the similarities of the palm print angle and a plurality of palm print angles of the plurality of historical palm print angle sequences, and analyzing the similarities of the image angle and a plurality of image angles of the plurality of historical image angle sequences; on the other hand, The current charging amount is obtained, and multiple historical amount sequences of the multiple pending users are obtained, and multiple amount fluctuation similarities are obtained by respectively analyzing them; finally, according to the multiple amount fluctuation similarities, multiple palm print angle similarities and multiple image angle similarities, the multiple palm print probabilities and the multiple image probabilities are compensated and adjusted to obtain multiple user probabilities, and the pending user with the largest user probability is output as the user identification result; that is, through multimodal information fusion, the user's recognition probability can be compensated and optimized in multiple dimensions, avoiding the possible misidentification or missed identification of a single feature, thereby significantly improving the precision, accuracy and reliability of user identity authentication in complex environments.

[0045] Embodiment 2, as Figure 2 As shown, based on the same inventive concept as the user identification method combined with biometric features provided in the first embodiment, the embodiment of the present invention further provides a user identification device combined with biometric features, including: The user palmprint recognition module 01 is used to collect the palmprint information of the user through the palmprint collection device, perform user palmprint recognition, and obtain multiple pending users and multiple palmprint probabilities; the user image recognition module 02 is used to collect the image information of the user through the image collection device when the maximum palmprint probability is not greater than the preset probability threshold, perform user image recognition, and obtain multiple image probabilities; the angle similarity analysis module 03 is used to identify the palmprint angle of the palmprint information and the image angle of the image information, obtain multiple historical palmprint angle sequences and multiple historical image angle sequences of the multiple pending users, analyze the palmprint angle and the multiple palmprint angle similarities of the multiple historical palmprint angle sequences, and analyze the multiple image angle similarities of the image angle and the multiple historical image angle sequences; the user identification result output module 04 is used to obtain the current charging amount and obtain the multiple historical amount sequences of the multiple pending users, respectively analyze and obtain multiple amount fluctuation similarities, combine the multiple palmprint angle similarities and the multiple image angle similarities, compensate and adjust the multiple palmprint probabilities and the multiple image probabilities, obtain multiple user probabilities, and output the pending user with the largest user probability as the user identification result.

[0046] The user identification device combined with biometric features is also used to: collect the user's palmprint information through a palmprint collection device; input the palmprint information into a pre-constructed palmprint recognition channel, identify the similarity with multiple sample palmprint information in the palmprint recognition channel, output the largest multiple palmprint similarities and the corresponding users, and obtain multiple pending users, wherein the palmprint recognition channel is obtained based on the training of the sample palmprint information set and the sample palmprint similarity set; output the palmprint similarities corresponding to the multiple pending users as multiple palmprint probabilities.

[0047] The user identification device combined with biometric features is also used to: determine whether the maximum palm print probability is greater than a preset probability threshold, and if so, output the palm print user corresponding to the maximum palm print probability as the user identification result; if not, collect the image information of the user through an image acquisition device; input the image information into a pre-constructed image recognition channel to identify the similarity with the sample image information of the multiple pending users, and obtain multiple image similarities, wherein the image recognition channel is trained based on a sample image information set and a sample image similarity set; and output the multiple image similarities as multiple image probabilities.

[0048] The user identification device combined with biometric features is also used to: collect sample palmprint information sets and sample image information sets based on user identification data in historical time, and mark the palmprint angle and image angle of each sample palmprint information and each sample image information to obtain a sample palmprint angle set and a sample image angle set; use the sample palmprint information set and the sample palmprint angle set as supervised training data to train a palmprint angle recognition branch; use the sample image information set and the sample image angle set as supervised training data to train an image angle recognition branch; combine the palmprint angle recognition branch and the image angle recognition branch to obtain a biometric angle recognition channel; input the palmprint information and image information into the palmprint angle recognition branch and the image angle recognition branch in the biometric angle recognition channel, and obtain the palmprint angle and image angle through recognition output.

[0049] The user identification device combined with biometric features is also used to: collect multiple historical palm print angle sequences and multiple historical image angle sequences based on the historical identification data records of the multiple pending users; calculate the first user average palm print angle of the first historical palm print angle sequence of the first pending user; calculate the palm print angle deviation ratio between the palm print angle and the first user average palm print angle, and obtain the first palm print angle similarity by subtracting the palm print angle deviation ratio from 1; continue to calculate to obtain multiple palm print angle similarities between the palm print angle and the multiple historical palm print angle sequences, and calculate to obtain multiple image angle similarities between the image angle and the multiple historical image angle sequences.

[0050] The user identification device combined with biometric features is also used to: obtain the current charging amount; collect multiple historical amount sequences based on the historical consumption data records of the multiple pending users; calculate the average amounts of multiple users in the multiple historical amount sequences; calculate multiple amount deviation ratios between the charging amount and the average amounts of the multiple users, and respectively subtract the multiple amount deviation ratios from 1 to obtain multiple amount fluctuation similarities.

[0051] The user identification device combined with biometric features is also used to: use the multiple palm print angle similarities, multiple image angle similarities and multiple amount fluctuation similarities to calculate multiple compensation adjustment coefficients respectively; use the multiple compensation adjustment coefficients to multiply the palm print probability and image probability of the corresponding pending user respectively to obtain multiple compensated palm print probabilities and multiple compensated image probabilities; perform weighted calculation on the multiple compensated palm print probabilities and multiple compensated image probabilities to obtain multiple user probabilities, and output the pending user with the largest user probability as the user identification result.

[0052] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0053] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0054] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0055] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0056] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0057] Although preferred embodiments of the present invention have been described, additional changes and modifications may occur to these embodiments once those skilled in the art understand the basic inventive concepts.

[0058] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and variations.

Claims

1. A user identification method combining biometric features, characterized in that: Methods include: The palm print information of the user is collected by a palm print collection device, and the palm print of the user is recognized to obtain multiple pending users and multiple palm print probabilities; When the maximum palm print probability is not greater than a preset probability threshold, the image information of the user is collected by an image collection device, and user image recognition is performed to obtain multiple image probabilities; Identifying the palm print angle of the palm print information and the image angle of the image information, acquiring multiple historical palm print angle sequences and multiple historical image angle sequences of the multiple pending users, analyzing the similarities between the palm print angle and multiple palm print angles of the multiple historical palm print angle sequences, and analyzing the similarities between the image angle and multiple image angles of the multiple historical image angle sequences; The current charging amount is obtained, and multiple historical amount sequences of the multiple pending users are obtained, and multiple amount fluctuation similarities are obtained by respectively analyzing them. Combined with the multiple palm print angle similarities and the multiple image angle similarities, the multiple palm print probabilities and the multiple image probabilities are compensated and adjusted to obtain multiple user probabilities, and the pending user with the largest user probability is output as the user identification result.

2. The user identification method combining biometrics according to claim 1, characterized in that: The palm print information of the user is collected by the palm print collection device, and the palm print of the user is recognized to obtain multiple pending users and multiple palm print probabilities, including: Collecting the user's palm print information through a palm print collection device; Input the palmprint information into a pre-built palmprint recognition channel, identify similarities with multiple sample palmprint information in the palmprint recognition channel, output multiple maximum palmprint similarities and corresponding users, and obtain multiple pending users, wherein the palmprint recognition channel is trained based on a sample palmprint information set and a sample palmprint similarity set; The palmprint similarities corresponding to the multiple pending users are output as multiple palmprint probabilities.

3. The user identification method combining biometrics according to claim 1, characterized in that: When the maximum palm print probability is not greater than a preset probability threshold, the image information of the user is collected by an image collection device, and user image recognition is performed to obtain multiple image probabilities, including: Determine whether the maximum palm print probability is greater than a preset probability threshold, and if so, output the palm print user corresponding to the maximum palm print probability as the user identification result; If not, collecting image information of the user through an image collection device; Input the image information into a pre-built image recognition channel, identify similarities with the sample image information of the plurality of pending users, and obtain a plurality of image similarities, wherein the image recognition channel is trained based on a sample image information set and a sample image similarity set; The multiple image similarities are output as a plurality of image probabilities.

4. The user identification method combining biometrics according to claim 1, characterized in that: Identifying a palm print angle of the palm print information and an image angle of the image information includes: According to the user identification data in the historical time, a sample palmprint information set and a sample image information set are collected, and the palmprint angle and image angle of each sample palmprint information and each sample image information are annotated to obtain a sample palmprint angle set and a sample image angle set; Using the sample palmprint information set and the sample palmprint angle set as supervised training data to train a palmprint angle recognition branch; Using the sample image information set and the sample image angle set as supervised training data to train the image angle recognition branch; Combining the palm print angle recognition branch and the image angle recognition branch to obtain a biometric angle recognition channel; The palm print information and the image information are input into the palm print angle recognition branch and the image angle recognition branch in the biometric angle recognition channel, and the palm print angle and the image angle are obtained by recognition output.

5. The user authentication method combining biometrics according to claim 1, characterized in that: Acquiring multiple historical palm print angle sequences and multiple historical image angle sequences of the multiple pending users, analyzing the similarities between the palm print angle and multiple palm print angles of the multiple historical palm print angle sequences, and analyzing the similarities between the image angle and multiple image angles of the multiple historical image angle sequences, including: According to the historical identification data records of the plurality of pending users, collecting a plurality of historical palm print angle sequences and a plurality of historical image angle sequences; Calculate the average palm print angle of the first user of the first historical palm print angle sequence of the first to-be-determined user; Calculating a palm print angle deviation ratio between the palm print angle and the first user's average palm print angle, and subtracting the palm print angle deviation ratio from 1 to obtain a first palm print angle similarity; Continue to calculate and obtain multiple palmprint angle similarities between the palmprint angle and the multiple historical palmprint angle sequences, and calculate and obtain multiple image angle similarities between the image angle and the multiple historical image angle sequences.

6. The user authentication method combining biometrics according to claim 1, characterized in that: The current charging amount is obtained, and multiple historical amount sequences of the multiple pending users are obtained, and multiple amount fluctuation similarities are respectively analyzed to obtain the similarities, including: Get the current charge amount; Collecting multiple historical amount sequences according to the historical consumption data records of the multiple pending users; Calculating average amounts of multiple users of the multiple historical amount sequences; Calculate multiple amount deviation ratios between the charged amount and the average amount of the multiple users, and subtract the multiple amount deviation ratios from 1 to obtain multiple amount fluctuation similarities.

7. The user authentication method combining biometrics according to claim 1, characterized in that: Combining the multiple palm print angle similarities and the multiple image angle similarities, compensating and adjusting the multiple palm print probabilities and the multiple image probabilities to obtain multiple user probabilities, and outputting the pending user with the largest user probability as the user identification result, including: Using the multiple palm print angle similarities, the multiple image angle similarities and the multiple amount fluctuation similarities, multiple compensation adjustment coefficients are calculated respectively; The plurality of compensation adjustment coefficients are respectively multiplied by the palm print probability and the image probability of the corresponding to-be-determined user to obtain a plurality of compensated palm print probabilities and a plurality of compensated image probabilities; A weighted calculation is performed on the multiple compensated palm print probabilities and the multiple compensated image probabilities to obtain multiple user probabilities, and the pending user with the largest user probability is output as the user identification result.

8. A user authentication device combining biometric features, characterized in that: The steps for implementing the user authentication method combined with biometric features as described in any one of claims 1 to 7 include: A user palm print recognition module is used to collect the user's palm print information through a palm print collection device, perform user palm print recognition, and obtain multiple pending users and multiple palm print probabilities; A user image recognition module, for collecting the image information of the user through an image acquisition device, performing user image recognition, and obtaining multiple image probabilities when the maximum palm print probability is not greater than a preset probability threshold; an angle similarity analysis module, for identifying the palm print angle of the palm print information and the image angle of the image information, obtaining multiple historical palm print angle sequences and multiple historical image angle sequences of the multiple pending users, analyzing the palm print angle and multiple palm print angle similarities of the multiple historical palm print angle sequences, and analyzing the image angle and multiple image angle similarities of the multiple historical image angle sequences; The user identification result output module is used to obtain the current charging amount and multiple historical amount sequences of the multiple pending users, respectively analyze and obtain multiple amount fluctuation similarities, combine the multiple palm print angle similarities and the multiple image angle similarities, compensate and adjust the multiple palm print probabilities and the multiple image probabilities, obtain multiple user probabilities, and output the pending user with the largest user probability as the user identification result.

Images