User identification methods, devices, smart home appliances, and storage media

Abstract

This invention provides a user identification method, device, smart home appliance, and storage medium. The method includes determining a positive vein image and a negative vein image based on a limb vein image of a user to be identified, and performing feature extraction and identification on the positive and negative vein images using a trained feature extraction network to obtain a user identification result for the user to be identified. In the solution provided in this application, acquiring a user's limb vein image and obtaining the user identification result overcomes the problem of low security of fingerprint images.

Description

Technical Field

[0001] This invention relates to the field of identification technology, and specifically to a user identification method, device, smart home appliance, and storage medium. Background Technology

[0002] The rapid development of biometric technology has driven the widespread and in-depth development of visual processing and signal processing technologies; with the continuous progress of science and technology, the requirements for the security of identity authentication are also gradually increasing.

[0003] Currently, fingerprint recognition technology is becoming increasingly unsecured due to issues such as easy theft, and it can no longer meet the mainstream technological needs. Summary of the Invention

[0004] To alleviate the low security issues of current fingerprint recognition technology, embodiments of the present invention provide a user identification method, device, smart home appliance, and storage medium.

[0005] In a first aspect, embodiments of the present invention provide a user identification method, applied to smart home appliances; the method includes:

[0006] Based on the limb vein images of the user to be identified, determine the positive and negative vein images;

[0007] The trained feature extraction network extracts features from the positive vein image and the negative vein image to obtain the first image features of the positive vein image and the second image features of the negative vein image.

[0008] Based on the trained first classification network and the limb vein images of registered users, the first image features and the second image features are identified respectively to obtain the first identification result;

[0009] Based on the trained second classification network and the limb vein images of the registered user, the fused image features of the first image features and the second image features are identified to obtain a second identification result;

[0010] Based on the first identification result and the second identification result, the user identification result of the user to be identified is obtained.

[0011] In some embodiments, prior to the step of determining the positive and negative vein images based on the limb vein images of the user to be identified, the method further includes:

[0012] When a smart home appliance is detected to be triggered, the infrared camera is enabled.

[0013] The infrared camera is used to acquire images of the veins in the limbs of the user to be identified.

[0014] In some embodiments, the step of determining positive and negative vein images based on limb vein images of the user to be identified includes:

[0015] The limb vein image of the user to be identified is determined as the positive vein image;

[0016] The limb vein image of the user to be identified is flipped to generate the negative vein image.

[0017] In some embodiments, the step of extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain a first image feature of the positive vein image and a second image feature of the negative vein image includes:

[0018] The forward vein image is preprocessed by the first preprocessing network in the first feature extraction network to obtain a forward image to be extracted; the forward image to be extracted is subjected to weight matrix feature extraction by the first autoencoder network in the first feature extraction network to obtain a forward weight matrix feature; based on the forward weight matrix feature, the forward image to be extracted is subjected to image feature extraction by the first feature processing network in the first feature extraction network to obtain the first image feature;

[0019] The negative vein image is preprocessed by the second preprocessing network in the second feature extraction network to obtain a negative image to be extracted; the negative image to be extracted is then subjected to weight matrix feature extraction by the second autoencoder network in the second feature extraction network to obtain negative weight matrix features; based on the negative weight matrix features, the negative image to be extracted is then subjected to image feature extraction by the second feature processing network in the second feature extraction network to obtain the second image features.

[0020] In some embodiments, the step of identifying the first image features and the second image features based on the trained first classification network and the limb vein image of the registered user to obtain a first identification result includes:

[0021] Based on the positive limb vein image corresponding to the limb vein image of the registered user, the first image feature is identified through the first classification network to obtain the positive image recognition result of the first image feature;

[0022] Based on the negative limb vein image corresponding to the limb vein image of the registered user, the second image feature is identified through the first classification network to obtain the negative image recognition result of the second image feature;

[0023] The first recognition result is obtained based on the positive image recognition result, the negative image recognition result, and the corresponding score weights.

[0024] In some embodiments, the step of obtaining the user identification result of the user to be identified based on the first identification result and the second identification result includes:

[0025] Obtain the multimodal fusion weights;

[0026] Based on the first identification result, the second identification result, and the multimodal fusion weight, the user identification result of the user to be identified is obtained.

[0027] In some embodiments, before the step of extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain a first image feature of the positive vein image and a second image feature of the negative vein image, the method further includes:

[0028] Obtain the encoder-decoder network to be trained;

[0029] The encoder-decoder network to be trained is trained based on vein images in a unified database to obtain the trained encoder-decoder network.

[0030] Based on the network parameters of the encoder in the trained encoder-decoder network, the network parameters of the autoencoder network are determined.

[0031] In some embodiments, the user identification method further includes:

[0032] A neural network to be trained is constructed; the neural network includes a first feature extraction network, a second feature extraction network, a first recognition network, a second recognition network, and an output network, wherein the first feature extraction network and the second feature extraction network are connected in parallel, and the first recognition network and the second recognition network are connected in parallel; the first feature extraction network includes a first preprocessing network, a first autoencoder network, and a first feature processing network connected in sequence, and the second feature extraction network includes a second preprocessing network, a second autoencoder network, and a second feature processing network connected in sequence, wherein the network parameters of the first preprocessing network and the second preprocessing network are the same, the network parameters of the first autoencoder network and the second autoencoder network are the same, and the network parameters of the first feature processing network and the second feature processing network are the same; the first recognition network includes multiple first classification networks with different network structures, and the second recognition network includes multiple second classification networks with different network structures;

[0033] The neural network to be trained is trained based on vein images in the unified database to determine the network parameters of the first feature processing network, the network structure and network parameters of the first classification network and the second classification network.

[0034] Secondly, embodiments of the present invention provide a user identification device, disposed in a smart home appliance; the device includes:

[0035] The first module is used to determine the positive and negative vein images based on the limb vein images of the user to be identified;

[0036] The second module is used to extract features from the positive vein image and the negative vein image through the trained feature extraction network to obtain the first image features of the positive vein image and the second image features of the negative vein image;

[0037] The third module is used to identify the first image features and the second image features based on the trained first classification network and the limb vein images of the registered user, respectively, to obtain the first identification result;

[0038] The fourth module is used to identify the fused image features of the first image features and the second image features based on the trained second classification network and the limb vein images of the registered user, and to obtain a second identification result;

[0039] The fifth module is used to obtain the user identification result of the user to be identified based on the first identification result and the second identification result.

[0040] Thirdly, embodiments of the present invention provide a smart home appliance, the smart home appliance including a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, it implements the method described in the first aspect.

[0041] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by one or more processors, implements the method described in the first aspect.

[0042] Compared with the prior art, one or more embodiments of the present invention can bring at least the following beneficial effects:

[0043] This invention provides a user identification method, device, smart home appliance, and storage medium. The method includes determining a positive vein image and a negative vein image based on a limb vein image of a user to be identified; extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain a first image feature of the positive vein image and a second image feature of the negative vein image; identifying the first image feature and the second image feature based on a trained first classification network and a limb vein image of a registered user to obtain a first identification result; identifying a fused image feature of the first image feature and the second image feature based on a trained second classification network and a limb vein image of the registered user to obtain a second identification result; and obtaining a user identification result for the user to be identified based on the first identification result and the second identification result. In the solution provided in this application, a user's limb vein image is acquired, and bidirectional extraction features of the limb vein image of the user to be identified are obtained through a feature extraction network. Then, based on a first classification network and the limb vein image of the registered user, the bidirectional extraction features are identified to obtain a first identification result. Based on a second classification network and the limb vein image of the registered user, the fused features of the bidirectional extraction features are identified to obtain a second identification result. Finally, the user identification result of the user to be identified is obtained based on the first identification result and the second identification result. This application uses limb vein images to replace fingerprint images for user identification, overcoming the problem of low security of fingerprint images. At the same time, the recognition efficiency of limb vein images is also very high, ensuring the accuracy of identification. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the first type of user identification method provided in the embodiments of the present invention;

[0046] Figure 2a This is a schematic diagram of the training process provided in an embodiment of the present invention;

[0047] Figure 2b This is a schematic diagram of the identification process provided in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of a user identification device provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0050] Example 1

[0051] Figure 1 This illustrates a first flowchart of the user identification method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the user identification method provided in this application includes:

[0052] Step S110: Based on the limb vein images of the user to be identified, determine the positive vein image and the negative vein image.

[0053] The vein recognition technology involved in this application is far superior to other biometric technologies such as fingerprint recognition in terms of high security and ease of use. It utilizes the characteristic that human hemoglobin absorbs near-infrared light when passing through veins, and can collect subcutaneous vein images as features for biometric extraction. It has the characteristics of high anti-counterfeiting, ease of use (not affected by hand environment), high accuracy, and rapid recognition, making it very suitable for use in public places.

[0054] Existing image recognition technologies are mainly divided into two categories: non-learning models and learning models. Non-learning models mainly use feature extraction methods such as signal filtering, extraction of binary texture information, and feature point extraction, and then use Euclidean distance as the final matching strategy. Learning models, which combine feature extraction methods such as PCA and filters with classification models, are adopted by most people. They can solve problems such as image quality and texture loss in non-learning models. However, these recognition technologies cannot meet the user's recognition requirements.

[0055] In some embodiments, prior to the step of determining the positive and negative vein images based on the limb vein images of the user to be identified, the method further includes: enabling an infrared sensor when a smart home appliance is detected to be triggered; and acquiring the limb vein images of the user to be identified through the infrared sensor.

[0056] In this application, smart home appliances can be home appliances that require user identification, such as vein recognition door locks.

[0057] Specifically, when the user's limbs come into contact with the image data acquisition module of the smart home appliance, the smart home appliance enables the infrared camera to acquire images of the limb veins under near-infrared light in order to collect the biometric features of the user.

[0058] Furthermore, the limb vein images acquired by the infrared camera are preprocessed to improve image quality. Limb vein image preprocessing includes mean filtering, region of interest (ROI) extraction, region cropping, size normalization, and image enhancement. ROI extraction includes edge detection, skeletonization, closure operations, and noise removal from the acquired image. Size normalization is implemented using the resize function. Image enhancement employs methods such as linear transformation and grayscale histograms. The preprocessed limb vein image is then used as the limb vein image of the user to be identified in subsequent steps.

[0059] In some embodiments, the step of determining a positive vein image and a negative vein image based on the limb vein image of the user to be identified includes: determining the limb vein image of the user to be identified as the positive vein image; and flipping the limb vein image of the user to be identified to generate the negative vein image.

[0060] Specifically, the limb vein image of the user to be identified is determined as the positive vein image; the positive vein image is rotated 180 degrees, and the resulting image is used as the negative vein image.

[0061] Step S120: Extract features from the positive vein image and the negative vein image using the trained feature extraction network to obtain the first image features of the positive vein image and the second image features of the negative vein image.

[0062] In some embodiments, before the step of extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain a first image feature of the positive vein image and a second image feature of the negative vein image, the method further includes: obtaining an encoder-decoder network to be trained; training the encoder-decoder network to be trained based on vein images in a unified database to obtain a trained encoder-decoder network; and determining the network parameters of an autoencoder network based on the network parameters of the encoder in the trained encoder-decoder network. The autoencoder network is used to reconstruct the original features from the input to extract the original data features, while simultaneously reducing the model size for better adaptation to small microcomputers.

[0063] In some embodiments, the step of extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain a first image feature of the positive vein image and a second image feature of the negative vein image includes: preprocessing the positive vein image using a first preprocessing network in the first feature extraction network to obtain a positive image to be extracted; extracting weight matrix features from the positive image to be extracted using a first autoencoder network in the first feature extraction network to obtain a positive weight matrix feature; extracting image features from the positive image to be extracted using a first feature processing network in the first feature extraction network based on the positive weight matrix feature to obtain the first image feature; preprocessing the negative vein image using a second preprocessing network in the second feature extraction network to obtain a negative image to be extracted; extracting weight matrix features from the negative image to be extracted using a second autoencoder network in the second feature extraction network to obtain a negative weight matrix feature; and extracting image features from the negative image to be extracted using a second feature processing network in the second feature extraction network based on the negative weight matrix feature to obtain the second image feature.

[0064] Step S130: Based on the trained first classification network and the limb vein image of the registered user, the first image features and the second image features are identified respectively to obtain the first identification result.

[0065] In some embodiments, the step of identifying the first image feature and the second image feature based on the trained first classification network and the limb vein image of the registered user to obtain a first recognition result includes: identifying the first image feature through the first classification network based on the positive limb vein image corresponding to the limb vein image of the registered user, to obtain a positive image recognition result of the first image feature; identifying the second image feature through the first classification network based on the negative limb vein image corresponding to the limb vein image of the registered user, to obtain a negative image recognition result of the second image feature; and obtaining the first recognition result based on the positive image recognition result, the negative image recognition result, and the corresponding score weights. That is, the first recognition result is obtained using a score-level fusion method.

[0066] Step S140: Based on the trained second classification network and the limb vein image of the registered user, identify the fused image features of the first image features and the second image features to obtain a second identification result.

[0067] In this embodiment, the first image features and the second image features need to be fused to obtain fused image features, and then the fused image features are identified based on the limb vein image of the registered user to obtain a second identification result; that is, the second identification result is obtained by using the feature-level fusion method.

[0068] In this application, SVM and simple ELM networks can be used for comparative experiments to select the optimal classification network.

[0069] Step S150: Based on the first identification result and the second identification result, obtain the user identification result of the user to be identified.

[0070] In some embodiments, the step of obtaining the user identification result of the user to be identified based on the first identification result and the second identification result includes: obtaining multimodal fusion weights; and obtaining the user identification result of the user to be identified based on the first identification result, the second identification result, and the multimodal fusion weights.

[0071] In this application, the multimodal fusion technique may employ transfer learning to reduce training parameters and improve training speed.

[0072] In some embodiments, the user identification method further includes: constructing a neural network to be trained; the neural network includes a first feature extraction network, a second feature extraction network, a first recognition network, a second recognition network, and an output network, wherein the first feature extraction network and the second feature extraction network are connected in parallel, and the first recognition network and the second recognition network are connected in parallel; the first feature extraction network includes a first preprocessing network, a first autoencoder network, and a first feature processing network connected in sequence, and the second feature extraction network includes a second preprocessing network, a second autoencoder network, and a second feature processing network connected in sequence, wherein the network parameters of the first preprocessing network and the second preprocessing network are the same, the network parameters of the first autoencoder network and the second autoencoder network are the same, and the network parameters of the first feature processing network and the second feature processing network are the same; the first recognition network includes multiple first classification networks with different network structures, and the second recognition network includes multiple second classification networks with different network structures; the neural network to be trained is trained based on vein images in the unified database to determine the network parameters of the first feature processing network, the network structure and network parameters of the first classification network and the second classification network.

[0073] That is, in the method provided by the present invention, the user's limb vein image is acquired, and bidirectional extraction features of the limb vein image of the user to be identified are obtained through a feature extraction network. Then, the bidirectional extraction features are identified based on a first classification network and the limb vein image of the registered user to obtain a first identification result. The fused features of the bidirectional extraction features are identified based on a second classification network and the limb vein image of the registered user to obtain a second identification result. Finally, the user identification result of the user to be identified is obtained based on the first identification result and the second identification result. This application uses limb vein images to replace fingerprint images for user identification, overcoming the problem of low security of fingerprint images. At the same time, the image recognition technology has also been improved, making the recognition efficiency of limb vein images very high and ensuring the recognition accuracy.

[0074] The present application will now be further explained in conjunction with a specific scenario. In this scenario, smart home appliances such as smart door locks with finger vein recognition function will be used as examples to illustrate the user identification method provided by the present application.

[0075] Figure 2a This is a schematic diagram of the training process provided in an embodiment of the present invention; as shown below. Figure 2a As shown, the training process provided in this application includes two stages, which are described in detail below.

[0076] In training phase 1, it is necessary to complete the training of the encoder-decoder network for the field of finger vein image recognition.

[0077] This stage includes: acquiring the encoder-decoder network to be trained; training the encoder-decoder network to be trained based on vein images in a unified database to obtain the trained encoder-decoder network; and determining the network parameters of the autoencoder network based on the network parameters of the encoder in the trained encoder-decoder network. This application employs transfer learning to reduce training complexity.

[0078] The encoder network aims to learn a compressed representation of the input. In constructing a classification autoencoder network, a multilayer perceptron autoencoder model is developed, which samples the values ​​of all input columns and then outputs the same value. In this application, the encoder learns how to interpret the input and compress it into an internal representation defined by the bottleneck layer; that is, it learns a weight matrix that preserves the original image information. The decoder takes the encoder's output and attempts to reconstruct the input image. Training is complete when image reconstruction is successful. Once the encoder is trained, the decoder is discarded, and the encoder's network parameters are retained for use in subsequent networks. The autoencoder network can employ the optimization objective loss function Minmize Loss = dist(X, X...). R), where dist is the distance function used. The data training method is to minimize the loss function through continuous iteration, and to continuously minimize the reconstruction error between the input and output. The purpose of training is to find the optimal parameter matrix of the input image as the output.

[0079] In training phase 2, it is necessary to complete the training of the neural network for finger vein image recognition.

[0080] like Figure 2a As shown, the neural network provided in this application includes a first feature extraction network, a second feature extraction network, a first recognition network, a second recognition network, and an output network. The first feature extraction network and the second feature extraction network are connected in parallel, and the first recognition network and the second recognition network are connected in parallel. The first feature extraction network includes a first preprocessing network (data preprocessing a), a first autoencoder network (autoencoder a), and a first feature processing network (feature processing network a) connected in sequence. The second feature extraction network includes a second preprocessing network (data preprocessing b), a second autoencoder network (autoencoder b), and a second feature processing network (feature processing network b) connected in sequence. The network parameters of the first preprocessing network and the second preprocessing network are the same. The network parameters of the first autoencoder network and the second autoencoder network are the same. The network parameters of the first feature processing network and the second feature processing network are the same. The first recognition network includes multiple first classification networks with different network structures (feature matching a + classifier a forms one type of first classification network, feature matching b + classifier b forms another type of first classification network). The second recognition network includes multiple second classification networks with different network structures (feature matching c + classifier c forms one type of second classification network).

[0081] The training involved in this stage includes: training the neural network to be trained based on vein images in the unified database (i.e., the training set in the finger vein image set) to determine the network parameters of the first feature processing network, the network structure and network parameters of the first classification network and the second classification network.

[0082] Figure 2b This is a schematic diagram of the identification process provided in an embodiment of the present invention; as shown below. Figure 2b As shown, the method provided in this application includes the following steps:

[0083] Step 1: Acquire vein images and generate positive and negative vein images.

[0084] In this embodiment, the vein image of the user to be identified is acquired by the image data acquisition module of the vein recognition door lock, which is responsible for collecting the user's biometric features. In some embodiments, when the vein recognition door lock is detected to be triggered, the infrared camera is enabled; the infrared camera acquires the limb vein image of the user to be identified. That is, when the limb of the user to be identified touches the image data acquisition module of the vein recognition door lock, the vein recognition door lock enables the infrared camera to acquire the limb vein image under near-infrared light to collect the user's biometric features.

[0085] In this step, to ensure the accuracy of image recognition, the acquired vein images need to be enhanced.

[0086] Enhancement processing is performed on limb vein images acquired by an infrared camera to improve image quality. This enhancement process includes mean filtering, region of interest (ROI) extraction, region cropping, size normalization, and image enhancement. ROI extraction includes edge detection, skeletonization, closure operations, and noise removal from the acquired image. Size normalization is implemented using the resize function. Image enhancement employs methods such as linear transformation and grayscale histograms. The enhanced limb vein image is then used as the limb vein image of the user to be identified.

[0087] Based on this, the aforementioned method is used to generate a positive vein image from the acquired vein image and to generate a negative vein image by flipping the vein image.

[0088] Step 2: Image cropping and other preprocessing.

[0089] In this step, in order to ensure a uniform size for the neural network, preprocessing is required to perform image cropping and other preprocessing on the positive and negative vein images, respectively, through data preprocessing a and data preprocessing b.

[0090] Step 3: Image feature extraction process.

[0091] In this embodiment, as Figure 2b As shown, the vein recognition process based on bidirectional features uses a trained autoencoder and feature extraction network to extract features from the cropped positive and negative vein images, respectively, to obtain the first image features and the second image features.

[0092] Specifically, such as Figure 2bAs shown, a positive vein image is input into a first feature extraction network. The positive vein image undergoes preprocessing such as cropping through a first preprocessing network a to obtain a positive image to be extracted. Then, a first autoencoder network a extracts weight matrix features from the positive image to obtain positive weight matrix features. Finally, based on the positive weight matrix features, the first feature processing network a extracts image features from the positive image to obtain the first image features. Similarly, a negative vein image is input into a second feature extraction network. The negative vein image undergoes preprocessing such as cropping through a second preprocessing network b to obtain a negative image to be extracted. Then, a second autoencoder network b extracts weight matrix features from the negative image to obtain negative weight matrix features. Finally, based on the negative weight matrix features, the second feature processing network b extracts image features from the negative image to obtain the second image features.

[0093] Step 4: Data fusion and classification.

[0094] In this embodiment, a combination of feature-level fusion and fractional-level fusion is used for image feature matching and classification decisions. Specifically, as shown below... Figure 2b As shown, based on the positive and negative limb vein images corresponding to the registered user's limb vein images, the first image features are identified using feature matching a and classifier a in the first classification network (corresponding to fractional fusion) with the positive limb vein image corresponding to the registered user's limb vein image. Simultaneously, the second image features are identified using feature matching b and classifier b in the first classification network with the negative limb vein image corresponding to the registered user's limb vein image. This yields the positive image recognition result of the first image features and the negative image recognition result of the second image features. Then, combining the score weight 1 of classifier a (e.g., 70%) and the score weight 2 of classifier b (e.g., 30%), the positive and negative image recognition results are processed to obtain the first recognition result. Based on the registered user's limb vein image, the second recognition result is obtained by using feature matching c and classifier c in the second classification network (corresponding to feature-level fusion) to identify the fused image features obtained by fusing the first and second image features.

[0095] Step 5: Obtain the user identification results.

[0096] In this embodiment, as Figure 2b As shown, the user identification result of the user to be identified is obtained by calculating the first identification result, the second identification result, and the multimodal fusion weight (e.g., the weight corresponding to the feature-level fusion mode is 80%, the weight corresponding to the fractional-level fusion mode is 20%) based on the first identification result, the second identification result, and the multimodal fusion weight (e.g., the weight corresponding to the feature-level fusion mode is 80%, the weight corresponding to the fractional-level fusion mode is 20%).

[0097] As can be seen from the above scenario, in this application, the user's limb vein image is acquired, and bidirectional extraction features of the limb vein image of the user to be identified are obtained through a feature extraction network. Then, based on the first classification network and the limb vein image of the registered user, the bidirectional extraction features are identified to obtain a first identification result. Based on the second classification network and the limb vein image of the registered user, the fused features of the bidirectional extraction features are identified to obtain a second identification result. Finally, the user identification result of the user to be identified is obtained based on the first identification result and the second identification result. This application uses limb vein images to replace fingerprint images for user identification, overcoming the problem of low security of fingerprint images. At the same time, the recognition efficiency of limb vein images is also very high, ensuring the accuracy of identification.

[0098] Example 2

[0099] Figure 3 A schematic diagram of a user identification device is shown. This embodiment provides a user identification device that is installed in a smart home appliance; for example... Figure 3 As shown, the user identification device includes:

[0100] The first module 310 is used to determine the positive vein image and the negative vein image based on the limb vein image of the user to be identified;

[0101] The second module 320 is used to extract features from the positive vein image and the negative vein image through the trained feature extraction network to obtain the first image features of the positive vein image and the second image features of the negative vein image;

[0102] The third module 330 is used to identify the first image features and the second image features based on the trained first classification network and the limb vein images of the registered user, respectively, to obtain a first identification result;

[0103] The fourth module 340 is used to identify the fused image features of the first image features and the second image features based on the trained second classification network and the limb vein images of the registered user, and to obtain a second identification result;

[0104] The fifth module 350 is used to obtain the user identification result of the user to be identified based on the first identification result and the second identification result.

[0105] The beneficial effects of the device in this embodiment can be found in Embodiment 1, and will not be repeated here.

[0106] Those skilled in the art will understand that the above-described modules or steps can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by the computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. This invention is not limited to any specific hardware and software combination.

[0107] Example 3

[0108] This embodiment provides a smart home appliance, which includes a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, it implements the user identification method as described in Embodiment 1.

[0109] In this embodiment, the processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the methods in the above embodiments. The methods implemented when the computer program running on the processor is executed can be referred to the specific embodiments of the methods provided in the foregoing embodiments of this invention, and will not be repeated here.

[0110] Example 4

[0111] This embodiment provides a computer-readable storage medium storing a computer program. When executed by one or more processors, the computer program implements the method described in Embodiment 1.

[0112] Based on the limb vein images of the user to be identified, determine the positive and negative vein images;

[0113] The trained feature extraction network extracts features from the positive vein image and the negative vein image to obtain the first image features of the positive vein image and the second image features of the negative vein image.

[0114] Based on the trained first classification network and the limb vein images of registered users, the first image features and the second image features are identified respectively to obtain the first identification result;

[0115] Based on the trained second classification network and the limb vein images of the registered user, the fused image features of the first image features and the second image features are identified to obtain a second identification result;

[0116] Based on the first identification result and the second identification result, the user identification result of the user to be identified is obtained.

[0117] In this embodiment, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The method is detailed in Embodiment 1 and will not be repeated here.

[0118] In summary, embodiments of the present invention provide a user identification method, device, smart home appliance, and storage medium. The method includes determining a positive vein image and a negative vein image based on a limb vein image of a user to be identified; extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain a first image feature of the positive vein image and a second image feature of the negative vein image; identifying the first image feature and the second image feature based on a trained first classification network and a limb vein image of a registered user to obtain a first identification result; identifying a fused image feature of the first image feature and the second image feature based on a trained second classification network and a limb vein image of the registered user to obtain a second identification result; and obtaining a user identification result for the user to be identified based on the first identification result and the second identification result. In the solution provided in this application, a user's limb vein image is acquired, and bidirectional extraction features of the limb vein image of the user to be identified are obtained through a feature extraction network. Then, based on a first classification network and the limb vein image of the registered user, the bidirectional extraction features are identified to obtain a first identification result. Based on a second classification network and the limb vein image of the registered user, the fused features of the bidirectional extraction features are identified to obtain a second identification result. Finally, the user identification result of the user to be identified is obtained based on the first identification result and the second identification result. This application uses limb vein images to replace fingerprint images for user identification, overcoming the problem of low security of fingerprint images. At the same time, the recognition efficiency of limb vein images is also very high, ensuring the accuracy of identification.

[0119] In the several embodiments provided in this invention, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system and method embodiments described above are merely illustrative.

[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A user identification method, characterized in that, Applied to smart home appliances; the method includes: Based on the limb vein images of the user to be identified, a positive vein image and a negative vein image are determined, wherein the negative vein image is obtained by flipping the positive vein image; The trained feature extraction network extracts features from the positive vein image and the negative vein image to obtain the first image features of the positive vein image and the second image features of the negative vein image. Based on the trained first classification network and the limb vein images of registered users, the first image features and the second image features are identified respectively to obtain the first identification result; Based on the trained second classification network and the limb vein images of the registered user, the fused image features of the first image features and the second image features are identified to obtain a second identification result; Based on the first identification result and the second identification result, the user identification result of the user to be identified is obtained; The step of identifying the first image features and the second image features based on the trained first classification network and the limb vein image of the registered user to obtain the first identification result includes: Based on the positive limb vein image corresponding to the limb vein image of the registered user, the first image feature is identified through the first classification network to obtain the positive image recognition result of the first image feature; Based on the negative limb vein image corresponding to the limb vein image of the registered user, the second image feature is identified through the first classification network to obtain the negative image recognition result of the second image feature; The first recognition result is obtained based on the positive image recognition result, the negative image recognition result, and the corresponding score weights; The step of obtaining the user identification result of the user to be identified based on the first identification result and the second identification result includes: Obtain the multimodal fusion weights; Based on the first identification result, the second identification result, and the multimodal fusion weights, the user identification result of the user to be identified is obtained; The step of extracting features from the positive vein image and the negative vein image using a trained feature extraction network to obtain the first image features of the positive vein image and the second image features of the negative vein image includes: The forward vein image is preprocessed by the first preprocessing network in the first feature extraction network to obtain a forward image to be extracted; the forward image to be extracted is subjected to weight matrix feature extraction by the first autoencoder network in the first feature extraction network to obtain a forward weight matrix feature; based on the forward weight matrix feature, the forward image to be extracted is subjected to image feature extraction by the first feature processing network in the first feature extraction network to obtain the first image feature; The negative vein image is preprocessed by the second preprocessing network in the second feature extraction network to obtain a negative image to be extracted; the negative image to be extracted is then subjected to weight matrix feature extraction by the second autoencoder network in the second feature extraction network to obtain negative weight matrix features; based on the negative weight matrix features, the negative image to be extracted is then subjected to image feature extraction by the second feature processing network in the second feature extraction network to obtain the second image features; The first feature extraction network and the second feature extraction network are connected in parallel and have the same network parameters.

2. The user identification method according to claim 1, characterized in that, Prior to the step of determining the positive and negative vein images based on the limb vein images of the user to be identified, the method further includes: When a smart home appliance is detected to be triggered, the infrared camera is enabled. The infrared camera is used to acquire images of the veins in the limbs of the user to be identified.

3. The user identification method according to any one of claims 1 to 2, characterized in that, Before the step of extracting features from the positive vein image and the negative vein image using the trained feature extraction network to obtain the first image features of the positive vein image and the second image features of the negative vein image, the method further includes: Obtain the encoder-decoder network to be trained; The encoder-decoder network to be trained is trained based on vein images in a unified database to obtain the trained encoder-decoder network. Based on the network parameters of the encoder in the trained encoder-decoder network, the network parameters of the autoencoder network are determined.

4. The user identification method according to claim 3, characterized in that, Also includes: A neural network to be trained is constructed; the neural network includes a first feature extraction network, a second feature extraction network, a first recognition network, a second recognition network, and an output network, wherein the first feature extraction network and the second feature extraction network are connected in parallel, and the first recognition network and the second recognition network are connected in parallel; the first feature extraction network includes a first preprocessing network, a first autoencoder network, and a first feature processing network connected in sequence, and the second feature extraction network includes a second preprocessing network, a second autoencoder network, and a second feature processing network connected in sequence, wherein the network parameters of the first preprocessing network and the second preprocessing network are the same, the network parameters of the first autoencoder network and the second autoencoder network are the same, and the network parameters of the first feature processing network and the second feature processing network are the same; the first recognition network includes multiple first classification networks with different network structures, and the second recognition network includes multiple second classification networks with different network structures; The neural network to be trained is trained based on vein images in the unified database to determine the network parameters of the first feature processing network, the network structure and network parameters of the first classification network and the second classification network.

5. A user identification device for implementing the user identification method according to any one of claims 1-4, characterized in that, The device is installed in smart home appliances; the device includes: The first module is used to determine the positive and negative vein images based on the limb vein images of the user to be identified; The second module is used to extract features from the positive vein image and the negative vein image through the trained feature extraction network to obtain the first image features of the positive vein image and the second image features of the negative vein image; The third module is used to identify the first image features and the second image features based on the trained first classification network and the limb vein images of the registered user, respectively, to obtain the first identification result; The fourth module is used to identify the fused image features of the first image features and the second image features based on the trained second classification network and the limb vein images of the registered user, and to obtain a second identification result; The fifth module is used to obtain the user identification result of the user to be identified based on the first identification result and the second identification result.

6. A smart home appliance, characterized in that, The smart home appliance includes a memory and a processor; the memory stores a computer program, which, when executed by the processor, implements the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the method as described in any one of claims 1 to 4.

Images