Intelligent door lock input method, intelligent door lock and computer readable medium

Abstract

The embodiment of the invention discloses an intelligent door lock input method, an intelligent door lock and a computer readable medium. A specific embodiment of the method comprises the following steps: in response to determining that the intelligent door lock enters an input mode, executing the following processing steps: controlling an acquisition device to acquire finger image information; key point extraction is conducted on the finger image information, finger back key point information is obtained, and the finger back key point information comprises finger information, finger back vein information and finger back vein information; and inputting the finger back key point information into a finger back information registration library. The implementation mode can be used for improving the safety of the intelligent door lock.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, to an intelligent door lock input method, an intelligent door lock, and a computer readable medium. BACKGROUND

[0002] The intelligent door lock based on biometric recognition is the current research focus. At present, when inputting information of the intelligent door lock, the commonly used way is to capture the fingerprint of a user to complete the input of the intelligent door lock.

[0003] However, when inputting information of the intelligent door lock by using the above way, a technical problem often exists that the fingerprint of a finger can not be recognized due to peeling or injury, which leads to difficulty in normal use of the intelligent door lock, and the fingerprint of the finger as an external feature is easy to be forged, thereby reducing the security of the intelligent door lock.

[0004] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present disclosure and, as such, it can contain information that does not form the prior art that is already known to those of ordinary skill in the art in this country. SUMMARY

[0005] This section provides a summary of the concepts disclosed herein, which will be described in more detail in the following detailed description. This summary is not intended to identify key or essential features of the claimed technology nor is it intended to be used to limit the scope of the claimed technology.

[0006] Some embodiments of the present disclosure provide an intelligent door lock input method, an intelligent door lock, and a computer readable medium to solve one or more of the technical problems mentioned in the BACKGROUND section.

[0007] In a first aspect, some embodiments of the present disclosure provide an intelligent door lock input method, which comprises: in response to determining that an intelligent door lock enters an input mode, performing the following processing steps: controlling a collection device to collect finger image information; performing key point extraction on the finger image information to obtain finger back key point information, wherein the finger back key point information comprises finger information, finger back vein information, and finger back ridge information; and inputting the finger back key point information into a finger back information registration library.

[0008] In a second aspect, some embodiments of the present disclosure provide an intelligent door lock, which comprises: a collection device configured to collect finger image information; a door lock; one or more processors; a storage device having one or more programs stored thereon; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0009] In a third aspect, some embodiments of the present disclosure provide a computer readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method described in any implementation manner of the first aspect.

[0010] The above various embodiments of the present disclosure have the following beneficial effects: the smart door lock entry method of some embodiments of the present disclosure can improve the security of the smart door lock. Specifically, the reason for the reduced security of the smart door lock is that the finger fingerprint may not be recognized due to peeling, injury, etc., resulting in difficulty in normal use of the smart door lock, and the finger fingerprint as an external feature is easy to be forged. Based on this, the smart door lock entry method of some embodiments of the present disclosure, first, in response to determining that the smart door lock enters the entry mode, the following processing steps are performed: controlling the acquisition device to acquire finger image information. By acquiring the image information of the back of the finger, the situation that the door lock cannot be recognized due to the peeling, injury, etc. of the fingerprint is avoided, the influence of the external environment is reduced, and the stability and accuracy of the registration information are improved. Then, the key point extraction is performed on the above-mentioned finger image information to obtain the back of the finger key point information. The back of the finger key point information includes finger information, back of the finger vein information and back of the finger print information. Thus, the detailed features of the back of the finger can be extracted, including the vein information and texture information of the back of the finger, and the finger vein information is more difficult to forge than the fingerprint, which can improve the anti-tamper ability of the smart door lock. Finally, the back of the finger key point information is entered into the back of the finger information registration library. Thus, in subsequent use, accurate identification can be performed using these features that are not easily damaged and difficult to forge, effectively improving the security of the smart door lock. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, same or similar reference numerals can represent same or similar elements. It should be understood that the drawings are schematic and elements and features are not necessarily drawn to scale.

[0012] Figure 1 is a flowchart of some embodiments of the smart door lock entry method according to the present disclosure;

[0013] Figure 2 is a structural schematic diagram of some embodiments of the smart door lock according to the present disclosure. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] In addition, it should be further noted that only parts related to the present application are shown in the drawings for ease of description. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0016] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the adjectives "one", "multiple" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0019] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0020] Figure 1 The flow 100 of some embodiments of the smart door lock entry method according to the present disclosure is shown. The smart door lock entry method includes the following steps:

[0021] Step 101, in response to determining that the smart door lock enters the entry mode, the following processing steps are performed:

[0022] Step 1011, control the acquisition device to acquire the finger image information.

[0023] In some embodiments, the execution subject (e.g., a computing device) of the smart door lock entry method can control the acquisition device to acquire the finger image information. The smart door lock can include, but is not limited to, a door lock device, a touch display, a handle, an acquisition device, and an induction sensor. The acquisition device can be a device used to acquire finger image information. The acquisition device can include, but is not limited to, an infrared lamp and a camera. The induction sensor can be a device used to sense whether the user holds the handle of the smart door lock. The induction sensor can be a capacitive sensor. The acquisition device and the induction sensor can be in communication with the execution subject. In practice, first, the user can initiate an entry instruction through a mobile device wirelessly connected to the smart door lock or through the touch display of the smart door lock. Second, when the user holds the handle of the smart door lock, the induction sensor can generate an electrical signal. Then, when the execution subject receives the entry instruction and the electrical signal, it determines that the smart door lock enters an entry mode. Here, the execution subject can control the infrared lamp included in the acquisition device to expose and then control the camera to take pictures to obtain the finger dorsal vein image information. Second, the execution subject can also not control the infrared lamp included in the acquisition device to expose, but directly control the camera to take pictures to obtain the finger dorsal image information. The finger dorsal vein image information can be an image representing the user's finger vein information. The finger dorsal image information can be an image representing the user's finger dorsal information. The finger image information can include finger dorsal vein image information and finger dorsal image information, or image information obtained by superimposing finger dorsal image information and finger dorsal vein image information.

[0024] In practice, when the finger image information includes finger dorsal vein image information and finger dorsal image information, the following key point extraction and segmentation processing of the finger image information represent processing of the finger dorsal vein image information and the finger dorsal image information included in the finger image information, respectively.

[0025] Specifically, the entry mode can be a set of steps for performing user information entry. The entry mode can include the following processing steps: controlling the acquisition device to acquire the finger image information; extracting key points from the finger image information to obtain finger dorsal key point information; and entering the finger dorsal key point information into a finger dorsal information registration library.

[0026] In some optional implementations of some embodiments, after the execution subject controls the acquisition device to acquire the finger image information, it can further perform the following steps:

[0027] The first step is to perform skin color detection on the above-mentioned finger image information to obtain finger skin color information. The above-mentioned finger skin color information can be a numerical value that characterizes the skin color of the user's finger. In practice, first, the acquisition device can be controlled to collect information on the target area to obtain initial finger image information. The above-mentioned target area can be the area directly facing the camera included in the above-mentioned acquisition device. The above-mentioned target area can also be the area where the user places his finger. Specifically, the camera included in the acquisition device can be controlled to shoot the above-mentioned target area to obtain initial finger image information. The above-mentioned initial finger image information can be an image that characterizes the back information of the user's finger. Secondly, the above-mentioned initial finger image information can be subjected to skin color detection by a preset skin color detection algorithm to obtain finger skin color information.

[0028] As an example, the skin color detection algorithm may include but is not limited to at least one of the following: an adaptive skin color detector (AdaptiveSkinDetector), a threshold skin color recognition based on the HSV color space, and the like.

[0029] In the second step, in response to determining that the finger skin color information is higher than a preset depth threshold, fill light is applied to the target area, and the acquisition device is again controlled to acquire finger image information. The acquisition device may further include a fill light. The depth threshold may be numerical data. The numerical value of the depth threshold is not specifically limited. In practice, when it is determined that the finger skin color information is higher than a preset depth threshold, the fill light may be controlled to expose the target area. Subsequently, the camera included in the acquisition device may be controlled to capture the target area again to obtain finger image information.

[0030] In practice, for users with darker skin, the collected finger back images may be darker, which will affect the quality of subsequent feature extraction and may cause inaccurate registration information, thereby reducing the recognition effect and reliability of the smart door lock.

[0031] In some other optional implementations of some embodiments, after controlling the acquisition device to acquire finger image information, the execution subject may further perform the following steps:

[0032] In response to determining that the finger image information contains a scar, a ring, and / or the target user age information is less than a preset age threshold, a special entry mode is entered. The target user age information is the age information of the user corresponding to the finger image information. In practice, first, it can be determined whether the finger image information contains a scar and a ring (ring). Second, when the user initiates the entry instruction, the user can enter the user age information through the mobile terminal device wirelessly connected to the smart door lock or the touch display screen. The execution subject can determine the user age information as the target user age information. Then, when the finger image information contains a scar or a ring, the special entry mode can be entered. At the same time, when the target user age information is less than the preset age threshold, the special entry mode can be entered.

[0033] Specifically, the special entry mode can be a set of steps for performing special user information entry. The special entry mode can include the following processing steps: controlling the acquisition device to acquire finger image information; performing special key point extraction on the finger image information to obtain special finger back key point information; and entering the special finger back key point information into the finger back information registration library. The special key point extraction step has the same structure as the key point extraction step corresponding to the entry mode, but the parameters can be different. Second, when the user corresponding to the special finger back key point information uses his / her finger to unlock, the smart door lock can self-learn the features of the finger and update the corresponding special finger back key point information in the finger back information registration library. In this way, the influence of changes in finger features such as scar fading and child growth can be reduced, and the accuracy of smart door lock recognition can be improved.

[0034] Optionally, the execution subject determining whether the finger image information contains a scar and a ring can include the following steps:

[0035] The finger image information is subjected to abnormal target detection to obtain an abnormal target detection result. In practice, first, the finger image information can be subjected to abnormal target detection by a pre-trained abnormal target detection model to obtain an abnormal target detection result. The abnormal target detection model can be a target detection model for detecting whether the user's finger is abnormal. The abnormal target detection model can be obtained by training a target detection base model by using an abnormal target data set. The target detection base model can be a YOLO model or a Faster R-CNN model. Each abnormal target data in the abnormal target data set includes an abnormal target picture and a corresponding abnormal label. For example, if the finger in the abnormal target picture wears a ring, the corresponding abnormal label is "abnormal, ring". When the finger has a scar in the abnormal target picture, the corresponding abnormal label can be "abnormal, scar". The abnormal target detection result can include an abnormal state and an abnormal label. The abnormal state can be abnormal or normal. The abnormal label can be a null value or an abnormal factor. The abnormal factor can include but is not limited to a scar, a ring, etc. Finally, when the abnormal label corresponding to the abnormal target detection result is a scar or a ring, it is determined that the finger image information contains a scar or a ring.

[0036] In some other optional implementations of some embodiments, the execution subject can further perform the following steps:

[0037] In the first step, an entry period is generated according to the finger image information and / or the target user age information. The entry period can be a scar entry period and / or a growth entry period. In practice, when the abnormal label contained in the abnormal target detection result corresponding to the finger image information is a scar, a scar entry period can be generated. Here, the scar entry period can be a preset time period. The time period can be 3 months, which is not limited herein. Meanwhile, when the target user age information is less than a preset age threshold, a growth entry period can be generated according to a preset age period mapping table. The age threshold can be a numerical value representing age. The age threshold can be 18, which is not limited herein. The age period mapping table can be a data table of the corresponding relationship between a preset user age and a growth period. For example, the content of the age period mapping table can include "6 years old: 1 year", "15 years old: 3 years", etc.

[0038] Secondly, in response to determining that the current time point corresponds to the above-mentioned input period, an update input prompt information is sent to the user terminal. In practice, first, the time point corresponding to the operation of collecting the finger image information by the control collection device is added to the above-mentioned input period to obtain an update time point. Then, when it is determined that the current time point of the smart door lock is equal to the update time point, the update input prompt information can be sent to the user terminal. The user terminal can be the mobile terminal device. The update input prompt information can be information reminding the user to re-perform the smart door lock registration input.

[0039] In some other optional implementations of some embodiments, the execution subject can further perform the following steps after collecting the finger image information by the control collection device:

[0040] Firstly, the finger image information is pre-processed to obtain a pre-processed finger back image. The pre-processed finger back image can be the finger image information after pre-processing. In practice, the finger image information can be pre-processed by a pre-set pre-processing algorithm to obtain the pre-processed finger back image.

[0041] As an example, the pre-processing algorithm can include but is not limited to at least one of the following: Gaussian filtering, median filtering, bilateral filtering, etc.

[0042] Secondly, the pre-processed finger back image is edge detected to obtain a finger back edge image. The finger back edge image can be the pre-processed finger back image after edge detection. In practice, the pre-processed finger back image can be edge detected by a pre-set edge detection algorithm to obtain the finger back edge image.

[0043] As an example, the edge detection algorithm can include but is not limited to at least one of the following: Sobel operator, Canny operator, etc.

[0044] Thirdly, the finger back edge image is sharpness detected to obtain finger back sharpness information. The finger back sharpness information can be a numerical value for representing the sharpness of the finger back edge image. In practice, the finger back edge image can be sharpness detected by Laplacian or LoG to obtain the finger back sharpness information.

[0045] In the fourth step, the finger image quality information corresponding to the finger image information is generated according to the finger back sharpness information. The finger image quality information can be label information representing the quality of the finger image information. In practice, when the finger back sharpness information is less than or equal to a preset sharpness threshold, the finger image quality information corresponding to the finger image information is determined to be good. When the finger back sharpness information is greater than the preset sharpness threshold, the finger image quality information corresponding to the finger image information is determined to be poor. The sharpness threshold can be a preset value, which is not limited herein.

[0046] In the fifth step, in response to determining that the finger image quality information does not meet a preset quality condition, an adjustment finger posture instruction is sent, and the processing steps are executed again. The preset quality condition can be that the finger image quality information is good. In practice, when the finger image quality information is poor, a preset adjustment finger posture instruction can be displayed on the mobile terminal or the touch display to remind the user to adjust the finger posture. The adjustment finger posture instruction can be information reminding the user to replay the finger. The processing steps can be executed again to re-enter the information.

[0047] Alternatively, when the finger image quality information is poor, the acquisition device can also be adjusted. The processing steps can be executed again to re-enter the information. The finger back sharpness information corresponding to the acquisition device parameters can be determined by a preset sharpness parameter mapping table. The sharpness parameter mapping table can be a data table representing the correspondence between the preset sharpness values and the acquisition device parameters. The acquisition device parameters can include, but are not limited to, infrared lamp radiation illuminance, camera focal length, camera aperture, camera shutter time, and light illuminance.

[0048] In step 1012, key points are extracted from the finger image information to obtain back key point information.

[0049] In some embodiments, the execution subject can perform key point extraction on the finger image information to obtain finger back key point information. The finger back key point information can include finger information, finger back vein information, and finger back line information. The finger information can be label information used to identify a user's finger (e.g., index finger, ring finger). The finger back vein information can be information representing the characteristics of the finger back vein. The finger back vein information can be represented by a vector. The finger back line information can be information representing the characteristics of the finger back line. The finger back line information can be represented by a vector. In practice, first, the K-Nearest Neighbor Graph Iteration Vein Recognition Algorithm can be used to determine the finger back vein information corresponding to the finger back vein image information included in the finger image information. Second, a texture feature-based extraction algorithm can be used to determine the finger back line information corresponding to the finger back image information included in the finger image information. Then, the preset finger label information can be obtained as the finger information corresponding to the finger image information. Here, the finger label information can be information uploaded by the user through the mobile terminal or through the touch display screen. Finally, the finger information, the finger back vein information, and the finger back line information can be determined as the finger back key point information.

[0050] In practice, the key point extraction algorithm corresponds to adjustable parameters. For example, when the key point extraction algorithm is a texture feature-based extraction algorithm, the corresponding adjustable parameters can include filter size, key point threshold (used to determine whether the texture feature is significant enough to be a key point), etc. When the key point extraction algorithm is a K-Nearest Neighbor Graph Iteration Vein Recognition Algorithm, the corresponding adjustable parameters can include K value, distance threshold, iteration number, neighborhood size, etc. The adjustable parameters corresponding to the special entry mode are different from the adjustable parameters corresponding to the entry mode. The adjustable parameters corresponding to the special entry mode can make the finger back key point information obtained by the special entry mode have a larger data volume and be more accurate than the finger back key point information obtained by the entry mode. For example, the special entry mode can correspond to a larger K value, a larger iteration number, etc.

[0051] In some optional implementations of some embodiments, the execution subject performing key point extraction on the finger image information to obtain finger back key point information can include the following steps:

[0052] In a first step, in response to determining that the finger image information includes at least two finger information, the finger image information is segmented to obtain a single finger image information set. In practice, the user can place multiple fingers in the target area at one time, or hold the handle included in the smart door lock by multiple fingers, so that the finger image information collected by the execution body can include at least two finger information. First, when the finger image information includes at least two finger information, at least one boundary line between fingers can be determined by a preset edge detection algorithm. The edge detection algorithm can be a Canny edge detection algorithm. The boundary line between fingers can be represented by a line segment equation. Here, the finger image information can be a two-dimensional plane, and the boundary line can be a line segment in the two-dimensional plane. Then, the finger image information can be segmented according to the boundary between the at least one finger and the finger to obtain a single finger image information set. Each single finger image information in the single finger image information set can be an image representing single finger information segmented from the finger image information.

[0053] In a second step, key points are extracted from each single finger image information in the single finger image information set to obtain finger back key point information. The finger back key point information can include each single finger key point information. The finger back key point information can be information representing characteristics of each finger. In practice, first, each single finger image information in the single finger image information set can be segmented into a knuckle image group by a preset knuckle segmentation algorithm to generate a knuckle image group set. Here, a single finger image information corresponds to a knuckle image group. Second, for each knuckle image group in the knuckle image group set, key points can be extracted from each knuckle image in the knuckle image group by a preset key point extraction algorithm to generate a knuckle key point information group. Then, each knuckle key point information group in the generated knuckle key point information group can be determined as a single finger key point information to obtain a single finger key point information set. Finally, the single finger key point information set can be determined as the finger back key point information.

[0054] As an example, the key point extraction algorithm can include, but is not limited to, at least one of the following: a texture feature-based extraction algorithm, a K-neighbor graph iterative vein recognition algorithm, etc.

[0055] Optionally, the execution body controls the acquisition device to collect finger image information, and extracts key points from the finger image information to obtain finger back key point information, which can include the following steps:

[0056] In a first step, the acquisition device is controlled to collect finger image information.

[0057] Secondly, in response to determining that the finger image information satisfies the preset segmentation collection condition, the collection device is controlled to collect at least once to obtain each finger image information. In practice, firstly, the texture feature extraction algorithm based on texture features can be used to extract the texture features of the finger image information to obtain a texture feature vector. Secondly, when the texture feature vector satisfies the preset segmentation collection condition, the collection device is controlled to collect at least once to obtain each finger image information. The segmentation collection condition can be that the length of the texture feature vector corresponding to the finger image information is less than a preset value, i.e., the distance between the user's finger and the camera included in the collection device is relatively close.

[0058] Here, the smart door lock can include a sound device. The execution subject can prompt the user to constantly change the part of holding the handle of the smart door lock through the sound device. Each time the user changes the finger part, the collection device can be controlled to collect to obtain finger image information.

[0059] Thirdly, each finger image information is combined to obtain finger back image information. In practice, the image stitching technology can be used to combine each finger image information to obtain finger back image information.

[0060] Fourthly, the finger image of the finger back image information is cut to obtain at least one single finger back image information. In practice, the finger image cutting operation in the first step of the optional implementation mode can be used to cut the finger image of the finger back image information to obtain at least one single finger back image information. Details are not repeated here.

[0061] Fifthly, the at least one single finger back image information is subjected to finger back vein information extraction to obtain a finger back vein information set. In practice, the finger back vein information extraction algorithm can be used to extract the finger back vein information of each single finger back image information in the at least one single finger back image information to obtain a finger back vein information set. The finger back vein information extraction algorithm can be a K-neighbor graph iterative vein recognition algorithm.

[0062] Sixthly, the at least one single finger back image information is subjected to finger back vein extraction to obtain a finger back vein information set. In practice, the finger back vein extraction algorithm can be used to extract the finger back vein information of each single finger back image information in the at least one single finger back image information to obtain a finger back vein information set. The finger back vein extraction algorithm can be a texture feature-based extraction algorithm.

[0063] In the seventh step, the at least one single finger back image information is classified to obtain a finger information set. In practice, the finger information corresponding to each single finger back image information in the at least one single finger back image information can be determined according to the position of each single finger back image information in the original finger back image information, and the finger information set is obtained. For example, the finger information corresponding to the uppermost single finger back image information in the finger back image information can be the index finger, and so on.

[0064] In the eighth step, the finger back key point information is generated according to the finger information set, the finger back vein information set, and the finger back print information set. In practice, the finger back key point information can be determined according to the finger information set, the finger back vein information set, and the finger back print information set.

[0065] In practice, when the above technical solutions are adopted, the following technical problems are often encountered: when only the finger vein features or the finger back print features are used, the security of the intelligent door lock is often low, and at the same time, due to the high complexity of the high-precision model, strong hardware support is often required, which leads to difficulty in reducing the size and parameter amount of the model while ensuring the quality of user information input under limited computing resources. Therefore, the following solutions can be adopted.

[0066] Optionally, the finger image information can include finger back vein image information and finger back image information, and the key point extraction of the finger image information by the execution subject to obtain the finger back key point information can include the following steps:

[0067] In the first step, the finger back vein image information and the finger back image information are respectively subjected to a texture segmentation to obtain a finger back vein segmentation image and a finger back print segmentation image. The finger back vein segmentation image can be a gray image representing only the finger back vein texture. The finger back print segmentation image can be a gray image representing only the finger back finger texture. In practice, the finger back vein segmentation image and the finger back print segmentation image can be obtained by respectively performing texture segmentation on the finger back vein image information and the finger back image information through a preset segmentation algorithm. Here, the segmentation algorithm can be a U-Net algorithm.

[0068] In the second step, based on the pre-trained finger back key point extraction model, the following steps are performed:

[0069] The first sub-step is to extract the finger back cross-line features from the finger back ridge segmentation image to obtain a finger back cross-line feature map. The pre-trained finger back key point extraction model can include a feature extraction layer, a feature fusion layer, a multi-scale convolution layer, a global feature extraction layer, and a feature embedding layer. The finger back cross-line feature map can be a two-dimensional matrix representing the finger back cross-line features of the user's finger. In practice, the feature extraction layer can be used to extract the finger back cross-line features from the finger back ridge segmentation image to obtain the finger back cross-line feature map. The feature extraction layer can be a convolutional neural network, such as ResNet or MobileNet.

[0070] The second sub-step is to extract the finger back vein features from the finger back vein segmentation image to obtain a finger back vein feature map. The finger back vein feature map can be a two-dimensional vector representing the vein features of the user's finger. In practice, the feature extraction layer can be used to extract the finger back vein features from the finger back vein segmentation image to obtain the finger back vein feature map.

[0071] The third sub-step is to fuse the finger back cross-line feature map and the finger back vein feature map to obtain a finger back multi-modal feature map. The finger back multi-modal feature map can be represented by a two-dimensional matrix. In practice, the feature fusion layer can be used to concatenate the finger back cross-line feature map and the finger back vein feature map to obtain the finger back multi-modal feature.

[0072] The fourth sub-step is to perform multi-scale convolution on the finger back multi-modal feature map to obtain a multi-scale finger back feature map. The multi-scale finger back feature map can be represented by a two-dimensional matrix. In practice, the multi-scale convolution layer can be used to perform multi-scale convolution on the finger back multi-modal feature map to obtain the multi-scale finger back feature map. Specifically, first, different scale convolutions can be performed on the finger back multi-modal feature map to obtain a corresponding set of convolution feature maps. The different scale convolutions can include a 3x3 convolution kernel and a 5x5 convolution kernel. Each convolution feature map in the set can be represented by a matrix, and each convolution feature map has the same size. Then, the element-wise sum of each convolution feature map in the set can be obtained to obtain the multi-scale finger back feature map.

[0073] A fifth sub-step is to determine a global feature map corresponding to the multi-scale finger back feature map. The global feature map can be represented by a two-dimensional matrix. In practice, the global feature map corresponding to the multi-scale finger back feature map can be determined by the global feature extraction layer. Specifically, first, a weight matrix corresponding to the multi-scale finger back feature map can be determined by a preset attention network. The attention network can be a CBAM network. Second, a matrix product between each convolutional feature in the set of convolutional features and the weight matrix is determined as a weighted feature map, obtaining a set of weighted feature maps. Then, each weighted feature map in the set of weighted feature maps can be element-wise added to obtain a global feature map.

[0074] A sixth sub-step is to perform feature embedding on the global feature to obtain finger back key point information. In practice, first, the global feature can be embedded by the feature embedding layer to obtain a finger back feature embedding vector. The feature embedding layer can be a fully connected layer (Fully Connected Layer). The finger back feature embedding vector can represent finger back vein information and finger back print information. Then, a preset finger label information can be obtained as finger information corresponding to the finger image information. Here, the finger label information can be information uploaded by the user through the mobile terminal or through the touch display screen. Finally, the finger information and the finger back feature embedding vector can be determined as the finger back key point information.

[0075] Optionally, the finger back key point extraction model can be trained by the following steps:

[0076] A first step is to obtain a finger back sample data set. Each finger back sample data in the finger back sample data set can include a sample finger sequence number, a sample finger back image, and a sample finger back vein image. The sample finger back image can be an image corresponding to the finger back. The sample finger back vein image can be an image obtained by collecting the finger back vein.

[0077] In practice, one sample finger sequence number can correspond to multiple sample finger back image and sample finger back vein image. The multiple sample finger back image and sample finger back vein image can be images obtained by photographing finger back at different angles. When the sample finger sequence number corresponding to each finger back sample data in two finger back sample data is the same, the two finger back sample data can form a positive sample pair. When the sample finger sequence number corresponding to each finger back sample data in two finger back sample data is different, the two finger back sample data can form a negative sample pair. Each finger back sample data in the finger back sample data set has corresponding positive sample and negative sample. The training process of the finger back key point extraction model can be understood as maximizing the distance between the negative sample pairs and minimizing the distance between the positive sample pairs.

[0078] In the second step, the initial finger back key point extraction model is simplified to obtain a student model corresponding to the initial finger back key point extraction model. The initial finger back key point extraction model can have the same structure as the pre-trained finger back key point extraction model but different parameters. The initial finger back key point extraction model can include an initial feature extraction layer, an initial feature fusion layer, an initial multi-scale convolution layer, an initial global feature extraction layer, and an initial feature embedding layer. The initial finger back key point extraction model can be an initial finger back key point extraction model pre-trained by the finger back sample data set. The student model can be a lightweight model corresponding to the initial finger back key point extraction model. In practice, the initial finger back key point extraction model can be simplified by a preset network simplification technique to obtain the student model corresponding to the initial finger back key point extraction model.

[0079] As an example, the network simplification technique can include but is not limited to at least one of the following: replacing the original base network with a lighter base network, parameter quantization, etc.

[0080] In the third step, for each finger back sample data in the finger back sample data set, the following training steps are performed:

[0081] In the first sub-step, the teacher sample feature embedding corresponding to the finger back sample data is determined based on the initial finger back key point extraction model. In practice, the teacher sample feature embedding corresponding to the finger back sample data can be determined based on the initial finger back key point extraction model by the operations corresponding to the first to sixth sub-steps in the optional solution. Details are not repeated here.

[0082] A second sub-step is to determine a student sample feature embedding corresponding to the finger dorsal sample data based on the student model. In practice, the student sample feature embedding corresponding to the finger dorsal sample data can be determined based on the student model through the operations corresponding to the first to sixth sub-steps in the optional solution. Details are not described herein.

[0083] A third sub-step is to generate a triplet loss value according to a preset triplet loss function and the student sample feature embedding. The triplet loss function can be as follows:

[0084]

[0085] wherein, L S represents the triplet loss value. d o represents the Euclidean distance between vectors. d p represents the difference between the positive sample pair distance and the negative sample pair distance. S 0 represents the student sample feature embedding. S + represents the feature embedding of the positive sample corresponding to the finger dorsal sample data obtained through the student model. S - represents the feature embedding of the negative sample corresponding to the finger dorsal sample data obtained through the student model. τ1 and τ2 can be preset negative numbers, which are not limited herein. For example, τ1 can be -0.5 and τ2 can be -0.1. β is a hyperparameter for balancing the constraints between the negative sample and the positive sample, which can be obtained through cross-validation dataset training. The cross-validation dataset can be a set composed of finger dorsal sample data used for verification training.

[0086] As an example, assuming that β is 0.5. When the positive sample pair distance is much smaller than the negative sample pair distance, d p corresponding value is much smaller than τ1. L S is the sum between τ1 and 0.5 times the positive sample pair distance. When L S is negative, it means that the current sample pair meets the training requirements (the positive sample pair distance is much smaller than the negative sample pair distance).

[0087] When the positive sample pair distance is greater than the negative sample pair distance, d p corresponding value is positive and greater than τ1. L S is the sum between d p and 0.5 times the positive sample pair distance. L S is positive, indicating that the current sample pair does not meet the training requirements and needs to be updated according to the gradient descent parameter.

[0088] In practice, by the triplet loss function, the distance between the positive sample pairs is less than the distance between the negative sample pairs. Thus, the feature embeddings corresponding to different fingers are more obviously distinguished, thereby improving the effectiveness and security of the smart door lock registration process. Meanwhile, the triplet loss function can also constrain the distance between the positive sample pairs by τ2, so that the feature vectors of all the fingerprint images of the same finger in the feature space are more clustered by the student model, further enhancing the distinguishing ability of the model.

[0089] In the fourth sub-step, a distillation loss value is generated according to the teacher sample feature embedding and the student sample feature embedding. In practice, the mean square error (MSE) between the teacher sample feature embedding and the student sample feature embedding can be determined as the distillation loss value.

[0090] In the fifth sub-step, a joint loss value corresponding to the triplet loss value and the distillation loss value is determined. In practice, a weighted sum between the triplet loss value and the distillation loss value can be determined as the joint loss value. The weights used in the weighted sum can be obtained by training the cross-validation dataset.

[0091] In the sixth sub-step, the student model is trained by back propagation according to the joint loss value, an updated student model is obtained, and the updated student model is used as the student model to perform the training steps again.

[0092] In the fourth step, the obtained updated student model is determined as the finger back key point extraction model. In practice, the updated student model obtained in the third step can be determined as the finger back key point extraction model.

[0093] The optional step and its related content are an application point of one embodiment of the present disclosure, which solves the technical problem of being difficult to reduce the size and parameter quantity of the model while ensuring the quality of user information entry under limited computing resources. The factors that cause the above technical problem are often as follows: when only using the finger vein feature or the finger backprint feature, the security of the smart door lock is often low, and since the high-precision model has high complexity, it often needs strong hardware support. If the above factors are solved, the size and parameter quantity of the model can be reduced while ensuring the quality of user information entry. In order to achieve this effect, first, the original finger vein and finger back image is segmented to obtain a gray image that only retains vein and ridge features, which can avoid the interference of invalid data such as background in the original image on subsequent feature extraction. Secondly, the finger vein feature and the finger back cross-line feature are effectively combined through feature fusion and multi-scale convolution, capturing the details and global features of the finger vein and the finger back cross-line at different scales, improving the robustness and generalization ability of the model, and reducing the over-reliance on single-scale information. Then, the attention mechanism is used to determine the global feature map corresponding to the multi-scale finger back feature map, which can reduce the influence of irrelevant features and improve the accuracy of feature selection. Among them, the knowledge distillation is performed on the initial finger back key point extraction model to reduce the parameter quantity and computational complexity of the model while maintaining high accuracy. Specifically, the student model is trained using a joint loss, the triplet loss can enhance the discriminability between feature embeddings corresponding to different fingers and reduce the distance between feature embeddings corresponding to the same finger, thereby improving the discrimination ability of the model; the distillation loss can make the student model imitate the performance of the initial finger back key point extraction model and further reduce the complexity of the model. Through the joint training of the two losses, the size of the student model can be effectively reduced while retaining efficient feature expression capability. The pre-trained finger back key point extraction model has lower parameter quantity and faster inference speed, and is suitable for use in resource-limited environments.

[0094] In step 1013, the finger back key point information is entered into a finger back information registration library.

[0095] In some embodiments, the execution subject can enter the finger back key point information into the finger back information registration library. The finger back information registration library can be a database for storing finger back key point information. The smart door lock can include a memory. The database can be stored in the memory.

[0096] The above various embodiments of the present disclosure have the following beneficial effects: the smart door lock entry method of some embodiments of the present disclosure can improve the security of the smart door lock. Specifically, the reason for the reduced security of the smart door lock is that the finger print may not be recognized due to peeling, injury, etc., causing the smart door lock to be difficult to use normally, and the finger print as an external feature is easy to be forged. Based on this, the smart door lock entry method of some embodiments of the present disclosure, first, in response to determining that the smart door lock enters the entry mode, the following processing steps are performed: controlling the acquisition device to acquire finger image information. By acquiring the image information of the back of the finger, the situation that the door lock cannot be recognized due to the peeling, injury, etc. of the fingerprint is avoided, the influence of the external environment is reduced, and the stability and accuracy of the registration information are improved. Next, the key points of the above finger image information are extracted to obtain the back of the finger key point information. The back of the finger key point information includes finger information, back of the finger vein information and back of the finger print information. Therefore, the detailed features of the back of the finger can be extracted, including the vein information and texture information of the back of the finger, and the finger vein information is more difficult to forge than the fingerprint, which can improve the anti-tamper ability of the smart door lock. Finally, the back of the finger key point information is entered into the back of the finger information registration library. Therefore, in subsequent use, accurate identification can be performed using these features that are not easily damaged and difficult to forge, effectively improving the security of the smart door lock.

[0097] Reference will now be made to Figure 2 which shows a structural schematic diagram of a smart door lock 200 suitable for implementing some embodiments of the present disclosure. Figure 2 The illustrated smart door lock is only an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0098] As shown in Figure 2 , the smart door lock 200 can include an acquisition device configured to acquire finger image information. The door lock can include a lock cylinder and a lock body. A processing device (such as a central processing unit, a graphics processing unit, etc.) 201 can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 202 or programs loaded from a storage device 208 into a random access memory (RAM) 203. In the RAM 203, various programs and data required for the operation of the smart door lock 200 are also stored. The processing device 201, the ROM 202, and the RAM 203 are connected to each other through a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0099] Typically, the following devices may be connected to the I / O interface 205: an input device 206 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 208 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 209. The communication device 209 may allow the smart door lock 200 to communicate with other devices wirelessly or by wire to exchange data. Figure 2 The smart door lock 200 with various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices can be implemented or have instead. Figure 2 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0100] The smart door lock 200 may include a collection device and a door lock. The collection device is configured to collect finger image information. The door lock may include a lock core and a lock body.

[0101] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 209, or installed from the storage device 208, or installed from the ROM 202. When the computer program is executed by the processing device 201, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0102] Note that the computer readable medium in some embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by an instruction execution system, apparatus or device, or that can be used by or in connection with an instruction execution system, apparatus or device. In some embodiments of the present disclosure, the computer readable signal medium can include a computer readable program code propagated in or on a carrier medium, in which the computer readable program code is embodied. Such propagated computer readable program code can take many forms, including but not limited to, an electromagnetic signal, an optical signal or any suitable combination of the foregoing. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable medium can be transmitted using any suitable medium, including but not limited to, wire, cable, wireless, RF, infrared or any suitable combination of the foregoing.

[0103] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0104] The computer readable medium can be included in the smart door lock, or can exist separately from the smart door lock. The computer readable medium carries one or more programs which, when executed by the smart door lock, cause the smart door lock to: in response to determining that the smart door lock enters an entry mode, perform the following processing steps: control the acquisition device to acquire finger image information; perform key point extraction on the finger image information to obtain finger back key point information, wherein the finger back key point information includes finger information, finger back vein information, and finger back ridge information; and enter the finger back key point information into a finger back information registration library.

[0105] Computer program code for carrying out operations of some embodiments of the present disclosure can be written in any one or more of a variety of programming languages or combinations of languages, including an object-oriented programming language such as Java, Smalltalk, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0106] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks depicted in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block and combination of blocks in the block diagrams and / or flow diagrams can be implemented by dedicated hardware-based systems which perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0107] The functionality described herein above can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program- specific Integrated Circuits (ASICs), Program- specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0108] The above description is merely exemplary of the disclosure and the application made use of the principles of the technology. It is to be understood that the application scope of the embodiments of the disclosure is not limited to the specific combinations of technical features described above, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features thereof without departing from the inventive concept. For example, the technical solutions formed by replacing the above features with technical features having similar functions disclosed in the embodiments of the disclosure (but not limited to) with each other.

Claims

1. A smart door lock registration method, comprising: In response to determining that the smart door lock enters the entry mode, the following processing steps are performed: Controlling the acquisition device to acquire finger image information; Extract key points from the finger image information to obtain finger back key point information, wherein the finger back key point information includes finger information, finger back vein information, and finger back print information; The finger back key point information is entered into a finger back information registration database.

2. The method according to claim 1, wherein After the control acquisition device acquires the finger image information, the method further includes: Performing skin color detection on the finger image information to obtain finger skin color information; In response to determining that the finger skin color information is higher than a preset depth threshold, fill light is applied to the target area, and the acquisition device is controlled again to acquire finger image information.

3. The method according to claim 1, wherein After the control acquisition device acquires the finger image information, the method further includes: In response to determining that the finger image information contains scars, rings and / or the target user's age information is less than a preset age threshold, a special entry mode is entered, wherein the target user's age information is the age information of the user corresponding to the finger image information.

4. The method according to claim 3, wherein: The method further comprises: generating an entry cycle according to the finger image information and / or the target user age information; In response to determining that the current time point corresponds to the entry period, update entry prompt information is sent to the user terminal.

5. The method according to claim 1, wherein The step of extracting key points from the finger image information to obtain finger back key point information includes: In response to determining that the finger image information includes at least two finger information, performing finger image segmentation on the finger image information to obtain a single finger image information set; Key point extraction is performed on each single finger image information in the single finger image information set to obtain finger back key point information, wherein the finger back key point information includes each single finger key point information.

6. The method according to claim 1, wherein The control acquisition device acquires finger image information; And extracting key points from the finger image information to obtain finger back key point information includes: Controlling the acquisition device to acquire finger image information; In response to determining that the finger image information meets the preset segmented acquisition condition, controlling the acquisition device to perform at least one acquisition to obtain each finger image information; Combining the image information of each finger to obtain finger back image information; Performing finger image segmentation on the finger back image information to obtain at least one single finger back image information; Extracting finger back vein information from the at least one single finger back image to obtain a finger back vein information set; Extracting finger back prints from the at least one single finger back image information to obtain a finger back print information set; performing finger information classification on the at least one single finger back image information to obtain a finger information set; Finger back key point information is generated according to the finger information set, the finger back vein information set and the finger back print information set.

7. The method according to any one of claims 1 to 6, wherein: After the control acquisition device acquires the finger image information, the method further includes: Preprocessing the finger image information to obtain a preprocessed finger back image; Performing edge detection on the preprocessed finger back image to obtain a finger back edge image; Performing sharpness detection on the finger back edge image to obtain finger back sharpness information; generating finger image quality information corresponding to the finger image information according to the finger back sharpness information; In response to determining that the finger image quality information does not meet the preset quality condition, sending an instruction to adjust the finger posture, and performing the processing step again.

8. A smart door lock, comprising: A collection device configured to collect finger image information; door locks; one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

9. A computer-readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A door, wherein The door includes the smart door lock as claimed in claim 8.

Images