Door lock control method, system and intelligent door lock

Abstract

The present invention is applicable to the technical field of smart door locks, and provides a door lock control method, system and smart door lock. The method includes using a binocular infrared camera to synchronously obtain a depth image and a palm print texture image of a user's palm, and constructing a video sequence for dynamic feature analysis to determine whether the user's palm is alive. If it is determined to be alive, a Gabor filter is applied to the palm print texture image of the current frame to extract a palm print texture feature vector, and a low-dimensional palm print feature vector is generated after a principal component analysis dimensionality reduction process, and cosine similarity is calculated with a preset palm print feature vector. At the same time, features are extracted from the depth image, and its Euclidean distance with the preset depth feature vector is calculated. The calculation results of the two are weighted to obtain a comprehensive evaluation result of palm print recognition, and whether to perform the unlocking operation of the door lock is determined according to the comprehensive evaluation result. The present invention solves the problem that existing door locks cannot simultaneously meet security, user experience and adaptability to complex environments.

Description

Technical Field

[0001] The present invention relates to the technical field of smart door locks, and in particular to a door lock control method, system and a smart door lock. Background Art

[0002] With the development of smart home, smart door locks, as an important part of smart home, have been widely used and developed rapidly. At present, there are diversified smart door lock products on the market, which adopt technologies including fingerprint recognition, password lock, RFID (radio frequency identification) and face recognition. These technologies have different advantages and meet users' needs for door lock security, convenience and intelligence.

[0003] Among them, fingerprint recognition technology has gradually become an important part of smart door locks due to its relatively high security and convenience. Users can quickly unlock the door lock by simply entering their fingerprints, avoiding the inconvenience of carrying physical keys. Password lock technology is widely used in homes and offices due to its simplicity and ease of use. Users can unlock the door lock by entering a preset password, avoiding the risk of losing the physical key. RFID technology is mainly used in hotels, apartments and other scenarios, providing users with a fast and convenient unlocking experience. In recent years, face recognition technology, as an emerging biometric technology, has also gradually entered the smart door lock market. It uses cameras to capture the user's facial features for identity authentication, providing users with a more intelligent and contactless unlocking method.

[0004] However, these technologies still have certain limitations and shortcomings in practical applications. Among them, fingerprint recognition is limited by fingerprint quality and environmental conditions. For example, when the user's fingerprint is dry, stained or wet, the recognition rate drops significantly, which may lead to unlocking failure. In addition, in some cases, fingerprint recognition is also at risk of being imitated or copied, and it is difficult to ensure absolute security. Although the password lock technology is simple to operate, it faces problems such as password forgetting, password leakage and complex password management, which seriously affect the user experience. In addition, the password is easy to be cracked by brute force or obtained by peeping, which poses a great security risk. Although RFID technology is convenient, RFID cards are easy to be lost, stolen or illegally copied, and there is a risk of being used maliciously, resulting in low security. In addition, RFID technology relies on physical media, and users must carry the card with them. If it is lost or damaged, it cannot be unlocked normally, affecting the user experience. The practical application of face recognition technology still faces some technical challenges. For example, when the light is poor or the user's face angle is not suitable, the recognition effect may drop significantly. At the same time, factors such as facial expression changes, age increase and makeup will also affect the recognition accuracy. In addition, face recognition involves the collection and processing of user privacy data. If it is not properly protected, it may cause the risk of privacy leakage.

[0005] In summary, although the existing smart door lock technology has certain convenience and intelligence, it still has many shortcomings in terms of security, convenience, user experience, and stability and reliability in complex environments. Fingerprints can be forged, passwords can be easily cracked, RFID cards can be easily copied, and face recognition performs poorly in harsh environments. The security of these technologies cannot fully protect the property and privacy of users in the face of complex attack methods. In addition, at the user experience level, problems such as the need to reset the password when it is forgotten and the inconvenience of carrying RFID cards have affected the convenience of the technology. In the face of complex environments (such as high temperature, high humidity, low light, etc.), the stability and reliability of existing technologies are also insufficient, and continuous and effective work cannot be guaranteed. These problems have brought inconvenience to users in practical applications and restricted the further development of smart door lock technology.

[0006] Therefore, in view of the limitations and defects of these existing technologies, how to further improve user experience and improve system reliability and stability while ensuring the security of door locks has become a technical challenge that needs to be urgently solved in the field of smart door lock technology. There is an urgent need for a new smart door lock technology solution that can improve security, improve user experience, and adapt to complex environments. Summary of the invention

[0007] Based on this, the purpose of the present invention is to provide a door lock control method, system and intelligent door lock, so as to fundamentally solve the problem that existing door locks cannot simultaneously meet security, user experience and adapt to complex environments.

[0008] A door lock control method according to an embodiment of the present invention includes:

[0009] Use a binocular infrared camera to synchronously obtain a depth image and a palm print texture image of the user's palm;

[0010] The depth of field images and palm print texture images acquired within a preset time period are constructed into a video sequence, and dynamic feature analysis is performed on the video sequence to determine whether the user's palm is alive;

[0011] If the user's palm is judged to be alive, a Gabor filter is applied to the palm print texture image of the current frame to extract the palm print texture feature vectors in multiple scales and directions;

[0012] Performing principal component analysis and dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector, and calculating the cosine similarity between the generated palmprint feature vector and the preset palmprint feature vector;

[0013] Extract features from the depth image of the current frame, generate a depth feature vector, and calculate the Euclidean distance between the generated depth feature vector and a preset depth feature vector;

[0014] A weighted calculation is performed based on the calculated cosine similarity and Euclidean distance to obtain a comprehensive evaluation result of palmprint recognition;

[0015] According to the comprehensive evaluation results of palm print recognition, it is determined whether to perform the door lock unlocking operation.

[0016] In addition, a door lock control method according to the above embodiment of the present invention may also have the following additional technical features:

[0017] Furthermore, the step of synchronously acquiring the depth of field image and the palm texture image of the user's palm using a binocular infrared camera includes:

[0018] Use a binocular infrared camera to capture the current image in real time and detect whether the user's palm area is captured;

[0019] If the user's palm area is detected, two cameras in the binocular infrared camera are used to collect two sets of images from different perspectives, and a depth of field image of the user's palm is generated based on the parallax calculation of the two sets of images;

[0020] Use one of the binocular infrared cameras to collect a palm print texture image of the user's palm.

[0021] Furthermore, the step of detecting whether the user's palm area is collected includes:

[0022] Preprocess the acquired current image, convert it into a grayscale image and perform denoising;

[0023] Perform skin color detection and background modeling on the preprocessed image to identify multiple candidate areas in the image;

[0024] Each candidate region is processed according to morphological operations, and the morphological features of each candidate region are evaluated using edge detection processing to determine whether it is the user's palm area.

[0025] Furthermore, the step of performing dynamic feature analysis on the video sequence to determine whether the user's palm is alive includes:

[0026] Performing time series analysis on the video sequence to extract dynamic change features of each frame of the video, wherein the dynamic change features include depth change and texture change of the image;

[0027] Modeling the dynamic change characteristics of each frame of the image to generate a time series feature vector, wherein the time series feature vector describes the continuous dynamic changes of the palm in the video sequence;

[0028] The preset liveness judgment model is used to analyze the generated time series feature vector to determine whether the palm is alive.

[0029] Furthermore, the step of applying a Gabor filter to the palmprint texture image of the current frame to extract palmprint texture feature vectors in multiple scales and directions includes:

[0030] Selecting Gabor filters of multiple scales and directions, wherein each Gabor filter corresponds to different scale and direction parameters, so as to extract palmprint texture features of different scales and directions in the palmprint texture image;

[0031] Apply each selected Gabor filter to the palmprint texture image of the current frame, and calculate filter response images at different scales and directions, respectively, wherein each filter response image reflects the palmprint texture feature at the corresponding scale and direction;

[0032] Perform feature extraction on the filter response image at each scale and direction, extract the palmprint texture features in each filter response image, and convert the extracted palmprint texture features into feature vector representation;

[0033] The palmprint texture feature vectors at all scales and directions are merged to generate a comprehensive palmprint texture feature vector.

[0034] Furthermore, the step of performing principal component analysis dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector includes:

[0035] Calculating the covariance matrix of the extracted palmprint texture feature vector;

[0036] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors of the covariance matrix;

[0037] Sort the eigenvectors in descending order according to the size of the eigenvalues, and select the first k eigenvectors with the largest eigenvalues ​​as the principal component eigenvectors after dimensionality reduction, where k is the preset dimensionality reduction dimension;

[0038] The extracted palmprint texture feature vector is projected onto the selected first k principal component feature vectors to generate a low-dimensional palmprint feature vector.

[0039] Furthermore, the step of extracting features from the depth image of the current frame to generate a depth feature vector includes:

[0040] Divide the depth of field image of the current frame into a plurality of non-overlapping regions;

[0041] Calculate the statistical characteristics of each divided area, wherein the statistical characteristics include depth average value, standard deviation, maximum value and minimum value;

[0042] Extracting change features and morphological features in each region, wherein the change features include depth gradients, and the morphological features include connectivity, shape features, and edge features in the region;

[0043] The statistical features, variation features, and morphological features of each region are spliced ​​together to form the feature vector of each region;

[0044] The feature vectors of all regions are combined to generate a depth feature vector of the depth image of the current frame.

[0045] Furthermore, before the step of applying the Gabor filter to the palmprint texture image of the current frame, the method further includes:

[0046] Performing Laplace filtering on the palmprint texture image of the current frame to obtain an edge-enhanced palmprint texture image;

[0047] Perform Gaussian filtering and denoising on the edge-enhanced palmprint texture image to obtain a smooth palmprint texture image;

[0048] Calculate the gradient amplitude and direction of the palmprint texture image after Gaussian filtering, and quantize the gradient direction;

[0049] Scan each pixel along its gradient direction to determine whether the gradient amplitude of each pixel is greater than the gradient amplitudes of its two adjacent pixel points along the gradient direction;

[0050] If not, suppress pixels whose gradient magnitude is no greater than that of adjacent pixels;

[0051] Compare the gradient magnitude of each pixel with a first preset threshold and a second preset threshold;

[0052] If the gradient amplitude of the target pixel point is less than the first preset threshold, the target pixel point is suppressed;

[0053] If the gradient amplitude of the target pixel point is greater than the second preset threshold, the target pixel point is retained;

[0054] If the gradient amplitude of the target pixel is between the first preset threshold and the second preset threshold, whether to retain the target pixel is determined according to whether the gradient amplitude of the adjacent pixel is greater than the second preset threshold.

[0055] Another embodiment of the present invention is also intended to provide a door lock control system, the system comprising:

[0056] An image acquisition module is used to synchronously acquire a depth image and a palm print texture image of a user's palm using a binocular infrared camera;

[0057] A feature analysis module is used to construct a video sequence from the depth of field image and palm print texture image acquired within a preset time period, and perform dynamic feature analysis on the video sequence to determine whether the user's palm is alive;

[0058] A feature extraction module is used to apply a Gabor filter to the palm print texture image of the current frame to extract palm print texture feature vectors in multiple scales and directions if the feature analysis module determines that the user's palm is alive;

[0059] The first processing and calculation module is used to perform principal component analysis and dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector, and calculate the cosine similarity between the generated palmprint feature vector and a preset palmprint feature vector;

[0060] A second processing and calculation module is used to extract features from the depth of field image of the current frame, generate a depth of field feature vector, and calculate the Euclidean distance between the generated depth of field feature vector and a preset depth of field feature vector;

[0061] The recognition evaluation module is used to perform weighted calculation based on the calculated cosine similarity and Euclidean distance to obtain a comprehensive evaluation result of palmprint recognition;

[0062] The door lock control module is used to determine whether to perform the door lock unlocking operation based on the comprehensive evaluation results of palm print recognition.

[0063] Another embodiment of the present invention aims to provide a smart door lock, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the door lock control method as described above when executing the program.

[0064] The door lock control method provided by the embodiment of the present invention can effectively prevent attacks by fake images, photos or models by using liveness detection technology, and ensure that only real user palms can be identified, thereby enhancing the security of the door lock system; by using a binocular infrared camera to obtain depth of field images and palm print texture images, and combining with dynamic feature analysis of video sequences, it can determine the liveness of the palm, thereby effectively preventing being deceived by non-live palms (such as photos or models); by applying a Gabor filter to extract palm print texture features in multiple scales and directions, and performing principal component analysis (PCA) dimensionality reduction on the extracted features, the dimension of the features is effectively reduced, and the most recognizable key information is retained, while While reducing the computational complexity, making the recognition speed faster and the performance higher; by calculating the cosine similarity and Euclidean distance, the accurate comparison and evaluation of the palm print features and the depth of field features are realized, and the identity of the current user can be judged more accurately. The weighted calculation combines the advantages of both cosine similarity and Euclidean distance, reduces the possibility of misidentification and missed identification, and improves the robustness of the recognition result; by combining binocular infrared technology with image analysis methods, it can adapt to different light, environment and angle conditions, thereby ensuring that the user identity can be accurately and stably identified in various practical application environments, avoiding the problem of excessive reliance on environmental factors in traditional technologies, and solving the problem that existing door locks cannot meet the requirements of security, user experience and adaptability to complex environments at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flow chart of a door lock control method in a first embodiment of the present invention;

[0066] Figure 2 is a structural diagram of a door lock control system in a second embodiment of the present invention;

[0067] Figure 3 is a structural diagram of a smart door lock in a third embodiment of the present invention;

[0068] The following specific implementations will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0069] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0070] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0072] Embodiment 1

[0073] See also Figure 1 , which shows a door lock control method in a first embodiment of the present invention. For the sake of convenience, only the part related to the embodiment of the present invention is shown. The door lock control method provided by the embodiment of the present invention includes:

[0074] Step S10, using a binocular infrared camera to synchronously acquire a depth of field image and a palm print texture image of the user's palm;

[0075] In one embodiment of the present invention, the method is applied to a smart door lock, wherein the smart door lock is equipped with a binocular infrared camera, which includes two infrared cameras (left camera and right camera) at a certain distance from each other. By simulating the visual principle of human eyes, it captures two perspective images of the same scene, so that the pixel offset of the same object (palm) captured by the two cameras at different positions can be measured, and then according to the difference between the two perspectives, combined with the known distance between the cameras and the geometric model, the distance from each point on the palm surface to the camera can be estimated, thereby generating a depth of field image. The depth of field image is used to provide three-dimensional contour information of the palm, including the shape and position of the palm in space. The infrared camera emits infrared light and receives the reflected infrared light information. Since the infrared light can penetrate the surface of the skin, the interference under visible light is reduced, so that it does not rely on ambient light, and can capture the details of the surface of the object under low light conditions or even in a completely dark environment, so that it is suitable for capturing the palm texture image of the palm under any light conditions. The palm texture image is used to provide detailed information on the palm surface, such as the shape and texture features of the palm print.

[0076] More specifically, the palm print texture image is the core data for identity recognition, in which the texture, lines and details of the palm print are unique. Through the detailed information of the palm print texture image, the unique features of each user can be effectively extracted to achieve identity authentication. The depth image records the three-dimensional morphological information of the palm, especially the undulations and depth distribution of the palm surface. At this time, the depth image is used to detect whether the palm is a real three-dimensional object, so as to distinguish the real palm from the flat 2D image or forged object, and prevent the use of palm print photos or models for deception, thereby enhancing the security of the system. In addition, the depth image can also further verify whether the palm is alive by identifying the shape change, contour curvature and other features of the palm, prevent the use of simulation materials for deception, and thus improve the accuracy of recognition. Therefore, the depth image and palm print texture image of the user's palm collected at this time can be used for palm print recognition and liveness inspection. It should also be pointed out that the distance between the binocular infrared cameras and the installation angle need to be accurately calibrated to ensure that the depth image and palm print texture image of the user's palm can be effectively captured.

[0077] In one embodiment of the present invention, the step of synchronously acquiring the depth of field image and the palm texture image of the user's palm using a binocular infrared camera includes:

[0078] Use a binocular infrared camera to capture the current image in real time and detect whether the user's palm area is captured;

[0079] If the user's palm area is detected, two cameras in the binocular infrared camera are used to collect two sets of images from different perspectives, and a depth-of-field image of the user's palm is generated based on the parallax calculation of the two sets of images;

[0080] Use one of the binocular infrared cameras to collect a palm print texture image of the user's palm.

[0081] Specifically, the binocular infrared camera continuously collects image data in front of the door lock in real time. During this process, the door lock uses a variety of recognition methods to identify whether there is a user's palm area in the image. Furthermore, once the user's palm area is detected, the binocular infrared camera activates the binocular vision function, and uses two left and right cameras to collect two sets of images of the user's palm from different angles. Specifically, the principle of the binocular infrared camera is that the collected images will have a certain parallax through the physical distance (baseline) between the two cameras and the difference in viewing angle. The parallax information reflects the depth information of the object (the palm in the embodiment of the present invention). At this time, by stereo matching the two sets of images, that is, finding the corresponding pixel points in the two images, according to the parallax difference between the corresponding points, the depth information of the palm in the three-dimensional space is calculated to generate a depth of field image, which usually uses a stereo vision algorithm (such as SGBM, BM, etc.) to complete the parallax calculation.

[0082] Specifically, the generation process of the depth of field image includes image preprocessing, disparity calculation and depth of field image generation. The image preprocessing is to align and denoise the images collected by the left and right cameras. Due to the slight deviation when the camera is installed, the images captured by the two cameras may have geometric distortion and alignment problems. Image alignment is to transform the two images to the same plane so that their scan lines are parallel. It is usually geometrically transformed through known camera parameters. At this time, the corrected image will eliminate the angle difference between the cameras. The disparity calculation is to calculate the depth difference of each corresponding pixel of the left and right images using the disparity algorithm. Specifically, the disparity refers to the difference in the position of the pixel points of the same object in the images captured by the two cameras at different viewing angles. The size of the disparity value is inversely proportional to the distance from the object to the camera, that is, the larger the disparity, the closer the object is; the smaller the disparity, the farther the object is. At this time, the disparity value of each pixel in the image is found through stereo matching, thereby generating a disparity map, where the disparity map is a grayscale image, in which the grayscale value of each pixel represents the disparity size of the corresponding pixel. Stereo matching is to extract corresponding feature points or feature areas from two images. Common features include corner points, edges, texture blocks, etc. For palm images, feature extraction can focus on details such as palm prints and finger contours. Depth of field image generation is to convert disparity into depth information and generate a depth of field image, in which the grayscale value of each pixel represents the distance of the palm in front of the camera. Specifically, according to the disparity value, the disparity can be converted into depth information by the following formula:

[0083]

[0084] Among them, Z is the distance from the object to the camera (that is, the depth information), f is the focal length of the camera, B is the baseline distance between the two cameras (that is, the physical distance between the cameras), and d is the parallax value. Therefore, the depth information (that is, the depth value) of each pixel can be obtained through the above calculation formula, and then the depth of field image can be generated.

[0085] While generating the depth of field image, one of the binocular infrared cameras is used to collect the palm texture image of the user's palm, that is, to obtain the fine texture structure of the user's palm through a high-resolution infrared image. The infrared camera has good resistance to ambient light interference and can clearly collect palm texture details under various lighting conditions (such as weak light and strong light), which is crucial to the accuracy of palm recognition.

[0086] Furthermore, in order to ensure the temporal and spatial consistency of the palm texture image and the depth image, the binocular infrared camera performs synchronous processing during the data acquisition process to ensure that the two sets of images are acquired at the same time point, and performs spatial correction on the acquired palm texture image and depth image to ensure that the image coordinates of the two can correspond one to one. The acquired depth image and palm texture image can be matched by timestamp or frame tag. Therefore, the binocular infrared camera can simultaneously obtain the depth image and palm texture image of the user's palm, wherein the depth image is used for liveness detection to ensure that the palm is real and live, rather than a photo or a fake hand model; and the palm texture image is used for subsequent palm feature extraction and recognition.

[0087] Furthermore, the following two recognition methods can be used to determine whether there is a user's palm area in the above-mentioned recognition image. In the first recognition method, the door lock is also equipped with a short-range infrared sensor or other type of proximity sensor in the area where the binocular infrared camera is located. The sensor monitors the area in front in real time to detect whether there is an object entering the sensing range of the sensor. When the user's palm approaches the binocular infrared camera, the sensor can detect the approach of the palm and trigger subsequent image acquisition.

[0088] In the second identification method, the step of detecting whether the user's palm area is captured specifically includes:

[0089] Preprocess the acquired current image, convert it into a grayscale image and perform denoising;

[0090] Perform skin color detection and background modeling on the preprocessed image to identify multiple candidate areas in the image;

[0091] Each candidate region is processed according to morphological operations, and the morphological features of each candidate region are evaluated using edge detection processing to determine whether it is the user's palm area.

[0092] Specifically, first, the current image collected in real time by the binocular infrared camera needs to be preprocessed to improve the accuracy and efficiency of subsequent processing. The preprocessing steps include grayscale conversion and denoising, where grayscale conversion is to convert the collected infrared image into a grayscale image and remove color information to simplify subsequent image processing operations. Denoising is to denoise the grayscale image by applying a filter (such as Gaussian filtering, mean filtering, etc.) to reduce the noise in the image, thereby improving the accuracy of candidate area recognition. Furthermore, the contrast of the image can be enhanced by using methods such as histogram equalization to make the palm area more obvious from the background.

[0093] After image preprocessing is completed, it is necessary to further identify the candidate palm area in the image. This step is processed in combination with skin color detection and background modeling. In the infrared image, the color and texture of the palm skin are quite different from the background, so the palm area can be judged by skin color detection. Its skin color detection is to identify the area that meets the color characteristics of human skin by analyzing the pixel values ​​in the image. Generally, color space conversion (such as conversion from RGB to YCbCr or HSV color space) is usually used, and then the area of ​​non-skin color is filtered out by setting a preset skin color threshold range, and the area that may be the palm is screened out. In the infrared image, similar area screening is required based on the infrared light reflection intensity characteristics rather than the traditional skin color threshold. Background modeling is to perform background modeling on the non-skin color area in the image, exclude background interference, reduce false detection, and make it used to distinguish the foreground from the background in a dynamic environment. Specifically, the background modeling method can use a foreground extraction algorithm (such as the MOG2 method) or a mixed Gaussian model (GMM) to distinguish the foreground (i.e., the palm) from the background, so that the foreground object can be effectively separated from the background. Therefore, through skin color detection and background modeling, multiple candidate areas in the image can be preliminarily identified, which may contain palms.

[0094] After identifying the candidate regions, morphological operations are performed on these candidate regions to further optimize the region boundaries and extract the morphological features of the candidate regions. Specifically, the dilation and erosion operations in the morphological operations are used to remove noise points and small areas, smooth the edges of the candidate regions, fill small holes in the candidate regions, make the region contours more coherent, eliminate noise and irregular areas, and strengthen the boundaries of the candidate regions. Dilation is to expand the edges of the region to connect broken edges or small noise points. Erosion is to shrink the edges of the region, eliminate noise points, and keep the main candidate regions.

[0095] Furthermore, in order to confirm whether each candidate area is the user's palm, the morphologically processed area is subjected to edge detection to evaluate its morphological features. The specific operation is as follows: the edge features of each candidate area are extracted using Canny edge detection or Sobel operator, wherein the palm area usually has a continuous external contour and obvious palm print edges, and these edge information can help confirm whether the shape of the candidate area conforms to the geometric features of the palm. Then, by analyzing the edge shape, size, proportion and other features of the candidate area, it is determined whether the candidate area conforms to the typical geometric features of the palm. For example, the ratio of the width and height of the palm, the smoothness of the contour, and the contour features of the palm and fingers can all be used as judgment criteria. When it is confirmed that there is a candidate area that is the user's palm area, the above-mentioned steps of collecting the depth of field image and the palm print texture image of the palm are taken.

[0096] Step S20, constructing the depth of field image and the palm print texture image acquired within a preset time period into a video sequence, and performing dynamic feature analysis on the video sequence to determine whether the user's palm is alive;

[0097] Among them, in one embodiment of the present invention, when it is determined that the user's palm is alive, step S30 is executed; otherwise, step S10 is repeated, and a warning prompt may be issued to remind the user that the palm verification has failed, and it is recommended to try again, or the possible reasons (such as incorrect palm position, etc.) are explained to the user; wherein, if it is still impossible to confirm that the user's palm is alive within a preset time or in multiple judgments, the current unlocking attempt is terminated, and the door lock operation is locked to prevent unauthorized access. At the same time, the event is recorded as an abnormal access attempt and saved in the log. This log can be used for subsequent review or to trigger further security measures (such as alarm, notification to administrator, etc.). Or after multiple liveness detection failures, the door lock can automatically switch to an alternative identity authentication method, such as password verification, RFID card verification, or mobile phone APP verification, to ensure safety and convenience.

[0098] Further, in this embodiment, image acquisition and synchronization are first performed, and a binocular infrared camera is used to synchronously acquire palm print texture images and depth of field images within a period of time. Each frame of image should contain a depth of field image and a palm print texture image, and the two images should be acquired synchronously to ensure temporal consistency. Then, according to application requirements, a suitable time period (for example, 1 second, 2 seconds, 5 seconds, etc.) is set, and the image data synchronously acquired within this time period will be used to construct a video sequence. The timestamp of each frame of image is the key to ensure that each image is arranged in the correct time sequence. Then, the depth of field image and palm print texture image acquired within the preset time period are arranged in order to generate a video sequence. The video sequence is a temporal continuity data composed of multiple image frames (each frame includes a depth of field image and a palm print texture image), which is usually organized at a specific frame rate (such as 30fps, i.e., 30 frames of image per second). It is necessary to ensure that the depth of field image and the palm print texture image are strictly synchronized in each frame, which can be ensured by marking the timestamp or frame sequence number to ensure that each image matches its corresponding time point. Then, the depth of field image and palm print texture image acquired in the same time period are arranged in chronological order and respectively used as two channels of the video sequence or superimposed into a composite image. The specific method can be parallel video channels or superimposed composite images, wherein the parallel video channel is the depth of field image and the palm print texture image as two channels of the video, forming a multi-channel video sequence, and each frame of the video is composed of two images at the same time. The superimposed composite image is to synthesize the depth of field image and the palm print texture image into a composite image through a certain fusion algorithm (such as superposition based on transparency, color mapping, etc.), and then use this as a single channel of the video output. Then, according to the needs of the application scenario, you can choose a common video file format (such as MP4, AVI, MKV, etc.), and use a suitable video encoder to save the video sequence as a video file. To ensure smooth playback of the video, you need to set a suitable frame rate (such as 30fps or 60fps) to ensure the smoothness and clarity of the video.

[0099] Furthermore, the above step of performing dynamic feature analysis on the video sequence to determine whether the user's palm is alive includes:

[0100] Performing time series analysis on the video sequence to extract dynamic change features of each frame of the video, wherein the dynamic change features include depth change and texture change of the image;

[0101] Modeling the dynamic change characteristics of each frame of the image to generate a time series feature vector, wherein the time series feature vector describes the continuous dynamic changes of the palm in the video sequence;

[0102] The preset liveness judgment model is used to analyze the generated time series feature vector to determine whether the palm is alive.

[0103] Specifically, the acquired depth of field image and palm print texture image are organized into a video sequence, and each frame image contains depth information and texture information. In order to perform dynamic feature analysis, it is necessary to extract the dynamic change features of each frame image from the video sequence, wherein the dynamic change features include the depth change and texture change of the image. Specifically, the depth image extracts the depth change of the image by calculating the depth difference between adjacent frames. The specific method is to obtain the depth difference of each pixel by differentiating the pixel values ​​of adjacent frame images, and then calculate the depth change of the overall image. The texture change can be extracted by calculating the difference of the palm print texture image between adjacent frames. The image feature extraction algorithm, such as SIFT (Scale Invariant Feature Transform) or ORB (Oriented FAST and RotatedBRIEF), is usually used to capture the change of texture features. Therefore, the depth change features and texture change features of each frame image can be extracted from the video sequence in the above manner. Then, the extracted depth change and texture change features of each frame image are modeled to form a time series feature vector. These feature vectors should contain the pattern of changes in each frame of the image in the time dimension. For example, the depth features and texture features of each frame can be represented by vectors to ensure that the order of time information is preserved. The feature vector of each frame can be multi-dimensional, representing the various features extracted in the current frame (such as depth features, texture features), and then these features are arranged in time order to form a feature matrix or vector of a long time series. At this time, each dimension in the time series vector represents a different dynamic change feature, so as to describe the continuous dynamic changes of the palm in the video sequence.

[0104] Further, a liveness judgment model is preset, which can be a method based on machine learning or deep learning, such as support vector machine (SVM), random forest, convolutional neural network (CNN), etc., or an analysis model based on biometrics (such as pulse changes, blood flow characteristics, etc.). The model is trained using a data set labeled as live and non-live. The time series feature vector in the data set should include video sequences in different palm states, labeled as "live" or "non-live". At this time, the generated time series feature vector is input into the liveness judgment model for analysis. The model analyzes the dynamic change pattern in the feature vector, such as pulse fluctuations, blood flow changes, etc., according to the pre-trained weights and structures, to determine whether it is live. If the model recognizes liveness features (for example, tiny pulse beats or blood flow changes in the palm), the user's palm is judged to be live. If no features consistent with liveness are identified (such as static or changes that do not conform to physiological laws), the user's palm is judged to be non-live. At this time, based on the results of the model analysis, it is determined whether the palm is live.

[0105] Step S30, if it is determined that the user's palm is alive, a Gabor filter is applied to the palm print texture image of the current frame to extract palm print texture feature vectors in multiple scales and directions;

[0106] In one embodiment of the present invention, when it is determined that the user's palm is alive, a Gabor filter is applied to the palm print texture image of the current frame to extract palm print texture feature vectors in multiple scales and directions. Specifically, the specific implementation of this step includes:

[0107] Selecting Gabor filters of multiple scales and directions, wherein each Gabor filter corresponds to different scale and direction parameters, so as to extract palmprint texture features of different scales and directions in the palmprint texture image;

[0108] Apply each selected Gabor filter to the palmprint texture image of the current frame, and calculate filter response images at different scales and directions, respectively, wherein each filter response image reflects the palmprint texture feature at the corresponding scale and direction;

[0109] Perform feature extraction on the filter response image at each scale and direction, extract the palmprint texture features in each filter response image, and convert the extracted palmprint texture features into feature vector representation;

[0110] The palmprint texture feature vectors at all scales and directions are merged to generate a comprehensive palmprint texture feature vector.

[0111] Specifically, the Gabor filter is a two-dimensional linear filter that can well capture the changes in local spatial frequency (texture) and respond to images at different scales and directions. In order to accurately extract the features of the palmprint texture image, it is necessary to select multiple Gabor filters of different scales and directions, each filter corresponding to a specific scale and direction. The specific parameter settings of each filter include scale parameters and direction parameters, wherein the scale parameter represents the size of the filter or the range of the frequency response, and multiple different scales are usually selected to capture the details and general structure in the palmprint texture. The direction parameter represents the directional response of the filter. In order to cover the textures of different directions that may exist in the palmprint, it is necessary to set multiple direction parameters (for example, 0 degrees, 45 degrees, 90 degrees, 135 degrees, etc.). Among them, the Gabor filter defines the filter kernel by combining the Gaussian function and the plane wave. The Gabor filter is the product of the Gaussian function and the sine wave (cosine wave). The Gaussian envelope part is used to limit the expansion of the filter in space, and the cosine part is used to capture the local frequency characteristics. Since the Gabor filter combines the characteristics of the Gaussian function and the sine wave, it can be analyzed simultaneously in the spatial domain and the frequency domain. The mathematical form of its Gabor filter is:

[0112]

[0113] Where λ is the wavelength of the filter, θ is the azimuth of the filter, ψ is the phase shift, σ is the standard deviation of the Gaussian function, γ is the spatial aspect ratio, and x' and y' are the x and y coordinates rotated in the direction θ, respectively.

[0114] Specifically, λ (Wavelength) controls the scale of the filter, that is, the period of the plane wave. It defines the frequency of the Gabor filter and determines the texture frequency characteristics that the filter can capture. A larger λ value can capture low-frequency textures, while a smaller λ value captures high-frequency textures. θ (Orientation) represents the direction of the Gabor filter, which determines the filter's response to different directions in the image. Common direction angles are 0°, 45°, 90°, 135°, etc. Filters in different directions help to extract texture features in various directions in the image. ψ (Phase offset) controls the phase of the cosine function in the filter kernel. The phase offset can determine the filter's response to changes in the image brightness gradient. It usually takes a value of 0 or π / 2, corresponding to symmetric and antisymmetric Gabor filters, respectively. σ (Standard deviation) controls the extent of the Gaussian function's expansion and affects the width of the Gabor filter in the spatial domain. A larger σ value enables the filter to capture textures in larger areas, while a smaller σ value makes the filter sensitive to texture changes in smaller areas. γ (Aspect ratio) controls the shape of the elliptical envelope of the Gaussian function, i.e., the aspect ratio. Usually, γ=1 means that the Gaussian function is circular, while γ<1 means that the envelope is elliptical and narrower in the horizontal direction than in the vertical direction. x' and y' are the coordinate values ​​of the x and y coordinates after rotating by an angle of θ, which rotates the coordinate system of the input image (x and y) to align with the filter direction. x and y are the horizontal and vertical coordinates of the pixel points in the palmprint texture image, respectively. The specific calculation formulas for x' and y' are:

[0115] x'=xcosθ+ysinθ

[0116] y'=-xsinθ+ycosθ

[0117] Therefore, when selecting the scale, by setting several scale values ​​λ, each scale corresponds to a different frequency component, usually increasing from low frequency to high frequency; and when selecting the direction, by setting several direction angles θ, such as 0°, 45°, 90°, 135°, to capture the texture features of the palm print in different directions. At this time, by adjusting and combining different λ and θ parameters, filters of different scales and directions can be generated to detect different texture features in the image.

[0118] Further, once the Gabor filter of appropriate scale and direction is selected, the Gabor filter is applied to the palmprint texture image of the current frame in sequence. The specific operation is as follows: use each Gabor filter to perform a two-dimensional convolution operation with the palmprint texture image of the current frame to obtain filter response images of different scales and directions to extract texture features of corresponding scales and directions in the image. The role of convolution is to capture the local texture features of the palmprint texture image at the corresponding scale and direction through the filter. The mathematical formula of the convolution operation is:

[0119] I filtered (x,y)=I(x,y)*G(x,y;λ,θ,ψ,σ,γ)

[0120] Among them, I filtered (x, y) represents the filter response image, I(x, y) is the pixel value (i.e., grayscale value) of the palmprint texture image at position (x, y), G(x, y; λ, θ, ψ, σ, γ) represents the Gabor filter, and * represents the convolution operation. At this time, the output result after convolution is the filter response image, which represents the texture response intensity of the palmprint texture image at this scale and direction. In this way, the Gabor filter of each scale and direction can extract the palmprint texture features of a specific frequency and direction in the palmprint texture image.

[0121] Furthermore, feature extraction is performed on the filter response image generated at each scale and direction to extract representative palmprint texture features from the filter response image and convert it into a palmprint texture feature vector, that is, the feature is represented in vector form. The feature extraction is usually performed by calculating the statistical features of the filter response image, where the statistical features include mean, variance, skewness and kurtosis, etc. These statistical features can reflect the local contrast and texture information in the filter response image.

[0122] Furthermore, after extracting the palmprint texture feature vectors at different scales and directions, it is necessary to merge these palmprint texture feature vectors to generate a comprehensive palmprint texture feature vector. The specific operation is to splice the palmprint texture feature vectors of different scales and directions in a certain order to form a high-dimensional feature vector, which contains the complete texture information of the palmprint texture image of the current frame at multiple scales and directions. For example, if the system uses Gabor filters of 5 scales and 8 directions, and the length of the feature vector extracted in each direction is n, the length of the final palmprint texture feature vector is 5×8×n.

[0123] At this time, through the above steps, the multi-scale and multi-directional features in the palmprint texture image can be efficiently and accurately extracted. Since its Gabor filter is composed of a sine wave and a Gaussian envelope function, it can generate a set of filters in different directions according to the set parameters, and can respond to the features in different directions in the image. This directional selectivity allows it to sensitively capture the texture or edge arranged along a specific direction in the image, which is suitable for detecting complex line structures in images such as palmprints. At the same time, another important feature of the Gabor filter is that the scale of the filter can be adjusted by changing its parameters, and filters of different scales can capture features of different sizes in the image. For palmprint texture images, texture features of different scales, such as thick lines and fine textures, will be captured by Gabor filters of different scales, thereby comprehensively acquiring the multi-scale features of the palmprint. In addition, the Gabor filter has good spatial and frequency localization characteristics, where spatial localization means that the filter can focus on the local area in the image to ensure accurate detection of the features of the local area; and frequency localization refers to the response of the filter to different frequencies (textures of different scales) in the image. This spatial-frequency localization characteristic makes the Gabor filter very efficient in texture detection and edge extraction, and can accurately identify the changes in local features and edge information. At the same time, because the Gabor filter is selective in both space and frequency, it is more robust to noise when extracting local features of the image. Palmprint texture images may contain some subtle noise or interference, but the Gabor filter can ignore these noises and focus on capturing the key texture structure of the palmprint. Therefore, the Gabor filter can very effectively capture local features of different scales and directions in the image through its directionality, scale, spatial-frequency localization and robustness to noise, so that it can capture the detailed texture and macro structure of the palmprint, and enhance the robustness and accuracy of palmprint recognition. At the same time, it has a strong ability to resist environmental interference, especially for palmprint texture images under different lighting conditions, it can still maintain a high recognition accuracy. At the same time, because the Gabor filter can extract multi-directional and multi-scale features, it can capture rich detail information at different levels of palmprint features, thereby greatly improving the accuracy of palmprint recognition.

[0124] Furthermore, when the user performs palm print unlocking, the user may have wet hands, and the effect of palm print recognition may be affected to a certain extent, because factors such as moisture or light may cause the palm texture to become blurred or distorted or the image noise to increase. In order to solve the above problems and improve the detection effect in the wet hand state, in other embodiments of the present invention, it also enhances the edge information in the palm print texture image through an image enhancement method, so as to effectively improve the recognizability of the palm print texture, highlight the outline of the palm print, and help improve the recognition effect in the wet hand state. Specifically, in other embodiments of the present invention, before the step of applying the Gabor filter to the palm print texture image of the current frame, it also includes:

[0125] Performing Laplace filtering on the palmprint texture image of the current frame to obtain an edge-enhanced palmprint texture image;

[0126] Perform Gaussian filtering and denoising on the edge-enhanced palmprint texture image to obtain a smooth palmprint texture image;

[0127] Calculate the gradient amplitude and direction of the palmprint texture image after Gaussian filtering, and quantize the gradient direction;

[0128] Scan each pixel along its gradient direction to determine whether the gradient amplitude of each pixel is greater than the gradient amplitudes of its two adjacent pixel points along the gradient direction;

[0129] If not, suppress pixels whose gradient magnitude is no greater than that of adjacent pixels;

[0130] Compare the gradient magnitude of each pixel with a first preset threshold and a second preset threshold;

[0131] If the gradient amplitude of the target pixel point is less than the first preset threshold, the target pixel point is suppressed;

[0132] If the gradient amplitude of the target pixel point is greater than the second preset threshold, the target pixel point is retained;

[0133] If the gradient amplitude of the target pixel is between the first preset threshold and the second preset threshold, whether to retain the target pixel is determined according to whether the gradient amplitude of the adjacent pixel is greater than the second preset threshold.

[0134] Specifically, the palm print texture image of the current frame is first processed by Laplace filtering to enhance the edge features of the image. Laplace filtering is an edge detection method based on second-order derivatives, which can highlight the edge area in the image and enhance the details of the palm print texture. By convolving the Laplace filter with the palm print texture image, an edge-enhanced palm print texture image can be obtained, which helps to capture the texture structure of the palm print more clearly. The mathematical expression of Laplace filtering is:

[0135]

[0136] in, The Laplacian operator represents the palmprint texture image, I(x,y) is the pixel value (ie, grayscale value) of the palmprint texture image at the position (x,y), and x and y are the horizontal and vertical coordinates of the pixel point in the palmprint texture image, respectively. Represents the second-order derivative of the image in the x direction, which measures the rate of change of the image pixel value in the x direction. It represents the second-order derivative of the image in the y direction, and measures the rate of change of the image pixel value in the y direction. Specifically, the Laplace filter is used for edge detection, and the edge features in the image are highlighted by calculating the second-order derivative of the image. If the grayscale value in a certain area of ​​the image changes dramatically (for example, at the edge), the Laplace operator will give a larger value, thereby effectively detecting the edge.

[0137] Furthermore, Gaussian filtering is applied to the edge-enhanced palmprint texture image for denoising to reduce the noise in the image and smooth the image. Gaussian filtering is a linear smoothing filter that can effectively remove high-frequency noise in the image and reduce the impact of moisture or light reflection. At this time, the Gaussian filter is convolved with the palmprint texture image after Laplace filtering to obtain a denoised and smoothed palmprint texture image, so that the edge details of the palmprint can be retained while reducing random noise in the image. The mathematical formula of Gaussian filtering is:

[0138]

[0139] Where G(x,y) is the weight value of the Gaussian filter, which is used to control the weight of the filter at each position of the image; σ is the standard deviation of the Gaussian function, which is used to control the smoothness of the filter, where a smaller σ enhances local details and a larger σ produces a smoother effect; exp represents the natural exponential function.

[0140] Furthermore, after the denoising process is completed, the gradient amplitude and gradient direction of the palmprint texture image of the current frame are calculated. The gradient is the direction and intensity of the brightness change in the image, which can reflect the edge information in the image. Specifically, the Sobel operator or other commonly used gradient operators are used to calculate the gradient of the palmprint texture image in the x-axis and y-axis directions, where the gradient, gradient amplitude and gradient direction calculation formulas are:

[0141]

[0142] Among them G x and G yare the gradients of the palmprint texture image in the x-axis and y-axis directions, that is, the rate of change of the pixel value of the palmprint texture image in the horizontal and vertical directions; |G| is the gradient amplitude, which indicates the intensity of the change of the pixel value of the palmprint texture image, reflecting the strength of the edge of the image. The larger the gradient amplitude, the more drastic the change of the grayscale value at the pixel point; θ1 is the direction of the gradient, indicating the direction of the change of the pixel value of the palmprint texture image. The angle θ1 is expressed in radians and is defined as the angle from the positive direction of the x-axis to the direction of the gradient in a counterclockwise direction. At this time, the gradient amplitude is used to measure the edge strength of each pixel in the image, and the gradient direction is used to indicate the direction of the edge. At this time, by calculating the gradient of the palmprint texture image in the x-axis and y-axis directions, the edge in the palmprint texture image can be detected, and the strength and position of the edge can be determined according to the gradient amplitude and direction. In order to facilitate subsequent processing, the calculated gradient direction is quantized, that is, the continuous gradient direction is discretized into a limited number of direction intervals. For example, a common quantization method is to quantize the gradient direction into four main directions (0°, 45°, 90°, and 135°), or eight directions, for simple and fast processing.

[0143] Furthermore, after the gradient amplitude and direction are calculated, a non-maximum suppression (NMS) operation is performed on the palmprint texture image to retain the significant edges in the palmprint texture image and suppress the non-edge areas. The specific steps are: for each pixel point, scan along its gradient direction to check whether the gradient amplitude of the pixel point is greater than the gradient amplitudes of its two adjacent pixel points along the gradient direction. If the gradient amplitude of the pixel point is less than the gradient amplitudes of its two adjacent pixel points, the gradient amplitude of the pixel point is set to 0, indicating that the pixel point does not belong to the edge area, thereby achieving edge refinement.

[0144] Among them, in order to further distinguish the strong edge and weak edge in the image, double threshold processing is introduced. By setting two different gradient amplitude thresholds (first preset threshold and second preset threshold) to process the gradient amplitude of the pixel point, if the gradient amplitude of the target pixel point is less than the low threshold, the pixel point is suppressed, that is, set to 0. If the gradient amplitude of the target pixel point is greater than the high threshold, the pixel point is retained as a strong edge point. If the gradient amplitude of the target pixel point is between the low threshold and the high threshold, it is determined whether to retain it according to the gradient amplitude of the adjacent pixels of the pixel point. If the gradient amplitude of the adjacent pixel point is greater than the high threshold, the pixel point is retained as a weak edge point; otherwise, the pixel point is suppressed. At this time, through double threshold processing, strong edges, weak edges and non-edge areas can be effectively distinguished, and important edge features in the image are retained, weak noise edges are removed, and the detail fidelity of the palmprint texture image is further improved.

[0145] Therefore, after the above processing, a smooth and refined palmprint texture image is finally obtained after edge enhancement, denoising, gradient calculation and edge refinement. The implementation of these steps can effectively enhance the important edge details in the palmprint texture image, while suppressing noise and unimportant details. Even under wet hand conditions, clear palmprint edges can be found in the palmprint texture image, thereby improving the texture recognition, thereby providing a more reliable image basis for subsequent Gabor filter processing and palmprint feature extraction. It helps to improve the accuracy and robustness of palmprint recognition, and maintain a high recognition rate even under complex environmental conditions.

[0146] Step S40, performing principal component analysis dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector, and calculating the cosine similarity between the generated palmprint feature vector and a preset palmprint feature vector;

[0147] In one embodiment of the present invention, the feature vector extracted by the Gabor filter may have a high dimension. For example, the palm print texture features at multiple scales and directions will cause the dimension of the feature vector to increase sharply. At this time, the high dimension will increase the complexity of data calculation, so it is necessary to reduce the dimension of the high-dimensional feature vector to a lower dimension to reduce the complexity of calculation. Specifically, the above-mentioned steps of performing principal component analysis dimensionality reduction processing on the extracted palm print texture feature vector to generate a low-dimensional palm print feature vector include:

[0148] Calculating the covariance matrix of the extracted palmprint texture feature vector;

[0149] Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors of the covariance matrix;

[0150] Sort the eigenvectors in descending order according to the size of the eigenvalues, and select the first k eigenvectors with the largest eigenvalues ​​as the principal component eigenvectors after dimensionality reduction, where k is the preset dimensionality reduction dimension;

[0151] The extracted palmprint texture feature vector is projected onto the selected first k principal component feature vectors to generate a low-dimensional palmprint feature vector.

[0152] Specifically, we first need to calculate the covariance matrix of the palmprint texture feature vector, where the covariance matrix is ​​used to measure the linear correlation of different dimensions in the feature vector. Suppose the original palmprint texture feature vector is a matrix X with a dimension of n×m, where n represents the number of samples and m represents the dimension of the palmprint texture feature. First, the feature vector is mean normalized, the mean of each feature is calculated, and the mean is subtracted from each feature vector to obtain the normalized matrix X' to eliminate the offset between the eigenvalues. Then calculate the covariance matrix C of the normalized palmprint texture feature vector matrix X. The calculation formula of the covariance matrix is:

[0153]

[0154] Among them, C is the covariance matrix, which is an m×m square matrix, representing the covariance between different feature dimensions, X' T Represents the transposed matrix of the normalized matrix X'.

[0155] Furthermore, the calculated covariance matrix C is subjected to eigenvalue decomposition to obtain its eigenvalues ​​and corresponding eigenvectors. The eigenvalue decomposition formula of the covariance matrix is:

[0156] C=PΛP T

[0157] Among them, Λ is a diagonal matrix, where the elements on the diagonal are the eigenvalues ​​of the covariance matrix, P is a matrix composed of the eigenvectors of the covariance matrix, and each column corresponds to an eigenvector, P T is the transposed matrix of the matrix P. The eigenvalue describes the variance in the direction of each eigenvector. At this time, the large eigenvalue corresponds to the main information in the data, while the smaller eigenvalue corresponds to noise or redundant information. At this time, a set of eigenvalues ​​and their corresponding eigenvectors of the covariance matrix C can be obtained through eigenvalue decomposition.

[0158] Furthermore, the eigenvalues ​​are arranged in descending order according to their size, and the eigenvectors corresponding to the first k eigenvalues ​​are selected as the principal component eigenvectors after dimensionality reduction according to the size of the sorted eigenvalues. Where k is the preset dimensionality reduction dimension, and the appropriate k value is usually selected based on the percentage of the data variance to be retained, for example, if 90% of the data variance is desired to be retained.

[0159] Finally, the original palmprint texture feature vector X is projected onto the selected k principal component feature vectors to generate a low-dimensional palmprint feature vector. The calculation formula for the projection is:

[0160] Y=XP k

[0161] Where Y is the low-dimensional palmprint feature vector after projection, with a dimension of n×k, and each row corresponds to a low-dimensional representation of a palmprint texture feature vector; P k It is a matrix consisting of the first k eigenvectors, with dimension m×k.

[0162] Therefore, through the above steps, by calculating the covariance matrix and finding the principal components, the eigenvector can be projected to the direction with the largest variance, retaining the principal component that best explains the data variance and reducing redundant features. At this time, the dimension of the data after dimensionality reduction is reduced, while retaining most of the important features, significantly improving the efficiency of subsequent calculations.

[0163] Furthermore, after generating a low-dimensional palmprint feature vector, the cosine similarity between the generated palmprint feature vector and the preset palmprint feature vector is calculated, where the preset palmprint feature vector is the palmprint texture image and its extracted feature vector collected when the user registered the palmprint information stored in the door lock, wherein the cosine similarity evaluates their similarity by calculating the cosine value of the angle between the two vectors, and the value range is between [-1,1], where 1 means that the two vectors are exactly the same, -1 means that the two vectors are completely opposite, and 0 means that the two are orthogonal, that is, there is no similarity. If the result is close to 1, it means that the two vectors are very similar, and if the result is close to 0, it means that the similarity is low; if the result is a negative number, it means that the vectors are very different, that is, the cosine similarity is used to measure the directional difference between the two feature vectors. In other words, it focuses on whether the directions of the two vectors are consistent, not their lengths (that is, the size of the eigenvalues). Even if the numerical values ​​of the two feature vectors are very different, if their directions are close, the cosine similarity will still consider them similar. Therefore, the cosine similarity ignores the length of the feature vector and focuses on the directionality of the feature vector. Therefore, in the palmprint feature, even if the overall brightness or contrast of the palmprint texture image changes (resulting in different eigenvalues), as long as its structure or shape remains consistent, cosine similarity can still identify similarities, making it effective against the influence of outliers, and it is particularly useful under some lighting conditions or noise. Therefore, cosine similarity is more commonly used when processing directional features or texture features of images, such as palmprint texture images.

[0164] Step S50, extracting features from the depth image of the current frame, generating a depth feature vector, and calculating a Euclidean distance between the generated depth feature vector and a preset depth feature vector;

[0165] In one embodiment of the present invention, the step of extracting features from the depth image of the current frame to generate a depth feature vector includes:

[0166] Divide the depth of field image of the current frame into a plurality of non-overlapping regions;

[0167] Calculate the statistical characteristics of each divided area, wherein the statistical characteristics include depth average value, standard deviation, maximum value and minimum value;

[0168] Extracting change features and morphological features in each region, wherein the change features include depth gradients, and the morphological features include connectivity, shape features, and edge features in the region;

[0169] The statistical features, variation features, and morphological features of each region are spliced ​​together to form the feature vector of each region;

[0170] The feature vectors of all regions are combined to generate a depth feature vector of the depth image of the current frame.

[0171] Specifically, the depth image of the current frame is first divided into multiple non-overlapping areas (such as uniform grids or adaptive areas) so that the depth information in each area can be analyzed and processed independently. Then, for each divided area, a series of statistical features are calculated, where the statistical features include but are not limited to the depth mean, standard deviation, maximum and minimum values, where the depth mean represents the average of the depth values ​​of all pixels in the area, reflecting the overall depth level of the area; the standard deviation represents the degree of discreteness of the depth values ​​in the area, which is used to measure the volatility of the depth values. The maximum and minimum values ​​represent the extreme values ​​of the depth values ​​in the area, describing the depth range of the area.

[0172] Then, the depth information in each region is further analyzed to extract the change features and morphological features, where the change features include the depth gradient, where the depth gradient refers to the rate of change of the depth value in the image, reflecting the change trend of the depth between pixels. The magnitude of the gradient indicates the magnitude of the depth change, and the direction indicates the direction of the change. The gradient can reveal the edges and surfaces of objects, such as significant depth changes at the edges or joints of the palm. Specifically, the depth gradient can be obtained by calculating the difference in neighborhood depth values ​​of each pixel in the depth image. The commonly used method is to apply the Sobel operator, Prewitt operator, etc. to the depth image to calculate the horizontal and vertical gradients.

[0173] Among them, morphological features are mainly used to describe the geometric structure and morphological information of objects in the image. In the depth of field image, the connectivity, shape features, edge features, etc. of the object can be described, where connectivity refers to the connection state between pixels in the image, which is usually used to identify the integrity or separation state of the object. For example, a palm can be regarded as a connected area. At this time, connectivity is used to identify the outline and overall shape of the palm to determine whether it is a complete palm. Specifically, it can detect the number of depth-similar connected components in the area, which can usually be achieved by combining a binary operation based on the depth value with a connected component analysis, where the connected component analysis (CCA) algorithm divides the connected pixel area into different connected components by analyzing the connectivity of adjacent pixels in the image. Shape features are used to describe the geometric shape of objects in the image, such as the edge, curved surface, depression, etc. of the object, where shape features are used to analyze the shape of fingers, depressions or protrusions of the palm in palm recognition. Specifically, it can use the calculated shape moments (Hu moments, Zernike moments, etc.) to describe the outline of the object and extract the area, perimeter, circularity and other features of the object. In addition, the geometric shape of the object can be described by calculating the shape contour features of the pixels in the image. The edge feature describes the characteristics at the boundary of the object, usually reflecting the change of the object shape, such as the junction between the finger and the palm. The edge feature can help identify the outline of the object, edge protrusions, etc. In the palm image, the edge of the finger is an important feature. Through algorithms such as Canny edge detection and Sobel operator, the edge information in the depth of field image can be extracted to mark the object boundary.

[0174] Furthermore, for each region, its statistical features (such as mean, standard deviation, maximum, minimum), variation features (depth gradient) and morphological features (connectivity, shape features, edge features) are spliced ​​to form a feature vector of the region, and then the feature vectors of all regions are merged to generate the depth feature vector of the entire depth image.

[0175] Further, the Euclidean distance between the generated depth feature vector and the preset depth feature vector is calculated, wherein the preset depth feature vector is the depth image and its extracted feature vector collected when the user registers the palm print information stored in the door lock, wherein the Euclidean distance is used to measure the absolute distance between the two images in the feature space, wherein the Euclidean distance focuses on the actual geometric distance between the two feature vectors in space, and measures the numerical difference of the feature vectors. The closer the two vectors are, the more similar they are. Therefore, the Euclidean distance is suitable for calculating the numerical difference between different dimensions. The depth image contains the three-dimensional contour and shape features of the palm. The depth image is essentially the depth value at different positions in the image, which is more suitable for measurement using the Euclidean distance. Further, the depth image is composed of a large number of pixels, and each pixel may contain multiple dimensions (such as depth value, color information, etc.). Therefore, the Euclidean distance calculation of the depth image may need to process very high-dimensional data, which will result in a huge amount of calculation and is not easy to process. At the same time, directly comparing the distance of each pixel is inefficient. And directly calculating the distance of each pixel may contain a lot of redundant information, which is not helpful for judging the similarity of the image. Therefore, in order to reduce the computational complexity and extract useful features, the depth of field image is usually converted into a depth of field feature vector first, so that the redundant information in the image (such as noise, repeated texture, etc.) can be removed through the feature extraction process, so that its feature vector can better express the key information of the image, which is more representative than the simple depth value. Therefore, calculating the Euclidean distance between feature vectors can more accurately measure the similarity between images, especially when processing images with complex backgrounds or noise. At the same time, the feature vector dimension is low, so the calculation of the Euclidean distance is more efficient, and it can better measure the similarity between images, avoiding the computational bottleneck of directly calculating in the original high-dimensional pixel space. Therefore, calculating the Euclidean distance between depth of field feature vectors can quantify the distance difference between each point of the palm in three-dimensional space, so that the difference in palm shape and depth information can be judged, and then identification and verification can be performed.

[0176] Step S60, performing weighted average calculation based on the calculated cosine similarity and Euclidean distance to obtain a comprehensive evaluation result of palmprint recognition;

[0177] Among them, in one embodiment of the present invention, after calculating the cosine similarity between the generated palmprint feature vector and the preset palmprint feature vector, and calculating the Euclidean distance between the generated depth of field feature vector and the preset depth of field feature vector, since the dimensions and ranges of the cosine similarity and the Euclidean distance are different (the cosine similarity is usually in the range of [-1,1], and the Euclidean distance is a non-negative number with a large numerical range), the two need to be standardized so that their ranges are relatively consistent, so as to perform weighted averaging. In order to unify the dimensions, the cosine similarity can be converted into a value between 0 and 1. The value of the Euclidean distance needs to be normalized to the range of [0,1], which can be normalized according to the maximum and minimum values ​​of the Euclidean distance. At the same time, in order to comprehensively consider the contributions of cosine similarity and Euclidean distance, their respective weights are defined to represent the importance of cosine similarity and Euclidean distance in the comprehensive evaluation. In general, the sum of the weights of cosine similarity and Euclidean distance is 1. At this time, according to the standardized cosine similarity and Euclidean distance, combined with the assigned weights, the comprehensive evaluation result is obtained by weighted averaging.

[0178] Step S70, determining whether to perform the door unlocking operation according to the comprehensive evaluation result of the palm print recognition;

[0179] Among them, in one embodiment of the present invention, according to the comprehensive evaluation result, a threshold can be set to determine whether it is a match. For example, if the comprehensive evaluation result is greater than a certain threshold (for example, 0.6), the palm print recognition result is considered to be a match. If the comprehensive evaluation result is less than the threshold, it is considered that the recognition fails or does not match. At this time, after the recognition evaluation is completed, the comprehensive evaluation result is output. If the comprehensive evaluation result is a successful recognition, an unlocking signal is sent, an unlocking instruction is sent to the control circuit of the door lock, and the motor or electromagnetic device is started to convert the door lock from a locked state to an unlocked state; if the recognition fails, an access denial signal is sent to refuse to unlock and prompt the user that the recognition fails or the identity verification fails, and the door lock remains locked. Specifically, the door lock unlocking process can communicate with the door lock control system through a UART serial interface. Among them, UART is a commonly used serial communication protocol that can realize data transmission between the door lock hardware and the palm print recognition system. During communication, the recognition result is sent to the door lock control unit through a data frame to control the single-chip microcomputer to perform the corresponding unlocking or locking operation.

[0180] In summary, the door lock control method in the above embodiment of the present invention, by combining the depth of field image and the palm print texture image and utilizing the liveness detection technology, can effectively prevent the attack of the fake image, photo or model and other prosthesis, and ensure that only the real user's palm can be identified, thereby enhancing the security of the door lock system; by utilizing the binocular infrared camera to obtain the depth of field image and the palm print texture image, and combining the dynamic feature analysis of the video sequence, it can determine the liveness of the palm, thereby effectively preventing being deceived by the non-live palm (such as a photo or a model); by applying the Gabor filter to extract the palm print texture features in multiple scales and directions, and performing the principal component analysis (PCA) dimensionality reduction on the extracted features, the dimension of the features is effectively reduced, and the most recognizable key information is retained. , while reducing the computational complexity, making the recognition faster and the performance higher; by calculating the cosine similarity and Euclidean distance, the accurate comparison and evaluation of the palm print features and the depth of field features are realized, and the identity of the current user can be judged more accurately. The weighted calculation combines the advantages of both cosine similarity and Euclidean distance, reducing the possibility of misidentification and missed identification, and improving the robustness of the recognition results; by combining binocular infrared technology with image analysis methods, it can adapt to different light, environment and angle conditions, thereby ensuring that the user identity can be accurately and stably identified in various practical application environments, avoiding the problem of excessive reliance on environmental factors in traditional technologies, and solving the problem that existing door locks cannot simultaneously meet security, user experience and adapt to complex environments.

[0181] Embodiment 2

[0182] See also Figure 2 , is a schematic diagram of the structure of a door lock control system provided by the second embodiment of the present invention. For the convenience of description, only the part related to the embodiment of the present invention is shown. The door lock control system includes:

[0183] An image acquisition module 11 is used to synchronously acquire a depth image and a palm print texture image of a user's palm using a binocular infrared camera;

[0184] The feature analysis module 12 is used to construct a video sequence from the depth of field image and the palm texture image acquired within a preset time period, and perform dynamic feature analysis on the video sequence to determine whether the user's palm is alive;

[0185] The feature extraction module 13 is used to apply a Gabor filter to the palm print texture image of the current frame to extract palm print texture feature vectors in multiple scales and directions if the feature analysis module determines that the user's palm is alive;

[0186] The first processing and calculation module 14 is used to perform principal component analysis and dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector, and calculate the cosine similarity between the generated palmprint feature vector and a preset palmprint feature vector;

[0187] The second processing and calculation module 15 is used to extract features from the depth image of the current frame, generate a depth feature vector, and calculate the Euclidean distance between the generated depth feature vector and a preset depth feature vector;

[0188] The recognition evaluation module 16 is used to perform weighted calculation based on the calculated cosine similarity and Euclidean distance to obtain a comprehensive evaluation result of palmprint recognition;

[0189] The door lock control module 17 is used to determine whether to perform the door lock unlocking operation according to the comprehensive evaluation result of the palm print recognition.

[0190] Furthermore, in one embodiment of the present invention, the image acquisition module 11 includes:

[0191] The detection and judgment unit uses a binocular infrared camera to collect the current image in real time and detect whether the user's palm area is captured;

[0192] The depth image acquisition unit is used to collect two sets of images from different viewing angles using two cameras in the binocular infrared camera when the detection and judgment unit detects the user's palm area, and generate a depth image of the user's palm according to the parallax calculation of the two sets of images;

[0193] The palm print texture image acquisition unit is used to collect the palm print texture image of the user's palm using one of the binocular infrared cameras.

[0194] Furthermore, in one embodiment of the present invention, the detection and judgment unit includes:

[0195] The image processing subunit is used to pre-process the acquired current image, convert it into a grayscale image and perform denoising;

[0196] The candidate region identification subunit is used to perform skin color detection and background modeling on the pre-processed image and identify multiple candidate regions in the image;

[0197] The palm region judgment subunit is used to process each candidate region according to morphological operations, and use edge detection processing to evaluate the morphological features of each candidate region to determine whether it is the user's palm region.

[0198] Furthermore, in one embodiment of the present invention, the feature analysis module 12 includes:

[0199] A dynamic change feature extraction unit is used to perform time series analysis on the video sequence and extract the dynamic change features of each frame image in the video, wherein the dynamic change features include depth change and texture change of the image;

[0200] A time series feature vector generating unit, used for modeling the dynamic change characteristics of each frame of the image, and generating a time series feature vector, wherein the time series feature vector describes the continuous dynamic change of the palm in the video sequence;

[0201] The feature vector analysis unit is used to analyze the generated time series feature vector using a preset liveness judgment model to judge whether the palm is alive.

[0202] Furthermore, in one embodiment of the present invention, the feature extraction module 13 includes:

[0203] A filter selection unit is used to select Gabor filters of multiple scales and directions, wherein each Gabor filter corresponds to different scale and direction parameters, so as to extract palmprint texture features of different scales and directions in the palmprint texture image;

[0204] A filter calculation unit, used for applying each selected Gabor filter to the palm print texture image of the current frame, and respectively calculating filter response images at different scales and directions, wherein each filter response image reflects the palm print texture feature at a corresponding scale and direction;

[0205] A feature extraction unit is used to extract features from the filter response image at each scale and direction, extract the palmprint texture features in each filter response image, and convert the extracted palmprint texture features into feature vector representation;

[0206] The palmprint texture feature vector generating unit is used to merge the palmprint texture feature vectors in all scales and directions to generate a comprehensive palmprint texture feature vector.

[0207] Furthermore, in one embodiment of the present invention, the first processing and computing module 14 includes:

[0208] A covariance matrix calculation unit, used for calculating the covariance matrix of the extracted palmprint texture feature vector;

[0209] An eigendecomposition unit is used to perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors of the covariance matrix;

[0210] A principal component eigenvector selection unit is used to sort the eigenvectors in descending order according to the size of the eigenvalues, and select the first k eigenvectors with the largest eigenvalues ​​as the principal component eigenvectors after dimensionality reduction, where k is a preset dimensionality reduction dimension;

[0211] The palmprint feature vector generating unit is used to project the extracted palmprint texture feature vector onto the selected first k principal component feature vectors to generate a low-dimensional palmprint feature vector.

[0212] Furthermore, in one embodiment of the present invention, the second processing and computing module 15 includes:

[0213] A region division unit, used for dividing the depth of field image of the current frame into a plurality of non-overlapping regions;

[0214] A feature calculation unit, used to calculate the statistical features of each divided area, wherein the statistical features include a depth average, a standard deviation, a maximum value, and a minimum value;

[0215] A feature extraction unit, used to extract change features and morphological features in each region, wherein the change features include depth gradients, and the morphological features include connectivity, shape features, and edge features in the region;

[0216] A feature concatenation unit is used to concatenate the statistical features, variation features, and morphological features of each region to form a feature vector of each region;

[0217] The depth feature vector generating unit is used to merge the feature vectors of all regions to generate the depth feature vector of the depth image of the current frame.

[0218] Furthermore, in one embodiment of the present invention, the system further comprises:

[0219] The edge enhancement module is used to perform Laplace filtering on the palm print texture image of the current frame to obtain an edge enhanced palm print texture image;

[0220] A filtering and denoising module is used to perform Gaussian filtering and denoising on the edge-enhanced palmprint texture image to obtain a smooth palmprint texture image;

[0221] A gradient calculation module is used to calculate the gradient amplitude and direction of the palmprint texture image after Gaussian filtering, and to quantify the gradient direction;

[0222] A gradient amplitude judgment module is used to scan each pixel point along its gradient direction to judge whether the gradient amplitude of each pixel point is greater than the gradient amplitudes of its two adjacent pixel points along the gradient direction;

[0223] A first pixel processing module is used for suppressing a pixel whose gradient amplitude is not greater than that of an adjacent pixel when the gradient amplitude judgment module judges that the gradient amplitude of a pixel is not greater than that of its two adjacent pixel points along the gradient direction;

[0224] A gradient magnitude comparison module, used to compare the gradient magnitude of each pixel with a first preset threshold and a second preset threshold;

[0225] A second pixel processing module, configured to suppress the target pixel if the gradient amplitude of the target pixel is less than a first preset threshold;

[0226] A third pixel processing module, configured to retain the target pixel if the gradient amplitude of the target pixel is greater than a second preset threshold;

[0227] The fourth pixel processing module is used to determine whether to retain the target pixel point according to whether the gradient amplitude of the adjacent pixel point is greater than the second preset threshold value if the gradient amplitude of the target pixel point is between the first preset threshold value and the second preset threshold value.

[0228] The door lock control system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0229] Embodiment 3

[0230] Another aspect of the present invention is to provide a smart door lock. Figure 3 , shown is a smart door lock in the third embodiment of the present invention, including a memory 20, a processor 10, and a program 30 stored in the memory 20 and executable on the processor. When the processor 10 executes the program 30, the door lock control method as in the above embodiment is implemented.

[0231] In some embodiments, the processor 10 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 20, such as executing access restriction programs.

[0232] Among them, the memory 20 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 20 can be an internal storage unit of the smart door lock, such as the hard disk of the smart door lock. In other embodiments, the memory 20 can also be an external storage device of the smart door lock, such as a smart memory card (SmartMedia Card, SMC) equipped on the smart door lock, a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 20 can also include both an internal storage unit of the smart door lock and an external storage device. The memory 20 can be used not only to store application software and various types of data installed in the smart door lock, but also to temporarily store data that has been output or is to be output.

[0233] It should be pointed out that Figure 3The structure shown does not constitute a limitation of the smart door lock. In other embodiments, the smart door lock may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0234] In summary, the smart door lock in the above-mentioned embodiment of the present invention can effectively prevent attacks by fake images, photos or models by combining depth of field images and palm print texture images and using liveness detection technology, ensuring that only real user palms can be identified, thereby enhancing the security of the door lock system; by using a binocular infrared camera to obtain depth of field images and palm print texture images, and combining with dynamic feature analysis of video sequences, it can determine the liveness of the palm, thereby effectively preventing being deceived by non-living palms (such as photos or models); by applying a Gabor filter to extract palm print texture features in multiple scales and directions, and performing principal component analysis (PCA) dimensionality reduction on the extracted features, the dimension of the features is effectively reduced, and the most recognizable key information is retained. At the same time, the computational complexity is reduced, making the recognition speed faster and the performance higher; by calculating the cosine similarity and Euclidean distance, the accurate comparison and evaluation of the palm print features and the depth of field features are realized, and the identity of the current user can be judged more accurately. The advantages of both cosine similarity and Euclidean distance are combined through weighted calculation, which reduces the possibility of misidentification and missed identification and improves the robustness of the recognition results; by combining binocular infrared technology with image analysis methods, it can adapt to different light, environment and angle conditions, thereby ensuring accurate and stable identification of user identities in various practical application environments, avoiding the problem of excessive reliance on environmental factors in traditional technologies, and solving the problem that existing door locks cannot simultaneously meet security, user experience and adapt to complex environments.

[0235] An embodiment of the present invention further provides a storage medium on which a program is stored. When the program is executed by a processor, the door lock control method as described in the above embodiment is implemented.

[0236] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional units or modules as needed, that is, the internal structure of the storage device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the implementation method can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0237] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "storage medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0238] More specific examples of storage media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the storage medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in another suitable manner as necessary, and then stored in the memory.

[0239] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0240] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0241] The above-described embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A door lock control method, characterized in that: The method comprises: Use a binocular infrared camera to synchronously obtain a depth image and a palm print texture image of the user's palm; The depth of field images and palm print texture images acquired within a preset time period are constructed into a video sequence, and dynamic feature analysis is performed on the video sequence to determine whether the user's palm is alive; If the user's palm is judged to be alive, a Gabor filter is applied to the palm print texture image of the current frame to extract the palm print texture feature vectors in multiple scales and directions; Performing principal component analysis and dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector, and calculating the cosine similarity between the generated palmprint feature vector and the preset palmprint feature vector; Extract features from the depth image of the current frame, generate a depth feature vector, and calculate the Euclidean distance between the generated depth feature vector and a preset depth feature vector; A weighted calculation is performed based on the calculated cosine similarity and Euclidean distance to obtain a comprehensive evaluation result of palmprint recognition; Based on the comprehensive evaluation results of palm print recognition, determine whether to perform the door lock unlocking operation.

2. The door lock control method according to claim 1, characterized in that: The step of synchronously acquiring the depth of field image and the palm texture image of the user's palm using a binocular infrared camera comprises: Use a binocular infrared camera to capture the current image in real time and detect whether the user's palm area is captured; If the user's palm area is detected, two cameras in the binocular infrared camera are used to collect two sets of images from different perspectives, and a depth-of-field image of the user's palm is generated based on the parallax calculation of the two sets of images; Use one of the binocular infrared cameras to collect a palm print texture image of the user's palm.

3. The door lock control method according to claim 2, characterized in that: The step of detecting whether the user's palm area is captured comprises: Preprocess the acquired current image, convert it into a grayscale image and perform denoising; Perform skin color detection and background modeling on the preprocessed image to identify multiple candidate areas in the image; Each candidate region is processed according to morphological operations, and the morphological features of each candidate region are evaluated using edge detection processing to determine whether it is the user's palm area.

4. The door lock control method according to claim 1, characterized in that: The step of performing dynamic feature analysis on the video sequence to determine whether the user's palm is alive includes: Performing time series analysis on the video sequence to extract dynamic change features of each frame of the video, wherein the dynamic change features include depth change and texture change of the image; Modeling the dynamic change characteristics of each frame of the image to generate a time series feature vector, wherein the time series feature vector describes the continuous dynamic changes of the palm in the video sequence; The preset liveness judgment model is used to analyze the generated time series feature vector to determine whether the palm is alive.

5. The door lock control method according to claim 1, characterized in that: The step of applying a Gabor filter to the palmprint texture image of the current frame to extract palmprint texture feature vectors in multiple scales and directions comprises: Selecting Gabor filters of multiple scales and directions, wherein each Gabor filter corresponds to different scale and direction parameters, so as to extract palmprint texture features of different scales and directions in the palmprint texture image; Apply each selected Gabor filter to the palmprint texture image of the current frame, and calculate filter response images at different scales and directions, respectively, wherein each filter response image reflects the palmprint texture feature at the corresponding scale and direction; Perform feature extraction on the filter response image at each scale and direction, extract the palmprint texture features in each filter response image, and convert the extracted palmprint texture features into feature vector representation; The palmprint texture feature vectors at all scales and directions are merged to generate a comprehensive palmprint texture feature vector.

6. The door lock control method according to claim 1, characterized in that: The step of performing principal component analysis dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector comprises: Calculating the covariance matrix of the extracted palmprint texture feature vector; Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors of the covariance matrix; Sort the eigenvectors in descending order according to the size of the eigenvalues, and select the first k eigenvectors with the largest eigenvalues ​​as the principal component eigenvectors after dimensionality reduction, where k is the preset dimensionality reduction dimension; The extracted palmprint texture feature vector is projected onto the selected first k principal component feature vectors to generate a low-dimensional palmprint feature vector.

7. The door lock control method according to claim 1, characterized in that: The step of extracting features from the depth image of the current frame to generate a depth feature vector comprises: Divide the depth of field image of the current frame into a plurality of non-overlapping regions; Calculate the statistical characteristics of each divided area, wherein the statistical characteristics include depth average value, standard deviation, maximum value and minimum value; Extracting change features and morphological features in each region, wherein the change features include depth gradients, and the morphological features include connectivity, shape features, and edge features in the region; The statistical features, variation features, and morphological features of each region are spliced ​​together to form the feature vector of each region; The feature vectors of all regions are combined to generate a depth feature vector of the depth image of the current frame.

8. The door lock control method according to claim 1, characterized in that: Before the step of applying the Gabor filter to the palmprint texture image of the current frame, the method further comprises: Performing Laplace filtering on the palmprint texture image of the current frame to obtain an edge-enhanced palmprint texture image; Perform Gaussian filtering and denoising on the edge-enhanced palmprint texture image to obtain a smooth palmprint texture image; Calculate the gradient amplitude and direction of the palmprint texture image after Gaussian filtering, and quantize the gradient direction; Scan each pixel along its gradient direction to determine whether the gradient amplitude of each pixel is greater than the gradient amplitudes of its two adjacent pixel points along the gradient direction; If not, suppress pixels whose gradient magnitude is no greater than that of adjacent pixels; Compare the gradient magnitude of each pixel with a first preset threshold and a second preset threshold; If the gradient amplitude of the target pixel point is less than the first preset threshold, the target pixel point is suppressed; If the gradient amplitude of the target pixel point is greater than the second preset threshold, the target pixel point is retained; If the gradient amplitude of the target pixel is between the first preset threshold and the second preset threshold, whether to retain the target pixel is determined according to whether the gradient amplitude of the adjacent pixel is greater than the second preset threshold.

9. A door lock control system, characterized in that: The system comprises: An image acquisition module is used to synchronously acquire a depth image and a palm print texture image of a user's palm using a binocular infrared camera; A feature analysis module is used to construct a video sequence from the depth of field image and palm print texture image acquired within a preset time period, and perform dynamic feature analysis on the video sequence to determine whether the user's palm is alive; A feature extraction module is used to apply a Gabor filter to the palm print texture image of the current frame to extract palm print texture feature vectors in multiple scales and directions if the feature analysis module determines that the user's palm is alive; The first processing and calculation module is used to perform principal component analysis and dimensionality reduction processing on the extracted palmprint texture feature vector to generate a low-dimensional palmprint feature vector, and calculate the cosine similarity between the generated palmprint feature vector and a preset palmprint feature vector; A second processing and calculation module is used to extract features from the depth of field image of the current frame, generate a depth of field feature vector, and calculate the Euclidean distance between the generated depth of field feature vector and a preset depth of field feature vector; The recognition evaluation module is used to perform weighted calculation based on the calculated cosine similarity and Euclidean distance to obtain a comprehensive evaluation result of palmprint recognition; The door lock control module is used to determine whether to perform the door lock unlocking operation based on the comprehensive evaluation results of palm print recognition.

10. A smart door lock, characterized in that: The invention comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the door lock control method according to any one of claims 1 to 8 is implemented.

Images