Door lock control method and apparatus, and electronic device and computer-readable medium

Abstract

A door lock control method and apparatus (200), and an electronic device (300) and a computer-readable medium. The method comprises: in response to detecting that a user enters a preset recognition range corresponding to a smart door lock, acquiring personal feature information corresponding to the user (step 101); performing blink detection on human eye feature information comprised in the personal feature information, so as to obtain blink frequency information (step 102); in response to determining that the blink frequency information meets a preset frequency condition, performing detection processing on human body feature information, so as to obtain posture feature information and facial feature information (step 103); performing matching processing on the posture feature information and pre-acquired personal posture information, so as to obtain first identity matching information (step 104); in response to determining that the first identity matching information meets a first preset identity matching condition, performing fusion processing on the facial feature information and the human eye feature information, so as to obtain fused feature information corresponding to the user (step 105); performing matching processing on the fused feature information and pre-acquired personal depth feature information, so as to obtain second identity matching information (step 106); and in response to determining that the second identity matching information meets a second preset identity matching condition, sending a preset unlocking instruction to a door lock controller corresponding to the smart door lock, so as to control the unlocking of the door lock (step 107).

Description

Door lock control method, device, electronic device and computer-readable medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a Chinese patent application filed on August 16, 2023, with application number 202311029722.5 and invention name “Door lock control method, device, electronic device and computer-readable medium”, all contents of which are incorporated into this application as a whole. Technical Field

[0003] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a door lock control method, device, electronic device, and computer-readable medium. Background Art

[0004] Door lock control methods are crucial for securely unlocking smart door locks. Currently, door lock control typically involves receiving one or more pieces of user information (e.g., facial information, eye information, or fingerprint information) from the same user for unlocking the smart door lock. The received user information is then compared with pre-registered identity information. Finally, if the information matches, the smart door lock is unlocked.

[0005] Summary of the Invention

[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure provide door lock control methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a door lock control method, the method comprising: in response to detecting that a user enters a preset recognition range corresponding to a smart door lock, obtaining personal characteristic information corresponding to the above-mentioned user, wherein the above-mentioned personal characteristic information includes human body characteristic information and human eye characteristic information; performing blink detection on the human eye characteristic information included in the above-mentioned personal characteristic information to obtain blink frequency information; in response to determining that the above-mentioned blink frequency information meets a preset frequency condition, performing detection and processing on the above-mentioned human body characteristic information to obtain body shape characteristic information and human face characteristic information; matching the above-mentioned body shape characteristic information with a pre-acquired set of personal body shape information to obtain first identity matching information ; In response to determining that the above-mentioned first identity matching information meets the first preset identity matching condition, the above-mentioned facial feature information and the above-mentioned human eye feature information are fused to obtain fused feature information corresponding to the above-mentioned user, wherein the above-mentioned first preset identity matching condition indicates that there is at least one person's body information in the above-mentioned person's body information set that matches the above-mentioned body feature information; the above-mentioned fused feature information is matched with the pre-acquired person's depth feature information set to obtain second identity matching information; in response to determining that the above-mentioned second identity matching information meets the second preset identity matching condition, the preset unlocking instruction is sent to the door lock controller corresponding to the above-mentioned smart door lock for controlling the door lock to unlock.

[0009] In a second aspect, some embodiments of the present disclosure provide a door lock control device, the device comprising: an acquisition unit, configured to, in response to detecting that a user enters a preset recognition range corresponding to a smart door lock, acquire personal characteristic information corresponding to the above-mentioned user, wherein the above-mentioned personal characteristic information includes human body characteristic information and human eye characteristic information; a first detection unit, configured to perform blink detection on the human eye characteristic information included in the above-mentioned personal characteristic information to obtain blink frequency information; a second detection unit, configured to, in response to determining that the above-mentioned blink frequency information meets a preset frequency condition, perform detection processing on the above-mentioned human body characteristic information to obtain body characteristic information and facial characteristic information; a first matching processing The unit is configured to match the above-mentioned body feature information with a pre-acquired body feature information set of persons to obtain first identity matching information; the fusion processing unit is configured to, in response to determining that the above-mentioned first identity matching information meets the first preset identity matching condition, fuse the above-mentioned facial feature information and the above-mentioned eye feature information to obtain fused feature information corresponding to the above-mentioned user, wherein the above-mentioned first preset identity matching condition indicates that there is at least one body feature of persons in the above-mentioned body feature information set that matches the above-mentioned body feature information; the second matching processing unit is configured to match the above-mentioned fused feature information with a pre-acquired depth feature information set of persons to obtain second identity matching information; the sending unit is configured to, in response to determining that the above-mentioned second identity matching information meets the second preset identity matching condition, send a preset unlocking instruction to the door lock controller corresponding to the above-mentioned smart door lock for controlling the door lock to unlock.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: at least one processor; a storage device storing at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements the method described in any implementation of the first aspect above.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] FIG1 is a flow chart of some embodiments of a door lock control method according to the present disclosure;

[0014] FIG2 is a schematic structural diagram of some embodiments of a door lock control device according to the present disclosure;

[0015] FIG3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0018] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "at least one".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] Related door lock control methods, for example, first receive one or more user information (such as facial information, eye information, fingerprint information) of the same user for unlocking a smart door lock, then compare the received user information with pre-entered identity information. Finally, when the information comparison is successful, controlling the smart door lock to unlock often has the following technical problems:

[0022] First, when there are at least two highly similar identity information, if only one user information is used for information comparison, it is easy for the user information to be matched with the wrong identity information, thereby reducing the security of the smart door lock;

[0023] Second, when receiving multiple user information for information comparison, since the multiple user information comes from different sources, if the features of each user information are directly extracted and re-fused, the accuracy of the fused user features will be reduced due to the isolated feature points generated by a single user information, making it difficult to match the correct identity information for secure unlocking, thereby reducing the security of the smart door lock;

[0024] Third, when the user information contains a large amount of noise (such as light spots), if multiple downsampling is performed when the user information is segmented, the spatial feature information in the user information is easily lost, making the segmented user information inaccurate and difficult to match the correct identity information for safe unlocking, thereby reducing the security of the smart door lock.

[0025] In order to solve the problems described above, the door lock control method, device, electronic device and computer-readable medium of some embodiments of the present disclosure can improve the security of smart door locks.

[0026] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0027] FIG1 shows a process 100 of some embodiments of a door lock control method according to the present disclosure. The door lock control method comprises the following steps:

[0028] Step 101, in response to detecting that a user enters a preset recognition range corresponding to the smart door lock, obtaining personal feature information corresponding to the user.

[0029] In some embodiments, the execution entity of the door lock control method (e.g., a computing device) may obtain personal characteristic information corresponding to the user via a wired or wireless connection in response to detecting that the user has entered a preset recognition range corresponding to the smart door lock. The smart door lock may include an infrared sensor, a first camera, a second camera, a display screen, and a door lock controller. The infrared sensor may be a sensor for sensing infrared light emitted by the human body. The first camera may be a wide-angle camera for capturing full-body images of the user. The second camera may be a camera for capturing images of the user's eyes. The display screen may be a touchscreen for manually entering a password string. The door lock controller may be a device for controlling the electromagnetic actuator to unlock the door. The preset recognition range may be a pre-set spatial range within which the user is located when the smart door lock recognizes the user. For example, the preset recognition range may be 0.5 to 1.5 meters in front of the smart door lock. The personal characteristic information may include human body characteristic information and eye characteristic information. The human characteristic information may include a human image sequence. The human image sequence may be a sequence of consecutive human image frames captured by the first camera. The human image may be a full-body image including the face. The eye feature information may include an eye region image sequence. The eye region image sequence may be a sequence of consecutive frames of eye region images captured by the second camera. The eye region image may be a rectangular image including only the eyes.

[0030] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0031] Step 102: Perform blink detection on the eye feature information included in the personnel feature information to obtain blink frequency information.

[0032] In some embodiments, the execution entity may perform blink detection on the eye feature information included in the person feature information to obtain blink frequency information. The blink frequency information may be the number of times a user blinks within a preset duration. The preset duration may be a pre-set duration. For example, the preset duration may be 1.5 seconds, and the blink frequency information may be 3 times. Blink detection may be performed on the eye region image sequence included in the eye feature information in combination with a preset edge detection method and classification method to obtain blink frequency information.

[0033] As an example, the edge detection method may include but is not limited to at least one of the following: a Sobel edge detection operator and a Canny edge detection algorithm. The classification method may include but is not limited to at least one of the following: a support vector machine and a CNN (Convolutional Neural Network) algorithm.

[0034] Step 103 : In response to determining that the blink frequency information meets the preset frequency condition, the human body feature information is detected and processed to obtain body feature information and facial feature information.

[0035] In some embodiments, in response to determining that the blink frequency information satisfies a preset frequency condition, the execution entity may detect and process the body feature information to obtain body shape feature information and facial feature information. The preset frequency condition may be that the number of blinks corresponding to the blink frequency information is not less than a preset frequency. The preset frequency may be a preset lower limit for the number of blinks. The body shape feature information may be a feature matrix representing the user's body shape. The facial feature information may be a facial image of the user. First, from a sequence of body images corresponding to the body feature information, a body image that meets a preset time condition is selected as the target body image. The preset time condition may be that the capture time corresponding to the body image is the latest time among the capture times corresponding to the sequence of body images. Then, edge detection is performed on the target body image using the edge detection algorithm to obtain body shape feature information. Thereafter, face detection is performed on the target body image using a preset face detection method to obtain face detection frame information. The face detection frame information may represent a rectangular detection frame corresponding to the face. The face detection frame information may include the coordinates of the upper left corner, width, and height of the corresponding rectangular detection frame. Finally, based on the face detection frame information, the target human image is cropped along the edge of the corresponding rectangular detection frame to obtain facial feature information.

[0036] As an example, the above-mentioned face detection method may include but is not limited to at least one of the following: a multi-task cascade convolutional neural network, a one-stage face detection network, and a YOLO (You Only Live Once, object detection) algorithm.

[0037] Step 104 : Match the body feature information with the pre-acquired body information set of the person to obtain first identity matching information.

[0038] In some embodiments, the execution entity may match the body shape feature information with a pre-acquired body shape information set to obtain first identity matching information. The body shape information of the person in the body shape information set may be pre-acquired from a human body information document. The human body information document may be a document for storing a feature matrix corresponding to a body shape contour. The body shape information of the person in the body shape information set may include a person identifier and a person body shape feature matrix. The person identifier may be the identifier of the person who has entered the body shape contour information. The person body shape feature matrix may represent the body shape contour information of the corresponding person. The first identity matching information may represent that the similarity between the body shape feature information and at least one person body shape information in the body shape information set is not less than a preset similarity threshold. The preset similarity threshold may be a preset lower limit of the similarity.

[0039] As an example, the execution entity may first perform a similarity analysis on the body feature information and each piece of body shape information in the body shape information set using a preset similarity analysis algorithm to generate similarity values, thereby obtaining a similarity value set. The similarity values ​​in the similarity value set may represent the degree of similarity between the body shape profile of the user and the body shape profile of the corresponding recorded person. Each similarity value in the similarity value set may then be normalized to obtain a target similarity value set. The target similarity values ​​in the target similarity value set may be values ​​between 0 and 1. A larger target similarity value indicates greater similarity. Next, at least one target similarity value that satisfies a preset similarity condition may be selected from the target similarity value set. The preset similarity condition may be that the target similarity value is the maximum value in the target similarity value set. Subsequently, in response to determining that a target similarity value is not less than a preset similarity threshold among the selected at least one target similarity value, the target similarity value that is not less than the preset similarity threshold among the at least one target similarity value is determined as a body similarity value, thereby obtaining a body similarity value set. Finally, the preset match success flag and the person identification group corresponding to the above-mentioned body posture similarity value group are determined as the first identity matching information. The preset match success flag may indicate that the similarity between the above-mentioned body posture feature information and at least one person's body posture information in the above-mentioned person's body posture information set is not less than the above-mentioned preset similarity threshold. Furthermore, in response to determining that the target similarity values ​​in the at least one selected target similarity value are all less than the above-mentioned preset similarity threshold, a preset match failure flag is determined as the first identity matching information. The preset match failure flag may indicate that the similarity between the above-mentioned body posture feature information and the person's body posture information in the above-mentioned person's body posture information set is all less than the above-mentioned preset similarity threshold.

[0040] As an example, the similarity analysis algorithm may be Euclidean distance.

[0041] Step 105 : In response to determining that the first identity matching information satisfies the first preset identity matching condition, the facial feature information and the eye feature information are fused to obtain fused feature information corresponding to the user.

[0042] In some embodiments, in response to determining that the first identity matching information satisfies a first preset identity matching condition, the execution entity may fuse the facial feature information and the eye feature information through various methods to obtain fused feature information corresponding to the user. The first preset identity matching condition may indicate that at least one person's body shape information in the set of body shape information matches the body shape feature information. The fused feature information may be a feature vector representing the user's facial features and iris features.

[0043] In some optional implementations of some embodiments, the execution entity may fuse the facial feature information and the eye feature information to obtain fused feature information corresponding to the user through the following steps:

[0044] The first step is to select an eye area image that meets a preset eye state condition from each eye area image corresponding to the above-mentioned human eye feature information. The above-mentioned preset eye state condition may be that both eyes are in an open state. First, the eye state information corresponding to each eye area image can be determined by the above-mentioned edge detection method to obtain an eye state information set. The eye state information in the above-mentioned eye state information set can represent the eye state. The eye state can be one of the following: open, closed. Then, an eye area image corresponding to the target eye state information is selected from each eye area image. The above-mentioned target eye state information may be eye state information representing that the eyes are in an open state.

[0045] The second step is to generate a human eye image group based on the selected eye area image. The above-mentioned human eye image group may include a human eye image corresponding to the left eye and a human eye image corresponding to the right eye. First, the selected eye area image is detected using a preset target detection method to obtain left eye detection frame information and right eye detection frame information. The above-mentioned left eye detection frame information can represent the rectangular detection frame corresponding to the user's left eye. The above-mentioned left eye detection frame information may include the upper left corner coordinates, width and height of the corresponding rectangular detection frame. The above-mentioned right eye detection frame information can represent the rectangular detection frame corresponding to the user's right eye. The above-mentioned right eye detection frame information may include the upper left corner coordinates, width and height of the corresponding rectangular detection frame. Then, according to the above-mentioned left eye detection frame information and right eye detection frame information, each human eye image included in the above-mentioned human eye image group is cropped along the edge of the corresponding detection frame to obtain a human eye image group.

[0046] As an example, the target detection method may be the YOLO algorithm.

[0047] In the third step, each eye image in the eye image group is segmented to obtain an iris region image group. The iris region image group may include an iris region image corresponding to the left eye and an iris region image corresponding to the right eye. The iris region image may be a binary image of the same size as the corresponding eye image. In the iris region image, the iris region may be represented by 1, and the non-iris region may be represented by 0. Various methods can be used to segment each eye image in the eye image group to obtain the iris region image group.

[0048] In some optional implementations of some embodiments, the execution entity may segment each human eye image in the human eye image group to obtain an iris region image group through the following steps:

[0049] Step 1: For each of the above human eye images, perform the following steps to generate an iris region image in the iris region image group:

[0050] Sub-step 1: Input the eye image into the first convolutional processing layer of a pre-trained iris region image generation model to perform convolution processing on the eye image to obtain a first eye feature map. The iris region image generation model may further include a second convolutional processing layer, a third convolutional processing layer, an eye feature encoding model, an eye feature decoding model, and a pooling layer. The eye feature encoding model and the eye feature decoding model may be convolutional neural network models.

[0051] As an example, the execution entity may perform convolution processing on the human eye image through a 4*4 convolution kernel to obtain a first human eye feature map.

[0052] Sub-step two: inputting the first human eye feature map into the human eye feature encoding model included in the iris area image generation model to generate a second human eye feature map.

[0053] In some optional implementations of some embodiments, the above-mentioned human eye feature encoding model may include a first feature extraction module, a second feature extraction module, and a third feature extraction module. Among them, the above-mentioned first feature extraction module, the second feature extraction module, and the third feature extraction module may be residual blocks with the same structure. The above-mentioned first feature extraction module, the second feature extraction module, and the third feature extraction module each include a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer. The first convolutional layer and the fourth convolutional layer may be depth-separable convolutional layers. The above-mentioned execution entity may generate a second human eye feature map by the following steps:

[0054] Step 1: Input the first human eye feature map into the first feature extraction module included in the human eye feature encoding model to obtain a first target feature map. Specifically, the following steps can be performed:

[0055] In the first step, the first eye feature map is input into the first convolutional layer corresponding to the first feature extraction module to sequentially perform depthwise separable convolution processing, normalization processing, and activation processing on the first eye feature map to obtain a first convolutional feature map. The depthwise separable convolution processing, normalization processing, and activation processing can be sequentially performed on the first eye feature map based on a first preset convolution kernel, a preset normalization processing algorithm, and an activation processing algorithm to obtain the first convolutional feature map.

[0056] As an example, the first preset convolution kernel may be a 3*3 convolution kernel. The normalization processing algorithm may be a BN (Batch Normalization) algorithm. The activation processing method may be a ReLU6 (Rectified Linear Unit 6, suppressing its maximum value) function.

[0057] In the second step, the first convolutional feature map is input into the second convolutional layer corresponding to the first feature extraction module to sequentially perform convolution processing and normalization processing on the first convolutional feature map to obtain a second convolutional feature map. The first convolutional feature map can be sequentially convolutionally and normalized according to a second preset convolution kernel and the normalization processing algorithm to obtain the second convolutional feature map. For example, the preset second convolution kernel can be a 1*1 convolution kernel.

[0058] In a third step, the second convolutional feature map is input into the third convolutional layer corresponding to the first feature extraction module to sequentially perform convolution processing, normalization processing, and activation processing on the second convolutional feature map to obtain a third convolutional feature map. The third convolutional feature map can be obtained by sequentially performing convolution processing, normalization processing, and activation processing on the second convolutional feature map based on the second preset convolution kernel, the normalization processing algorithm, and the activation processing method.

[0059] In a fourth step, the third convolutional feature map is input into the fourth convolutional layer corresponding to the first feature extraction module to sequentially perform depthwise separable convolution processing and normalization processing on the third convolutional feature map to obtain a first target feature map. The first target feature map can be obtained by sequentially performing depthwise separable convolution processing and normalization processing on the third convolutional feature map based on the first preset convolution kernel and the normalization processing algorithm.

[0060] Step 2: Input the first target feature map into the second feature extraction module included in the human eye feature encoding model to obtain a second target feature map. Specifically, the following steps can be performed:

[0061] In the first step, the first target feature map is input into the first convolution layer corresponding to the second feature extraction module to sequentially perform depthwise separable convolution processing, normalization processing, and activation processing on the first target feature map to obtain a fourth convolution feature map. The fourth convolution feature map can be obtained by sequentially performing depthwise separable convolution processing, normalization processing, and activation processing on the first target feature map based on the first preset convolution kernel, the normalization processing algorithm, and the activation processing algorithm.

[0062] In a second step, the fourth convolutional feature map is input into the second convolutional layer corresponding to the second feature extraction module to sequentially perform convolution processing and normalization processing on the fourth convolutional feature map to obtain a fifth convolutional feature map. The fifth convolutional feature map can be obtained by sequentially performing convolution processing and normalization processing on the fourth convolutional feature map according to the second preset convolution kernel and the normalization processing algorithm.

[0063] In a third step, the fifth convolutional feature map is input into the third convolutional layer corresponding to the second feature extraction module to sequentially perform convolution processing, normalization processing, and activation processing on the fifth convolutional feature map to obtain a sixth convolutional feature map. The sixth convolutional feature map can be obtained by sequentially performing convolution processing, normalization processing, and activation processing on the fifth convolutional feature map based on the second preset convolution kernel, the normalization processing algorithm, and the activation processing method.

[0064] In a fourth step, the sixth convolutional feature map is input into the fourth convolutional layer corresponding to the second feature extraction module to sequentially perform depthwise separable convolution processing and normalization processing on the sixth convolutional feature map to obtain a seventh convolutional feature map. The seventh convolutional feature map can be obtained by sequentially performing depthwise separable convolution processing and normalization processing on the sixth convolutional feature map based on the first preset convolution kernel and the normalization processing algorithm.

[0065] In the fifth step, an addition process is performed on the feature map input into the second feature extraction module and the seventh convolution feature map to obtain a second target feature map.

[0066] Step 3: Input the second target feature map into the third feature extraction module included in the human eye feature encoding model to obtain a third target feature map. Specifically, the following steps can be performed:

[0067] In the first step, the second target feature map is input into the first convolution layer corresponding to the third feature extraction module to sequentially perform depthwise separable convolution processing, normalization processing, and activation processing on the second target feature map to obtain an eighth convolution feature map. The eighth convolution feature map can be obtained by sequentially performing depthwise separable convolution processing, normalization processing, and activation processing on the second target feature map according to the first preset convolution kernel, the normalization processing algorithm, and the activation processing algorithm.

[0068] In a second step, the eighth convolutional feature map is input into the second convolutional layer corresponding to the third feature extraction module to sequentially perform convolution processing and normalization processing on the eighth convolutional feature map to obtain a ninth convolutional feature map. The ninth convolutional feature map can be obtained by sequentially performing convolution processing and normalization processing on the eighth convolutional feature map according to the second preset convolution kernel and the normalization processing algorithm.

[0069] In a third step, the ninth convolutional feature map is input into the third convolutional layer corresponding to the third feature extraction module to sequentially perform convolution processing, normalization processing, and activation processing on the ninth convolutional feature map to obtain a tenth convolutional feature map. The tenth convolutional feature map can be obtained by sequentially performing convolution processing, normalization processing, and activation processing on the ninth convolutional feature map according to the second preset convolution kernel, the normalization processing algorithm, and the activation processing method.

[0070] In a fourth step, the tenth convolutional feature map is input into the fourth convolutional layer corresponding to the third feature extraction module to sequentially perform depthwise separable convolution processing and normalization processing on the tenth convolutional feature map to obtain a third target feature map. The third target feature map can be obtained by sequentially performing depthwise separable convolution processing and normalization processing on the tenth convolutional feature map based on the first preset convolution kernel and the normalization processing algorithm.

[0071] Step 4: Input the third target feature map into the second feature extraction module included in the human eye feature encoding model to obtain a fourth target feature map. Step 4 can refer to the second target feature map generation process in step 2.

[0072] Step 5: Input the fourth target feature map into the third feature extraction module included in the human eye feature encoding model to obtain a fifth target feature map. Step 5 can refer to the third target feature map generation process of step 3.

[0073] Step 6: Input the fifth target feature map into the second feature extraction module included in the human eye feature encoding model to obtain a second human eye feature map. Step 6 can refer to the second target feature map generation process of step 2.

[0074] Sub-step three: input the second human eye feature map into the second convolution processing layer included in the iris area image generation model to perform convolution processing on the second human eye feature map to obtain a third human eye feature map.

[0075] Sub-step four: normalize the third eye feature map to obtain the first attention feature map.

[0076] As an example, the execution entity may normalize the third human eye feature map through a sigmoid activation function to obtain a first attention feature map.

[0077] Sub-step five: multiplying the first attention feature map and the second eye feature map to obtain a fourth eye feature map.

[0078] Sub-step six: input the second eye feature map into the pooling layer included in the iris region image generation model to perform pooling processing on the second eye feature map to obtain a fifth eye feature map. The pooling processing may be a global average pooling processing.

[0079] Sub-step seven: input the fifth human eye feature map into the third convolution processing layer included in the iris area image generation model to perform convolution processing on the fifth human eye feature map to obtain a sixth human eye feature map.

[0080] Sub-step eight: normalize the sixth eye feature map to obtain a second attention feature map.

[0081] Sub-step nine: multiplying the second attention feature map and the second human eye feature map to obtain a seventh human eye feature map.

[0082] Sub-step ten: generating a fused human eye feature map based on the fourth human eye feature map and the seventh human eye feature map.

[0083] As an example, the execution entity may perform addition processing on the fourth human eye feature map and the seventh human eye feature map to obtain a fused human eye feature map.

[0084] In sub-step 11, the first eye feature map and the fused eye feature map are input into an eye feature decoding model included in the iris region image generation model to generate an iris region image. For example, the eye feature decoding model may be a decoding module included in a CE-Net (Context Encoder Network) model.

[0085] Optionally, the iris region image generation model may be trained by the following steps:

[0086] The first step is to obtain a user eye sample set, wherein the user eye samples in the user eye sample set may include sample eye images and sample iris area images.

[0087] In the second step, user eye samples are selected from the above user eye sample set and the following training steps are performed:

[0088] In the first sub-step, the sample eye image of the selected user eye sample is input into an initial iris region image generation model to obtain an iris region image corresponding to the user eye sample. The initial iris region image generation model may be a residual network model.

[0089] The second sub-step is to compare the iris area image corresponding to the user's eye sample with the corresponding sample iris area image.

[0090] The third sub-step is to determine whether the initial iris area image generation model has achieved a preset optimization target based on the comparison result. The optimization target may be that the accuracy of the iris area image generated by the initial iris area image generation model is greater than a preset accuracy threshold.

[0091] In a fourth sub-step, in response to determining that the initial iris area image generation model achieves the above-mentioned optimization goal, the initial iris area image generation model is determined as the trained iris area image generation model.

[0092] Optionally, in response to determining that the initial iris area image generation model does not achieve the optimization goal, the execution entity may adjust relevant parameters in the initial iris area image generation model, reselect samples from the user eye sample set, and use the adjusted initial iris area image generation model as the initial iris area image generation model to continue the training step. The initial iris area image generation model may be adjusted using a backpropagation algorithm and a stochastic gradient descent method.

[0093] The above-mentioned segmentation process and its related contents, as an inventive feature of the embodiments of this disclosure, address the third technical issue mentioned in the background technology: "reduced security of smart door locks." This reduced security of smart door locks is often caused by the following: when user information contains a large amount of noise, multiple downsampling during the segmentation process can lead to a gradual loss of spatial feature information in the user information. This results in insufficient accuracy of the segmented user information, making it difficult to match the correct identity information for secure unlocking. If this issue is resolved, the security of the smart door lock can be improved. To achieve this, first, convolution processing is performed on the eye image to obtain a first eye feature map. This allows the eye image to be downsampled to obtain the first eye feature map. Second, the first eye feature map is input into a pre-trained eye feature encoding model to generate a second eye feature map. This allows further downsampling of the first eye feature map to extract features. Then, convolution processing is performed on the second eye feature map to obtain a third eye feature map. This third eye feature map is normalized to obtain a first attention feature map. In this way, the number of channels in the feature map can be compressed, and then normalized to obtain a weighted feature map based on spatial attention corresponding to the feature map. Next, the first attention feature map and the second eye feature map are multiplied to obtain a fourth eye feature map. Based on the spatial attention weighted feature map, the second eye feature map can be corrected and updated to reflect spatial information. Next, the second eye feature map is pooled to obtain a fifth eye feature map. The fifth eye feature map is convolved to obtain a sixth eye feature map. The sixth eye feature map is normalized to obtain a second attention feature map. Pooling can capture global information from the feature map and further obtain weighted feature maps corresponding to each channel. Next, the second attention feature map and the second eye feature map are multiplied to obtain a seventh eye feature map. This allows for adjustment of channel information. Next, a fused eye feature map is generated based on the fourth and seventh eye feature maps. This allows for information fusion and highlighting of the iris region. Finally, the first eye feature map and the fused eye feature map are input into a preset eye feature decoding model to generate an iris region image. This results in a binary iris region image that can be decoded. Therefore, by reducing the number of image downsampling times and combining the spatial attention mechanism with the channel attention mechanism, the spatial feature information in the user information can be corrected to reduce feature loss and reduce responsiveness to irrelevant noise, thereby improving the accuracy of the segmented user information. This allows for more accurate matching of correct identity information for secure unlocking, thereby enhancing the security of the smart door lock.

[0094] The fourth step is to crop each iris area image in the iris area image group to obtain a target iris image group. The target iris images in the target iris image group may be images of the area occupied by the iris' corresponding circumscribed rectangle. First, for each iris area image in the iris area image group, a circumscribed rectangular area is determined based on a preset rectangle circumscribing algorithm. The target iris image is then cropped along the edges of the circumscribed rectangular area.

[0095] In the fifth step, the facial image corresponding to the facial feature information is normalized to obtain a target facial image. The target facial image may be a facial image of the same size as the target iris image. The facial image corresponding to the facial feature information may be resized to obtain a facial image of the same size as the target iris image, serving as the target facial image.

[0096] In the sixth step, each target iris image in the target iris image group and the target face image are determined as images to be identified, thereby obtaining an image group to be identified.

[0097] In the seventh step, the image set to be identified is input into a pre-trained fusion feature information generation model to generate fusion feature information. The fusion feature information generation model can be a convolutional neural network model that takes the image set to be identified as input and outputs the fusion feature information.

[0098] In some optional implementations of some embodiments, the above-mentioned fused feature information generation model may include a convolution layer, a feature extraction layer, a pooling layer and a fully connected layer. Among them, the above-mentioned feature extraction layer may include a key feature extraction module. The above-mentioned key feature extraction module may include a first feature extraction submodule, a second feature extraction submodule and a third feature extraction submodule. The above-mentioned first feature extraction submodule and the above-mentioned second feature extraction submodule have the same structure, and both may include 2 operator fusion layers and 1 global average pooling layer. Among them, the above-mentioned operator fusion layer may include a BN (Batch Normalization) layer, a ReLU (Rectified Layer Units, non-linear unit), and a Convolution layer. The above-mentioned third feature extraction submodule may include 1 operator fusion layer and 1 global average pooling layer. The above-mentioned execution entity may generate fused feature information through the following steps:

[0099] Step 1: For each image to be identified in the above group of images to be identified, perform the following steps:

[0100] Sub-step one: input the above-mentioned image to be identified into the convolution layer included in the above-mentioned fusion feature information generation model to generate a first feature map.

[0101] As an example, the convolution layer included in the above-mentioned fusion feature information generation model can perform convolution processing on the above-mentioned image to be recognized through a 5*5 convolution kernel to obtain a first feature map.

[0102] Sub-step 2: Input the first feature map into the feature extraction layer included in the fusion feature information generation model to generate a high-level feature map sequence. The high-level feature maps in the high-level feature map sequence can be feature maps with richer semantic information and a larger receptive field. The first feature map can be used as the input feature map, and based on the input feature map, the following high-level feature map sequence generation steps are performed:

[0103] Step 1: Input the input feature map into the key feature extraction module in the feature extraction layer included in the fusion feature information generation model to generate a high-level feature map, and add the high-level feature map to the end of the preset update feature map sequence. The update feature map sequence is a variable-length sequence and is initially empty. The high-level feature map can be generated by the following steps:

[0104] Sub-step 1: Input the above-mentioned first feature map into the first feature extraction submodule included in the key feature extraction module to output the first key feature map. First, the above-mentioned first feature map is pre-processed by the two operator fusion layers included in the above-mentioned first feature extraction submodule to obtain the operator fusion feature map. Then, the above-mentioned first feature map is pooled by the global average pooling layer included in the above-mentioned first feature extraction submodule to obtain the pooled feature map. Thereafter, the above-mentioned pooled feature map is subjected to one-dimensional discrete convolution processing to obtain the channel weight value feature map. Then, the above-mentioned first feature map is multiplied by the above-mentioned channel weight value feature map to obtain the updated first feature map. Finally, the above-mentioned operator fusion feature map and the above-mentioned updated first feature map are determined as the first key feature map.

[0105] Sub-step 2: input the second key feature map into the second feature extraction submodule included in the key feature extraction module to output the second key feature map. Wherein, the above sub-step 2 can be performed with reference to sub-step 1 to output the second key feature map.

[0106] In sub-step 3, the second key feature map is input into a third feature extraction submodule included in the key feature extraction module to output a high-level feature map. First, the third feature extraction submodule may perform preprocessing and pooling on the second key feature map in sequence through an operator fusion layer and a global average pooling layer to obtain a high-level feature map.

[0107] Step 2: In response to determining that the number of updated feature maps in the updated feature map sequence is equal to a preset feature map threshold, determining the updated feature map sequence as a high-level feature map sequence. The preset feature map threshold may be a preset upper limit on the number of high-level feature maps to be generated.

[0108] Step 3: In response to determining that the number of each updated feature map in the above-mentioned updated feature map sequence is not greater than the preset feature map threshold, the first updated feature map at the end of the above-mentioned updated feature map sequence is used as the input feature map, and the above-mentioned high-level feature map sequence generation step is performed again.

[0109] Sub-step three: determining a high-level feature map in the high-level feature map sequence that satisfies a preset sequence number condition as the second feature map, wherein the preset sequence number condition may be that the sequence number corresponding to the high-level feature map is the maximum value among the sequence numbers corresponding to the high-level feature map sequence.

[0110] Sub-step 4: Input the second feature map into the pooling layer included in the fused feature information generation model to generate a third feature map. The pooling layer included in the fused feature information generation model may perform global average pooling on the second feature map to obtain the third feature map.

[0111] In sub-step five, the third feature map is input into a fully connected layer included in the fused feature information generation model to generate a target feature vector. The fully connected layer included in the fused feature information generation model may perform a fully connected process on the target feature vector to obtain the target feature vector.

[0112] Step 2: Perform linear processing on each of the obtained first feature maps to obtain a first fused feature map. A weighted summation can be performed on each of the obtained first feature maps to obtain the first fused feature map.

[0113] Step 3: Based on the feature extraction layer included in the fused feature information generation model, feature extraction processing is performed on the first fused feature map to obtain a high-level fused feature map. The first fused feature map can be input into a key feature extraction module in the feature extraction layer included in the fused feature information generation model to perform feature extraction processing on the first fused feature map to obtain a high-level fused feature map.

[0114] Step 4: Generate a second fused feature map based on the obtained high-level feature map sequences and the high-level fused feature map. The second fused feature map can represent fused features corresponding to iris features and facial features. First, the difference between the length of any high-level feature map sequence and 1 is determined as the number of fusions. Then, a preset count value is initialized to 1, and based on the count value and the high-level fused feature map, the following steps for generating the second fused feature map are performed:

[0115] Sub-step 1: Select high-level feature maps with the same sequence number and count value from each high-level feature map sequence and determine them as the target high-level feature map group.

[0116] Sub-step 2: performing weighted summation on the high-level fusion feature map and each target high-level feature map in the target high-level feature map group to obtain a target high-level fusion feature map.

[0117] Sub-step three: input the target high-level fusion feature map into the key feature extraction module in the feature extraction layer to generate an extracted high-level fusion feature map. The step of generating the extracted high-level fusion feature map can refer to the step of generating the high-level feature map.

[0118] Sub-step four: in response to determining that the count value is equal to the above-mentioned fusion number, determining the above-mentioned extracted high-level fusion feature map as the second fusion feature map.

[0119] Optionally, the above-mentioned execution entity can also determine the sum of the count value and 1 as the count update value in response to determining that the count value is less than the above-mentioned fusion times, and use the above-mentioned count update value as the count value and the above-mentioned extracted high-level fusion feature map as the high-level fusion feature map, and execute the above-mentioned second fusion feature map generation step again.

[0120] Step 5: Input the second fused feature map into the pooling layer included in the fused feature information generation model to generate a target fused feature map. In this case, a global average pooling process can be performed on the second fused feature map to obtain the target fused feature map.

[0121] Step 6: Input the target fusion feature map into the fully connected layer included in the fusion feature information generation model to generate a target feature fusion vector. In this case, the target fusion feature map can be fully connected to obtain the target feature fusion vector.

[0122] Step 7: Perform linear processing on the obtained target feature vectors and the target feature fusion vector to obtain a linear fusion vector. The linear fusion vector can be obtained by performing weighted summation on the obtained target feature vectors and the target feature fusion vector.

[0123] Step 7: Normalize the linear fusion vector to obtain fusion feature information. The linear fusion vector can be normalized by using a normalized exponential function to obtain fusion feature information.

[0124] Optionally, the above fusion feature information generation model can be trained through the following steps:

[0125] The first step is to obtain a user sample set, wherein the user samples in the user sample set may include a sample face image, a sample left eye iris image, a sample left eye iris image, and sample fusion feature information.

[0126] The second step is to select user samples from the above user sample set and perform the following training steps:

[0127] In the first sub-step, the sample face image, the sample left eye iris image and the sample left eye iris image of the selected user sample are respectively used as sample recognition images to obtain a sample recognition image group.

[0128] In the second sub-step, the sample recognition image group is input into an initial fusion feature information generation model to obtain fusion feature information corresponding to the user sample. The initial fusion feature information generation model may be a convolutional neural network model that can obtain fusion feature information based on multiple images to be recognized.

[0129] The third sub-step is to compare the fused feature information corresponding to the user sample with the corresponding sample fused feature information. The fused feature information corresponding to the user sample and the corresponding sample fused feature information can be compared using a cross entropy loss function.

[0130] The fourth sub-step is to determine whether the initial fused feature information generation model has achieved a preset optimization target based on the comparison result. The optimization target may be that the accuracy of the fused feature information generated by the initial fused feature information generation model is greater than a preset feature accuracy threshold.

[0131] In a fifth sub-step, in response to determining that the initial fused feature information generation model achieves the above-mentioned optimization goal, the initial fused feature information generation model is determined as the trained fused feature information generation model.

[0132] Optionally, in response to determining that the initial fused feature information generation model does not achieve the optimization goal, the execution entity may adjust relevant parameters in the initial fused feature information generation model, reselect samples from the user sample set, and use the adjusted initial fused feature information generation model as the initial fused feature information generation model to continue the training step. The initial fused feature information generation model may be adjusted using a backpropagation algorithm and a stochastic gradient descent method.

[0133] The above-mentioned fused feature information generation step and its related content, as an inventive feature of an embodiment of the present disclosure, address the second technical issue mentioned in the background technology: reduced security of smart door locks. This reduced security often occurs when receiving multiple user information for comparison. Since these information sources vary, directly extracting and re-fusing features from each user information can easily lead to isolated feature points generated by a single user information, reducing the accuracy of the fused user features and making it difficult to match the correct identity information for secure unlocking. If this issue is resolved, the security of the smart door lock can be improved. To achieve this, first, the facial image and iris image are normalized to the same size through size normalization. This preserves stable features of the facial image, such as the position, shape, and size of facial features, while discarding unstable features, such as facial texture. Second, for each of the facial and iris images, a convolution process is performed to extract coarse features, generating a first feature map. Based on this first feature map, multiple further feature extractions are performed to generate a sequence of high-level feature maps. Afterwards, for each feature map corresponding to the facial image and iris image, a fused feature map corresponding to the previous generation time can be obtained by weighted summing the feature maps corresponding to the aforementioned generation time and the fused feature map corresponding to the previous generation time. This facilitates the subsequent enhancement of the semantic fusion of facial and iris features through fused features. Next, the finally generated high-level feature map and fused feature map corresponding to the facial image and iris image are subjected to global pooling and full connection processing, respectively. This allows dimensionality reduction and classification to be performed on each feature map. Finally, fused feature information can be obtained by weighted summing and normalizing the feature vectors after the full connection processing. This allows the semantic fusion of facial and iris features to be enhanced through fused feature information. Therefore, when the sources of multiple user information are different, the feature fusion of each user information can be achieved by combining the above-mentioned image normalization processing and time-divided feature map fusion processing. This can reduce the probability of isolated feature points generated by a single user information and improve the accuracy of the fused user features. As a result, the correct identity information can be matched more accurately for safe unlocking, thereby improving the security of the smart door lock.

[0134] Step 106 : Match the fused feature information with the pre-acquired person deep feature information set to obtain second identity matching information.

[0135] In some embodiments, the execution entity may perform matching processing on the fused feature information and the pre-acquired personnel depth feature information set to obtain second identity matching information. The personnel depth feature information in the personnel depth feature information set may be information of vectors representing the facial and iris features of the corresponding user obtained in advance from the identity information database. The personnel depth feature information in the personnel depth feature information set may include a personnel identification and a depth feature vector. The depth feature vector may be a vector representing the facial and iris features of the user. The identity information database may be a database storing identity information. The identity information may include but is not limited to at least one of the following: facial feature information, personnel eye information, and personnel depth feature information. The second identity matching information may represent whether there is personnel depth feature information in the personnel depth feature information set whose similarity with the fused feature information is greater than or equal to a preset depth feature similarity value. The fused feature information may be matched with the pre-acquired personnel depth feature information set to obtain the second identity matching information through the following steps:

[0136] The first step is to select, from the above-mentioned personnel depth feature information set, personnel depth feature information whose personnel identifier is the same as any personnel identifier corresponding to the above-mentioned first identity matching information, to obtain at least one personnel depth feature information.

[0137] In the second step, the similarity analysis algorithm is used to perform a similarity analysis on the fused feature information and each of the at least one person depth feature information to obtain at least one depth feature similarity value. The depth feature similarity value in the at least one depth feature similarity value may represent the degree of similarity between the user's facial and iris features and the pre-recorded facial and iris features.

[0138] In the third step, each depth feature similarity value in the above depth feature similarity value set is normalized to obtain a normalized similarity value set. The normalized similarity values ​​in the above normalized similarity value set can be values ​​between 0 and 1. A larger normalized similarity value indicates greater similarity.

[0139] In the fourth step, a normalized similarity value greater than or equal to a preset depth feature similarity value is selected from the normalized similarity value set, wherein the preset depth feature similarity value may be a preset similarity value corresponding to the face and iris features.

[0140] In a fifth step, in response to determining that the selected normalized similarity value is not null, the person identifier corresponding to the selected normalized similarity value and the preset depth feature match success flag are determined as second identity matching information. The preset depth feature match success flag may indicate that the person depth feature information set contains person depth feature information whose similarity to the fused feature information is not less than the preset depth feature similarity value.

[0141] Optionally, the execution entity may further determine, in response to determining that the selected normalized similarity value is null, a preset depth feature matching failure flag as the second identity matching information. The preset depth feature matching failure flag may indicate that there is no depth feature information of a person whose similarity to the fused feature information is greater than or equal to a preset depth feature similarity value in the person depth feature information set.

[0142] Step 107: In response to determining that the second identity matching information satisfies the second preset identity matching condition, a preset unlocking instruction is sent to a door lock controller corresponding to the smart door lock for controlling the door lock to unlock.

[0143] In some embodiments, the execution entity may, in response to determining that the second identity matching information satisfies a second preset identity matching condition, send a preset unlocking instruction to the door lock controller corresponding to the smart door lock to control the door lock to unlock. The second preset identity matching condition may be that the second identity matching information includes a preset deep feature matching success indicator. The preset unlocking instruction may be a pre-set instruction for instructing the door lock controller to control the door lock to unlock. For example, the unlocking instruction may be "execute unlock." The door lock controller corresponding to the smart door lock may receive the unlocking instruction and control the electromagnetic actuator to unlock the door lock.

[0144] Optionally, the above execution entity may further perform the following steps:

[0145] The first step is, in response to determining that the above-mentioned first identity matching information does not meet the above-mentioned first preset identity matching condition, matching processing is performed on the above-mentioned human eye feature information and the pre-acquired personnel eye information set to obtain third identity matching information. Among them, the personnel eye information in the above-mentioned personnel eye information set may be information of a vector that characterizes the iris features of the corresponding user, which is pre-acquired from the identity information database. The personnel eye information in the above-mentioned personnel eye information set may include a personnel identification and an iris feature vector. The above-mentioned iris feature vector may be a vector that characterizes the iris features of the user. The above-mentioned third identity matching information may characterize whether there is personnel eye information in the above-mentioned personnel eye information set whose similarity with the above-mentioned human eye feature information is equal to or greater than a preset eye feature similarity value. The above-mentioned human eye feature information and the pre-acquired personnel eye information set may be matched to obtain the third identity matching information through the following steps:

[0146] In the first sub-step, a preset iris feature extraction model is used to extract the eye feature information to obtain an eye iris feature vector, wherein the eye iris feature vector can represent the features of the user's eye iris.

[0147] As an example, the iris feature extraction model may be an IrisCodeNet (Iris Feature Coding Network) model.

[0148] In the second sub-step, the similarity analysis algorithm is used to perform a similarity analysis on the eye feature information and the eye information of each person in the eye information set to generate an eye similarity value, thereby obtaining an eye similarity value set. The eye similarity values ​​in the eye similarity value set can represent the degree of similarity between the user's iris features and the pre-recorded iris features.

[0149] The third sub-step is to normalize each eye similarity value in the eye similarity value set to obtain a normalized eye similarity value set, wherein the normalized eye similarity value in the normalized eye similarity value set may be a value between 0 and 1.

[0150] The fourth sub-step is to select a normalized eye similarity value that is not less than a preset eye similarity value from the normalized eye similarity value set, wherein the preset eye similarity value may be a preset similarity value corresponding to an iris feature.

[0151] In a fifth sub-step, in response to determining that the selected normalized eye similarity value is not null, the person identifier corresponding to the selected normalized eye similarity value and a preset eye match success flag are determined as third identity matching information. The preset eye match success flag may indicate that the person eye information set contains person eye information whose similarity to the human eye feature information is not less than a preset eye feature similarity value.

[0152] Optionally, in response to determining that the selected normalized eye similarity value is null, the execution entity may further determine a preset eye matching failure flag as the third identity matching information. The preset eye matching failure flag may indicate that the eye information set does not contain any eye information of a person whose similarity to the eye feature information is greater than or equal to a preset eye feature similarity value.

[0153] In the second step, in response to determining that the third identity matching information satisfies the third preset identity matching condition, the associated voice device is controlled to play a preset password input prompt. The third preset identity matching condition may include the third identity matching information including a preset eye match success indicator. The voice device may be a device capable of playing and receiving voice. The password input prompt may be a pre-set voice prompt for password entry. For example, the password input prompt may be "Your identity information has expired. Please enter your password to confirm."

[0154] The third step is to verify the password key information in response to receiving the password key information input by the user and obtain password verification information. The password key information may be the character string input by the user through the display screen of the smart door lock. The password verification information may indicate whether the password key information is correct. First, a string matching process is performed on the password key information and the pre-stored door lock password information. The door lock password information may be a character string used to unlock the smart door lock. Then, in response to determining that the password key information is the same as the door lock password information, a preset verification pass flag is determined as the password verification information. The preset verification pass flag may indicate that the password key information is correct. Finally, in response to determining that the password key information is not the same as the door lock password information, a preset verification failure flag is determined as the password verification information. The preset verification failure flag may indicate that the password key information is incorrect.

[0155] In the fourth step, in response to determining that the password verification information satisfies a preset verification error condition, the unlocking instruction is sent to the door lock controller to control the door lock to unlock. The preset verification error condition may be that the password verification information indicates that the password key information is correct.

[0156] In practice, when the user's body shape changes, for example, when a user who has been away from home for a long time and has not used the above-mentioned smart door lock grows taller, gains weight or loses weight, it is easy for the body feature information to fail to match. At this time, the facial features will usually change significantly. Therefore, the user is initially identified by iris feature recognition, and then the password input is used for secondary identification, which can better confirm the user's identity and improve the security of the smart door lock.

[0157] Optionally, in response to determining that the second identity matching information does not meet the second preset identity matching condition, the execution entity may control an associated voice device to play a preset security alert message. The security alert message may include "Unknown visitor, please check." The execution entity may control the associated voice device to play the security alert message to the person inside the door. After receiving the security alert, the person inside the door can view the door status through an associated display terminal. The display terminal may be, but is not limited to, one of the following: a mobile phone or an intercom with a display screen.

[0158] Optionally, the first identity matching information may include body posture distinguishing feature information. The body posture distinguishing feature information may be a matrix representing the difference between a feature matrix corresponding to the body posture feature information and a feature matrix corresponding to a target body posture feature. The target body posture feature may be a pre-stored body posture feature matrix used for comparison with a body posture captured by a camera. The execution subject may further perform the following steps:

[0159] In the first step, in response to determining that the door lock has been unlocked, a historical sequence of distinguishing body posture characteristics corresponding to the user is obtained. The historical sequence of distinguishing body posture characteristics may be an ordered set of distinguishing body posture characteristics from a historical period. The historical period may be a date range of seven consecutive days prior to the current date.

[0160] The second step is to perform a similarity analysis on the posture distinguishing feature information and the historical posture distinguishing feature information sequence to obtain a posture change similarity value group. The posture change similarity values ​​in the posture change similarity value group can represent the degree of similarity between posture changes corresponding to two dates. The posture distinguishing feature information and each piece of historical posture distinguishing feature information in the historical posture distinguishing feature information sequence can be analyzed using the similarity analysis algorithm to generate a posture change similarity value, thereby obtaining the posture change similarity value group.

[0161] In a third step, in response to determining that the posture change similarity value group satisfies a preset feature similarity condition, the person's posture information is updated based on the posture distinguishing feature information to obtain updated person's posture information, and the updated person's posture information is determined as the person's posture information corresponding to the user. The updated person's posture information may be a feature matrix obtained by incrementally updating a feature matrix corresponding to the target posture feature. The feature matrix corresponding to the person's posture information may be incrementally updated based on a matrix corresponding to the posture distinguishing feature information to obtain updated person's posture information, and the updated person's posture information is determined as the person's posture information corresponding to the user.

[0162] Optionally, in response to determining that the posture change similarity value group does not meet the preset feature similarity condition, the execution entity may further update the historical posture distinguishing feature information sequence based on the posture distinguishing feature information. First, the first historical posture distinguishing feature information in the historical posture distinguishing feature information sequence is deleted to obtain a post-deletion historical posture distinguishing feature information sequence. Then, the distinguishing feature information is added to the end of the post-deletion historical posture distinguishing feature information sequence.

[0163] The above-described embodiments of the present disclosure have the following beneficial effects: The door lock control methods of some embodiments of the present disclosure can improve the security of smart door locks. Specifically, the reduced security of smart door locks is caused by the fact that, when at least two highly similar identity information exist, using only user information for information comparison can easily result in the user information being matched to the wrong identity information, thereby reducing the security of the smart door lock. Based on this, the door lock control methods of some embodiments of the present disclosure first obtain personal feature information corresponding to the user in response to detecting that a user has entered a preset recognition range corresponding to the smart door lock. The personal feature information includes body feature information and eye feature information. This information can be used to obtain features of users who may be near the smart door lock with the intent to unlock, facilitating subsequent comparison of the user features to determine the user's intent to unlock. Secondly, blink detection is performed on the eye feature information included in the personal feature information to obtain blink frequency information. In response to determining that the blink frequency information meets a preset frequency condition, the body feature information is detected and processed to obtain body feature information and facial feature information. In this way, based on the number of blinks a user makes in a short period of time, it can be determined that the user has an intention to unlock the device. The user's body features can also be classified to facilitate subsequent determination of whether the user meets the unlocking conditions. The body feature information is then matched against a pre-acquired set of body feature information to obtain first identity matching information. This allows for a preliminary determination of whether the user's body shape matches the pre-acquired body shape, facilitating subsequent determination of whether the user meets the unlocking conditions based on facial and eye features. Next, in response to determining that the first identity matching information meets a first preset identity matching condition, the facial and eye feature information are fused to obtain fused feature information corresponding to the user. The first preset identity matching condition indicates that at least one piece of body shape information in the body feature information set matches the body feature information. Therefore, when the user's body shape matches the pre-acquired body shape, fused features corresponding to the user's facial and eye features can be first determined to facilitate subsequent determination of whether the user meets the unlocking conditions based on the fused features. The fused feature information is then matched against a pre-acquired set of depth feature information to obtain second identity matching information. Thus, it can be determined whether the fused feature matches the pre-acquired depth feature of the person. Finally, in response to determining that the second identity matching information satisfies the second preset identity matching condition, a preset unlocking instruction is sent to the door lock controller corresponding to the smart door lock to control the door lock to unlock. Thus, when the fused feature matches the pre-acquired depth feature of the person, the user meets the unlocking condition, and the door lock can be unlocked via the unlocking instruction.Therefore, the door lock control method of some embodiments of the present disclosure adopts a method that combines coarse feature comparison and fine feature comparison. It can first perform an initial classification of users based on body posture information to screen out multiple identity information that is relatively close to the user's body posture information, and then perform a secondary fine classification of users based on the fusion features of face and eye. This can avoid the mismatch of identity information caused by using only one type of user information to participate in the information comparison. Thus, the smart door lock can be controlled to be unlocked safely, improving the security of the smart door lock. In addition, because continuous blinking in a short period of time is used as a way to confirm the intention to unlock, the number of false unlockings can be reduced in some scenarios (for example, when the user turns his back to the smart door lock).

[0164] Referring to Figure 2, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a door lock control device. These device embodiments correspond to the method embodiments shown in Figure 1. The door lock control device 200 can be specifically applied to various electronic devices.

[0165] As shown in FIG2 , the door lock control device 200 of some embodiments includes: an acquisition unit 201, a first detection unit 202, a second detection unit 203, a first matching processing unit 204, a fusion processing unit 205, a second matching processing unit 206, and a sending unit 207. The acquisition unit 201 is configured to, in response to detecting that a user enters a preset recognition range corresponding to the smart door lock, acquire the personnel feature information corresponding to the above-mentioned user, wherein the above-mentioned personnel feature information includes human body feature information and human eye feature information; the first detection unit 202 is configured to perform blink detection on the human eye feature information included in the above-mentioned personnel feature information to obtain blink frequency information; the second detection unit 203 is configured to, in response to determining that the above-mentioned blink frequency information meets the preset frequency condition, perform detection processing on the above-mentioned human body feature information to obtain body feature information and face feature information; the first matching processing unit 204 is configured to match the above-mentioned body feature information with the pre-acquired personnel body feature information set to obtain first identity matching information; the fusion processing unit 205, is configured to, in response to determining that the above-mentioned first identity matching information meets the first preset identity matching condition, fuse the above-mentioned facial feature information and the above-mentioned human eye feature information to obtain fused feature information corresponding to the above-mentioned user, wherein the above-mentioned first preset identity matching condition indicates that there is at least one person's body information in the above-mentioned person's body information set that matches the above-mentioned body feature information; the second matching processing unit 206, is configured to match the above-mentioned fused feature information with the pre-acquired person's depth feature information set to obtain second identity matching information; the sending unit 207, is configured to, in response to determining that the above-mentioned second identity matching information meets the second preset identity matching condition, send the preset unlocking instruction to the door lock controller corresponding to the above-mentioned smart door lock for controlling the door lock to unlock.

[0166] It is understood that the units described in the door lock control device 200 correspond to the steps in the method described with reference to Figure 1. Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the door lock control device 200 and the units included therein, and will not be repeated here.

[0167] 3, which shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic device shown in FIG3 is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0168] As shown in FIG3 , electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of electronic device 300 are also stored in RAM 303. Processing device 301, ROM 302, and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to bus 304.

[0169] Typically, the following devices can be connected to the I / O interface 305: a production income device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; a production expenditure device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 can allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 shows an electronic device 300 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively. Each box shown in Figure 3 can represent one device, or it can represent multiple devices as needed.

[0170] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0171] It should be noted that in some embodiments of the present disclosure, the computer-readable medium mentioned above may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection having at least one wire, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0172] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0173] The above-mentioned computer-readable medium may be included in the above-mentioned device; or it may exist independently and not be assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to detecting that a user enters the preset recognition range corresponding to the smart door lock, obtains the personnel characteristic information corresponding to the above-mentioned user, wherein the above-mentioned personnel characteristic information includes human body characteristic information and human eye characteristic information; performs blink detection on the human eye characteristic information included in the above-mentioned personnel characteristic information to obtain blink frequency information; in response to determining that the above-mentioned blink frequency information meets the preset frequency condition, performs detection processing on the above-mentioned human body characteristic information to obtain body shape characteristic information and face shape characteristic information; matches the above-mentioned body shape characteristic information with the pre-acquired personnel body shape information set to obtain to the first identity matching information; in response to determining that the above-mentioned first identity matching information meets the first preset identity matching condition, the above-mentioned facial feature information and the above-mentioned human eye feature information are fused to obtain fused feature information corresponding to the above-mentioned user, wherein the above-mentioned first preset identity matching condition represents that there is at least one person's body information in the above-mentioned body information set that matches the above-mentioned body feature information; the above-mentioned fused feature information is matched with the pre-acquired person depth feature information set to obtain second identity matching information; in response to determining that the above-mentioned second identity matching information meets the second preset identity matching condition, the preset unlocking instruction is sent to the door lock controller corresponding to the above-mentioned smart door lock for controlling the door lock to unlock.

[0174] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code includes at least one executable instruction for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0176] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor includes an acquisition unit, a first detection unit, a second detection unit, a first matching processing unit, a fusion processing unit, a second matching processing unit, and a sending unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring personnel feature information corresponding to the above-mentioned user."

[0177] The functions described above herein may be performed, at least in part, by at least one hardware logic component. For example, and without limitation, exemplary types of hardware logic components that may be used include: a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip (SOC), a complex programmable logic device (CPLD), and the like.

[0178] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

[0179] All embodiments of the present disclosure may be implemented individually or in combination with other embodiments, and are all considered to be within the scope of protection claimed by the present disclosure.

Claims

1. A door lock control method, comprising: In response to detecting that a user enters a preset recognition range corresponding to the smart door lock, obtaining personal characteristic information corresponding to the user, wherein the personal characteristic information includes human body characteristic information and human eye characteristic information; Performing blink detection on the human eye feature information included in the personnel feature information to obtain blink frequency information; In response to determining that the blink frequency information meets a preset frequency condition, detecting and processing the human body feature information to obtain body feature information and facial feature information; Matching the body feature information with a pre-acquired body information set of a person to obtain first identity matching information; In response to determining that the first identity matching information satisfies a first preset identity matching condition, fusing the face feature information and the eye feature information to obtain fused feature information corresponding to the user, wherein the first preset identity matching condition indicates that at least one person's body information in the person's body information set matches the body feature information; Matching the fused feature information with a pre-acquired set of personnel deep feature information to obtain second identity matching information; In response to determining that the second identity matching information satisfies a second preset identity matching condition, a preset unlocking instruction is sent to a door lock controller corresponding to the smart door lock for controlling the door lock to unlock.

2. The method according to claim 1, wherein: The method further comprises: In response to determining that the first identity matching information does not satisfy the first preset identity matching condition, matching the eye feature information with a pre-acquired eye information set of a person to obtain third identity matching information; In response to determining that the third identity matching information satisfies a third preset identity matching condition, controlling the associated voice device to play preset password input prompt information; In response to receiving the password key information input by the user, The information is verified and processed to obtain password verification information; In response to determining that the password verification information satisfies a preset verification error condition, the unlocking instruction is sent to the door lock controller for controlling the door lock to unlock.

3. The method according to claim 1 or 2, wherein: The method further comprises: In response to determining that the second identity matching information does not satisfy the second preset identity matching condition, the associated voice device is controlled to play preset safety prompt information.

4. The method according to any one of claims 1 to 3, wherein: The first identity matching information includes: body distinguishing feature information; and The method further comprises: In response to determining that the door lock has been unlocked, acquiring a historical posture distinguishing feature information sequence corresponding to the user; Performing similarity analysis on the posture distinguishing feature information and the historical posture distinguishing feature information sequence to obtain a posture change similarity value group; In response to determining that the posture change similarity value group meets the preset feature similarity condition, the personnel posture information is updated based on the posture distinguishing feature information to obtain updated personnel posture information, and the updated personnel posture information is determined as the personnel posture information corresponding to the user.

5. The method according to claim 4, wherein: The method further comprises: In response to determining that the posture change similarity value group does not meet the preset feature similarity condition, the historical posture distinguishing feature information sequence is updated based on the posture distinguishing feature information.

6. The method according to any one of claims 1 to 5, wherein: The fusing the facial feature information and the eye feature information to obtain fused feature information corresponding to the user includes: Selecting an eye region image that meets a preset eye state condition from each eye region image corresponding to the human eye feature information; Generate a group of human eye images based on the selected eye region images; Segmenting each human eye image in the human eye image group to obtain an iris region image group; Performing interception processing on each iris area image in the iris area image group to obtain a target iris image group; Normalizing the face image corresponding to the face feature information to obtain a target face image; Determine each target iris image and the target face image in the target iris image group as an image to be recognized, to obtain an image group to be recognized; The image group to be identified is input into a pre-trained fusion feature information generation model to generate fusion feature information.

7. A door lock control device, comprising: an acquisition unit, configured to acquire, in response to detecting that a user enters a preset recognition range corresponding to the smart door lock, personal characteristic information corresponding to the user, wherein the personal characteristic information includes human body characteristic information and human eye characteristic information; A first detection unit is configured to perform blink detection on the human eye feature information included in the personnel feature information to obtain blink frequency information; A second detection unit is configured to detect and process the human body feature information in response to determining that the blink frequency information meets a preset frequency condition, so as to obtain body feature information and facial feature information; A first matching processing unit is configured to match the body feature information with a pre-acquired body information set of a person to obtain first identity matching information; a fusion processing unit configured to, in response to determining that the first identity matching information satisfies a first preset identity matching condition, fuse the face feature information and the eye feature information to obtain fused feature information corresponding to the user, wherein the first preset identity matching condition indicates that at least one person's body information in the person's body information set matches the body feature information; A second matching processing unit is configured to match the fused feature information with a pre-acquired person deep feature information set to obtain second identity matching information; The sending unit is configured to send a preset unlocking instruction to the door lock controller corresponding to the smart door lock to control the door lock to unlock in response to determining that the second identity matching information meets the second preset identity matching condition.

8. An electronic device comprising: at least one processor; a storage device having at least one program stored thereon, When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images