Intelligent lock image feature recognition method and system based on deep learning

Abstract

The invention relates to the technical field of intelligent lock recognition, in particular to an intelligent lock image feature recognition method and system based on deep learning. The recognition method comprises the following steps: constructing a multi-branch deep neural network model, wherein the multi-branch deep neural network model comprises a backbone network, a feature extraction branch and a feature fusion branch; a framework training model is used to learn identity recognition, attitude estimation and environment perception tasks; and identifying the comprehensive feature representation by using a trained framework training model, and generating a control instruction of the intelligent lock according to an identification result. According to the intelligent lock image feature recognition method based on deep learning provided by the invention, comprehensive analysis of multi-dimensional information such as identity, posture and environment is realized by constructing the multi-branch deep neural network model.

Description

Technical Field

[0001] The present invention relates to the field of smart lock recognition technology, and specifically to a smart lock image feature recognition method and system based on deep learning. Background Art

[0002] The application of deep learning technology provides a new direction for image feature recognition of smart locks. However, how to effectively extract and utilize multi-scale features of images in complex and changeable actual scenarios while taking into account local details and global semantic information is still a challenging problem. Specifically, it is difficult to effectively balance the importance of local and global features and to simultaneously meet multi-task requirements such as identity recognition, posture estimation, and environmental perception. Especially in applications such as smart locks that have extremely high security requirements, how to achieve comprehensive analysis of multi-dimensional information such as identity, posture, and environment while ensuring recognition accuracy has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a method and system for smart lock image feature recognition based on deep learning to solve the technical problem that the existing smart lock image feature recognition information has few dimensions.

[0004] The technical solution of the present invention to solve the above technical problems is as follows:

[0005] On the one hand, a method for recognizing image features of a smart lock based on deep learning is provided, and the steps of the recognition method include:

[0006] Construct a multi-branch deep neural network model, which includes a backbone network, a feature extraction branch, and a feature fusion branch;

[0007] Collect and preprocess the real-time image of the environment around the smart lock, and input the preprocessed real-time image into the backbone network to extract the multi-scale feature map;

[0008] The multi-scale feature map is processed by the feature extraction branch to generate local features and global features;

[0009] The feature fusion branch fuses local features and global features to generate a comprehensive feature representation;

[0010] Use the framework to train models to learn identity recognition, pose estimation, and environment perception tasks;

[0011] Use the trained framework training model to identify the comprehensive feature representation and generate the control instructions of the smart lock based on the recognition results.

[0012] On the other hand, a smart lock image feature recognition system is provided, the recognition system comprising:

[0013] An image acquisition module, used to acquire images of the environment surrounding the smart lock;

[0014] A preprocessing module, used for preprocessing the collected images;

[0015] Deep neural network module, used to build multi-branch deep neural network models and extract features;

[0016] Feature fusion module, used to fuse local features and global features;

[0017] Multi-task learning module for optimizing multiple tasks;

[0018] A feature recognition module is used to perform feature recognition on real-time images;

[0019] The control instruction generation module is used to generate control instructions for the smart lock.

[0020] The beneficial effects of the present invention are:

[0021] The deep learning-based smart lock image feature recognition method provided by the present invention realizes the comprehensive analysis of multi-dimensional information such as identity, posture and environment by constructing a multi-branch deep neural network model. The method of the present invention adopts an improved ResNeXt structure as the backbone network, introduces void convolution and attention mechanism, and improves the receptive field and discrimination ability of feature extraction. By adding feature extraction branches at different levels, the method can capture multi-scale local information, so as to better adapt to complex and changeable scenes. The introduction of the multi-task learning framework enables the model to simultaneously optimize multiple related tasks such as identity recognition, posture estimation and environmental perception, thereby enhancing the versatility of feature representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a feature recognition method according to an embodiment of the present invention;

[0023] Figure 2 This is a flow chart of a feature extraction branch processing a multi-scale feature map in one embodiment of the present invention;

[0024] Figure 3 is a flow chart of a data enhancement step in one embodiment of the present invention;

[0025] Figure 4 A flowchart of comprehensive feature representation recognition in one embodiment of the present invention;

[0026] Figure 5 A flow chart of control instruction generation in one embodiment of the present invention;

[0027] Figure 6 This is a diagram of the architecture of a system for identifying a target according to an embodiment of the present invention;

[0028] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0029] 100. Image acquisition module; 200. Preprocessing module; 300. Deep neural network module; 400. Feature fusion module; 500. Multi-task learning module; 600. Feature recognition module; 700. Control instruction generation module. DETAILED DESCRIPTION

[0030] The present invention provides the following preferred embodiments:

[0031] Embodiment 1

[0032] In order to solve the multi-task requirements of identity recognition, posture estimation and environmental perception of smart locks in complex environments, this embodiment proposes a smart lock image feature recognition method based on deep learning.

[0033] like Figure 1 As shown, the steps of the identification method include:

[0034] S101. Construct a multi-branch deep neural network model, which includes a backbone network, a feature extraction branch, and a feature fusion branch.

[0035] S102, collecting and preprocessing real-time images of the environment around the smart lock, and inputting the preprocessed real-time images into the backbone network to extract multi-scale feature maps.

[0036] S103: Process the multi-scale feature map through the feature extraction branch to generate local features and global features.

[0037] S104: Fusing local features and global features through a feature fusion branch to generate a comprehensive feature representation.

[0038] S105. Use the framework to train the model and learn identity recognition, posture estimation and environment perception tasks.

[0039] S106. Use the trained framework training model to identify the comprehensive feature representation, and generate control instructions for the smart lock based on the identification results.

[0040] Specifically, this embodiment constructs a multi-branch deep neural network model, which includes a backbone network, a feature extraction branch, and a feature fusion branch. The backbone network adopts an improved ResNeXt structure, and the construction of the backbone network enables the network to have a larger receptive field and stronger feature extraction capabilities. The improvements in the ResNeXt structure include the introduction of hole convolution and attention mechanism modules in the standard ResNeXt block, thereby enhancing the extraction capability and selectivity of spatial features. These improvements enable the backbone network to extract more robust and representative features during the deep learning process.

[0041] Furthermore, in order to ensure the real-time response capability of the smart lock to changes in the surrounding environment, this embodiment uses a high-performance camera device configured with the smart lock to perform real-time image acquisition. The acquired images are preprocessed, including grayscale, normalization, noise suppression and other operations, to ensure that the data input to the neural network is consistent and high quality. Then, the preprocessed real-time image is input into the backbone network to extract multi-scale feature maps. This step comprehensively utilizes the multi-level feature extraction capabilities of the backbone network, so that the model can capture feature information at different scales, including detail features and overall features.

[0042] Furthermore, the design of the feature extraction branch aims to maximize the extraction of local and global features in the image. Feature extraction consists of two branches: local feature extraction branch and global feature extraction branch. The local feature extraction branch focuses on the details of small areas to capture subtle but important features. The global feature extraction branch extracts the overall contour and spatial structure by fusing contextual information. These two types of features are extracted separately through the operations of convolutional layers and pooling layers, and are comprehensively processed on high-level feature maps to form the output of the feature extraction branch.

[0043] It should be understood that local features and global features complement each other in this embodiment. Local features focus on details, which is particularly important for facial feature analysis in identity recognition. Global features focus on the whole and are used for posture estimation and environmental perception. Through the feature fusion branch, this embodiment fuses local features and global features to generate a comprehensive feature representation. The fusion adopts a learnable weighted summation formula so that different types of features can be reasonably weighted in the comprehensive feature representation. During the fusion process, the softmax function is used to ensure that the sum of the weights is 1, thereby realizing the importance distribution and interaction of different features.

[0044] Furthermore, the model is trained using a multi-task learning framework to learn identity recognition, pose estimation, and environmental perception tasks. A multi-task loss function is used during model training, which comprehensively considers the losses of each task and automatically balances the weights of different tasks through the gradient normalization method. During the training process, the feature representation is continuously optimized through back propagation, so that the model can better adapt to the needs of tasks such as identity recognition, pose estimation, and environmental perception. It should be understood that the multi-task learning framework can not only improve the performance of a single task, but also further utilize and optimize the associations between different tasks through shared feature representations.

[0045] Furthermore, the trained model is used to perform feature recognition on the real-time acquired images. First, the real-time acquired images are preprocessed, applying the same preprocessing steps as the training phase to maintain data consistency. Then, the trained multi-branch deep neural network model is used for feature extraction and to generate a comprehensive feature representation. Subsequent recognition steps include comprehensive evaluation of identity recognition, posture estimation, and environmental perception tasks. Identity recognition uses cosine similarity matching, posture estimation is achieved through key point detection and posture optimization, and environmental perception is analyzed using a prediction method based on probability distribution.

[0046] Furthermore, control instructions for the smart lock are generated based on the recognition results. The control logic of the smart lock is formulated based on a predetermined security policy, including identity authentication thresholds, legal range of postures, and environmental safety conditions. A comprehensive evaluation is performed through a fuzzy logic system, and the decision rules include: if the identity match is high, the posture is normal, and the environment is safe, the unlocking operation is performed; if the identity match is low, the posture is abnormal, or the environment is unsafe, the unlocking is refused; if the identity match is medium and other conditions are normal, the user may be required to perform additional verification. The decision result after comprehensive evaluation ultimately generates the corresponding control instructions to ensure the security and reliability of the smart lock.

[0047] Through this embodiment, the problem of how to effectively fuse local detail features and global context information in complex environments in smart lock application scenarios is solved to achieve high-precision, multi-task image feature recognition. The advantages of this embodiment are that the improved ResNeXt structure, dilated convolution and attention mechanism are used to effectively enhance the feature extraction capability; the multi-task recognition performance of the model is ensured through the multi-task learning framework; the feature fusion branch realizes the organic combination of local and global features, thereby providing a stable, reliable and accurate smart lock image feature recognition method.

[0048] Embodiment 2

[0049] In order to solve the problem of multi-scale feature extraction and fusion in smart lock image feature recognition, this embodiment further optimizes the method for constructing a multi-branch deep neural network model. The steps of constructing a multi-branch deep neural network model include:

[0050] The improved ResNeXt structure is used as the basis for building the backbone network structure.

[0051] Feature extraction branches are added at different levels of the backbone network to capture local information of different scales.

[0052] Design a feature fusion branch to integrate global context information.

[0053] Define the output form and fusion method of each branch.

[0054] Specifically, this embodiment uses an improved ResNeXt structure as the basis for constructing the backbone network structure. The ResNeXt structure is a further optimization based on ResNet, introducing the concept of grouped convolution, which can improve the expressiveness of the model while keeping the computational complexity relatively low. In this embodiment, the standard ResNeXt structure is improved to meet the specific needs of smart lock image feature recognition. The improvements include the introduction of hole convolution and attention mechanism modules.

[0055] Furthermore, the introduction of dilated convolution aims to increase the receptive field so that the network can capture a wider range of spatial information without increasing the number of parameters and computational complexity.

[0056] Furthermore, this embodiment adds feature extraction branches at different levels of the backbone network to capture local information of different scales. These feature extraction branches branch out at different depths of the backbone network, and each branch extracts features at a specific scale. The low-level feature extraction branches focus on fine-grained texture and edge information, the middle-level feature extraction branches focus on partial structures of objects, and the high-level feature extraction branches focus on more abstract semantic information.

[0057] It should be understood that feature extraction branches at different levels use different convolution kernel sizes and step sizes to adapt to feature extraction requirements at different scales, in order to obtain a wider range of contextual information.

[0058] Furthermore, this embodiment designs a feature fusion branch for integrating global context information. The feature fusion branch is designed to effectively integrate features of different scales and levels to generate a global feature representation containing rich semantic information. This branch adopts an adaptive feature fusion mechanism, including channel attention and spatial attention modules.

[0059] The channel attention module uses global average pooling and fully connected layers to learn the importance weights of each channel, thereby emphasizing the channel features that are important for the recognition task. The spatial attention module helps the model focus on key areas in the image by learning the importance of different spatial positions. The combination of these two attention mechanisms enables the feature fusion branch to adaptively select and emphasize the most valuable feature information for the current recognition task.

[0060] This embodiment also defines the output form and fusion method of each branch. For the feature extraction branch, the output of each branch is designed to have a feature map with the same spatial dimension, which is achieved through appropriate upsampling or downsampling operations. The output of the feature fusion branch is a high-dimensional feature vector that integrates local features and global context information at all scales.

[0061] It is understandable that this construction method of the multi-branch deep neural network model enables the network to focus on local details and global semantic information at the same time, which is of great significance for multiple tasks such as identity recognition, posture estimation and environmental perception of smart locks. Local features help capture subtle features such as faces and fingerprints, while global features help understand the overall posture and environmental context.

[0062] Through the multi-branch deep neural network model construction method of this embodiment, a comprehensive perception and accurate analysis of the environment around the smart lock is achieved. This method not only improves the efficiency and accuracy of feature extraction, but also enhances the model's ability to capture features of different scales, providing a solid foundation for subsequent multi-task learning and decision-making. This structural design enables the model to better adapt to the complex changes in the application scenarios of smart locks and improves the reliability of the system.

[0063] Embodiment 3

[0064] In order to solve the problems of limited receptive field, insufficient nonlinear expression ability and poor feature selectivity of the standard ResNeXt structure in the smart lock image feature recognition task, this embodiment further optimizes the ResNeXt structure.

[0065] In this embodiment, the improved ResNeXt structure includes the following modifications:

[0066] Introducing dilated convolution into the standard ResNeXt block to increase the receptive field, the calculation formula of the dilated rate r is:

[0067] where layer index is the index of the current layer in the network;

[0068] Use the Swi sh activation function instead of ReLU. The Swi sh function is defined as:

[0069] Where β is a learnable weight parameter;

[0070] An attention mechanism module is added to the last ResNeXt block of each stage, and the compression ratio of the attention mechanism module is 16.

[0071] Specifically, this embodiment introduces a dilated convolution in the standard ResNeXt block to increase the receptive field of the network. The dilated convolution inserts "holes" between the convolution kernel elements, so that the receptive field range is expanded without increasing the number of parameters and computational complexity. In this embodiment, the calculation of the dilated rate r is dynamically adjusted. The method of dynamically adjusting the dilated rate enables the network to focus on features of different scales at different depth layers, thereby improving the diversity of feature extraction.

[0072] It is important to understand that the introduction of dilated convolution not only increases the receptive field, but also maintains the resolution of the feature map. This has a positive effect on detail preservation and multi-scale feature extraction in smart lock image feature recognition. For example, in face recognition tasks, a larger receptive field helps capture the overall structural information of the face, while maintaining high resolution is conducive to extracting facial detail features, such as texture information of key parts such as eyes and nose.

[0073] Furthermore, this embodiment uses the Swi sh activation function instead of the traditional ReLU activation function. The Swi sh function has several advantages over the ReLU: first, it is a smooth, non-monotonic function, which helps the flow of information in the network; second, it has non-zero output in the negative region, which can alleviate the problem of neuron death; in addition, the introduction of the β parameter increases the flexibility of the function, allowing the network to adaptively adjust the shape of the activation function according to task requirements.

[0074] Understandably, these characteristics of the Swish function enable the network to better capture complex nonlinear relationships during training. In the application scenario of smart locks, this means that the network can more accurately model nonlinear factors such as changes in ambient lighting and facial expressions, thereby improving recognition accuracy. The learning process of the β parameter uses a back-propagation algorithm and is optimized synchronously with other network parameters.

[0075] Furthermore, in this embodiment, an attention mechanism module is added to the last ResNeXt block of each stage. The introduction of the attention mechanism module is intended to enhance the network's ability to select important features, so that the model can adaptively focus on the feature channels and spatial regions that are most important to the current task. The attention mechanism module used in this embodiment includes two submodules: channel attention and spatial attention, and its compression ratio is set to 16. It should be explained that the channel attention submodule first compresses the spatial information of each feature channel into a numerical value through a global average pooling operation, and then learns the importance weight of each channel through two fully connected layers. The first fully connected layer compresses the number of channels to 1 / 16 of the original, and the second fully connected layer restores it to the original number of channels. This "compression-activation-restore" design not only reduces the number of parameters, but also retains sufficient nonlinear expression capabilities. The spatial attention submodule focuses on learning the importance of different spatial positions in the feature map. It first performs maximum pooling and average pooling along the channel dimension, then splices the two obtained 2D feature maps, and generates spatial attention weights through a 7x7 convolutional layer. This enables the network to identify and emphasize key areas in the image, such as eyes, nose and other features in face recognition.

[0076] It is important to understand that the combination of channel attention and spatial attention enables the network to selectively enhance both feature channels and spatial dimensions, which is important for processing complex visual information in smart lock scenarios. For example, in environmental perception tasks, certain specific channels may correspond to lighting information, while certain spatial regions may contain key environmental cues, and the attention mechanism can help the network better utilize this information.

[0077] Through the improvement of the ResNeXt structure in this embodiment, multi-scale and efficient extraction of smart lock image features is achieved. The introduction of dilated convolution expands the receptive field, allowing the network to capture a wider range of contextual information; the Swish activation function enhances the network's nonlinear modeling capabilities and helps process complex visual patterns; the attention mechanism module improves the accuracy of feature selection, allowing the network to better adapt to different recognition tasks.

[0078] Embodiment 4

[0079] In order to solve the problem of multi-target detection and feature extraction in smart lock image feature recognition, this embodiment further optimizes the processing flow of the feature extraction branch.

[0080] like Figure 2 As shown, the steps of processing the multi-scale feature map through the feature extraction branch include:

[0081] S401. Generate candidate regions using a region proposal network. The anchor scale of the region proposal network is [32, 64, 128] and the aspect ratio is [0.5, 1, 2].

[0082] S402: Apply RoIA lig operation to each candidate region to extract a feature map of size 7x7.

[0083] S403. Input the extracted feature map into three parallel sub-networks, each sub-network contains two convolutional layers with a convolution kernel size of 3x3 and a step size of 1 and a fully connected layer, which are used for extracting facial features, gesture features and environmental object features respectively.

[0084] Specifically, this embodiment uses a region proposal network RPN to generate candidate regions. RPN is an efficient target detection method that can slide a window on a feature map to generate candidate regions of multiple scales and aspect ratios for each location. In this embodiment, the anchor scale of RPN is set to [32, 64, 128] pixels, and the aspect ratio is set to [0.5, 1, 2]. This setting takes into account the various target sizes and shapes that may appear in the smart lock scenario, such as faces at different distances, various gestures, and objects in the environment.

[0085] It is important to understand that the design of the anchor directly affects the detection effect. The 32-pixel anchor is mainly used to detect smaller targets, such as distant faces or small objects; the 64-pixel anchor is suitable for medium-sized targets; and the 128-pixel anchor is used to detect larger targets, such as nearby faces or large environmental objects. The aspect ratio setting takes into account the different shapes of the targets. An aspect ratio of 0.5 is suitable for targets extending vertically, an aspect ratio of 2 is suitable for targets extending horizontally, and an aspect ratio of 1 is suitable for targets that are approximately square.

[0086] Furthermore, a Ro IA lig operation is applied to each candidate region generated by RPN to extract a feature map of size 7x7. Ro IA lig is an improvement over traditional Ro IPoo lig, which avoids quantization operations by bilinear interpolation, thereby retaining more spatial information. The output size of 7x7 is an empirical value that strikes a balance between retaining sufficient details and controlling computational complexity.

[0087] It is understandable that the use of RoIa ligne operations affects the accuracy of feature extraction. In the application scenario of smart locks, accurate spatial information is important for identifying facial details, gestures, and the shapes of environmental objects. For example, in face recognition, accurate spatial information can help the network better capture the positional relationship of facial feature points; in gesture recognition, it can help the network understand the relative position and movement of fingers more accurately.

[0088] Furthermore, this embodiment inputs the extracted 7x7 feature map into three parallel sub-networks for extracting facial features, gesture features, and environmental object features, respectively. Each sub-network has the same structure, including two convolutional layers and one fully connected layer. Specifically, the convolution kernel size of the two convolutional layers is 3x3, and the step size is 1. This setting can increase the depth of the network while maintaining the size of the receptive field, thereby improving the ability of feature extraction.

[0089] Furthermore, the number of input channels of the first convolutional layer is the number of feature channels output by the backbone network, and the number of output channels is set to 256. The second convolutional layer maintains 256 output channels. Each convolutional layer is followed by batch normalization and ReLU activation function to speed up the training process and increase nonlinear expression capabilities. The output dimension of the fully connected layer is set according to the specific task requirements, usually 128 or 256 dimensions.

[0090] It is important to understand that the design of the three parallel sub-networks allows the network to specialize in extracting different types of features. The face feature sub-network may focus more on the structure and texture information of the face; the gesture feature sub-network may focus more on capturing the shape and posture of the hand; and the environmental object feature sub-network may focus more on extracting the contour and contextual information of the object.

[0091] Furthermore, this embodiment also introduces an attention mechanism in each sub-network. Specifically, a channel attention mechanism module is added between two convolutional layers, which is implemented using an SE block. The SE block first compresses the spatial dimension through global average pooling, then uses two fully connected layers to learn the correlation between channels, and finally generates the weight of each channel through a sigmoid function. This design enables the network to adaptively adjust the importance of different channels, thereby improving the efficiency and effectiveness of feature extraction.

[0092] Furthermore, in order to enhance the generalization ability of the sub-network, the present embodiment can use data enhancement technology during the training process. For the face feature sub-network, random horizontal flipping, small angle rotation and brightness contrast adjustment are used; for the gesture feature sub-network, random cropping, rotation and scale transformation are used; for the environmental object feature sub-network, random cropping, color jitter and Gaussian blur are used. These data enhancement technologies help improve the model's adaptability to various practical scenarios.

[0093] Through the feature extraction branch processing flow of this embodiment, efficient detection and feature extraction of multiple targets in the smart lock scene are achieved. The use of RPN improves the efficiency and accuracy of candidate region generation; RoIA lig operation ensures the spatial accuracy of features; the design of three parallel sub-networks realizes specialized extraction of different types of features. It not only improves the quality and efficiency of feature extraction, but also enhances the model's ability to handle complex scenes, providing support for subsequent recognition and decision-making tasks.

[0094] Embodiment 5

[0095] In order to solve the problem of effective fusion of local features and global features in smart lock image feature recognition, this embodiment further optimizes the processing flow of the feature fusion branch. The specific implementation method is as follows:

[0096] This embodiment adopts a weighted summation method to fuse local features and global features. The fusion formula is:

[0097] Among them, F i represents the i-th local feature, F g represents the global feature, α iand β are the corresponding weight parameters, and N is the number of local features. This weighted summation method allows the network to dynamically adjust the contribution of different features according to their importance, thereby achieving more flexible and effective feature fusion.

[0098] It is important to understand that the local feature F i Usually from different levels or different spatial positions of the feature extraction branch, they may contain information of different scales or different semantic levels. For example, in the face recognition task, low-level local features may contain detailed information such as facial texture, while high-level local features may contain high-level semantic information such as facial structure. The global feature Fg is usually generated by the last layer of the network, which contains the contextual information of the entire image.

[0099] Furthermore, in order to ensure the rationality of the weight parameters, this embodiment imposes constraints on the weight parameters: This constraint ensures that the sum of the weights of all features is 1, avoiding the situation where some features are over-emphasized or ignored. To achieve this constraint, this embodiment uses the softmax function to generate weight parameters. Specifically, the network first generates N+1 unnormalized weight values ​​wi (i=1, 2, ..., N+1), and then converts them into weight parameters that meet the constraint through the softmax function:

[0100]

[0101] This softmax normalization method not only ensures that the sum of the weights is 1, but also introduces nonlinear relationships, enhancing the expressiveness of the model.

[0102] It is understandable that the learning process of weight parameters is automatically completed through the back propagation algorithm. During the training process, the network will continuously adjust the unnormalized weight w according to the gradient information of the loss function. i , thereby optimizing α i This adaptive weight learning mechanism enables the network to dynamically adjust the importance of local and global features according to the characteristics of different input images.

[0103] Furthermore, in order to enhance the effect of feature fusion, this embodiment introduces a feature transformation module before weighted summation. i and the global feature F g First, a 1x1 convolutional layer is used to adjust the number of channels of all features to make them consistent, which helps in the subsequent feature fusion. Then, batch normalization and ReLU activation function are applied to the adjusted features to enhance the expressiveness of the features and the nonlinearity of the network.

[0104] In order to further improve the effect of feature fusion, this embodiment also adopts a multi-scale feature fusion strategy. Specifically, before calculating the final fusion feature, the features of different scales are upsampled or downsampled so that they have the same spatial resolution. This multi-scale feature fusion strategy can better utilize information of different scales and improve the model's ability to recognize objects of different sizes.

[0105] Through the feature fusion method of this embodiment, the effective integration of local features and global features is achieved. The weighted summation method allows the network to dynamically adjust the importance of different features according to task requirements; the use of the softmax function ensures the rationality of the weight parameters; the introduction of the feature transformation module and the attention mechanism further enhances the expressiveness of the features.

[0106] Embodiment 6

[0107] In order to solve the problem of multi-task learning in smart lock image feature recognition, especially the joint optimization problem of identity recognition, posture estimation and environment perception tasks, this embodiment further optimizes the loss function design of the model and the task weight balancing strategy. The specific implementation is as follows:

[0108] This embodiment adopts a multi-task learning framework and designs a comprehensive loss function to simultaneously optimize the three tasks of identity recognition, posture estimation, and environment perception. The overall loss function is defined as:

[0109] L total =w1·L id +w2·L pose +w3·L env +λ·L reg , this loss function contains three task-related loss terms and a regularization term, which balances the contribution of each part through weight coefficients w1, w2, w3 and λ.

[0110] Furthermore, for the identity recognition task, this embodiment uses the cross entropy loss L id , the cross entropy loss is defined as:

[0111] where C is the total number of identity categories, y i is the true label, usually one-hot encoded, p i is the predicted probability of the model for the i-th class. The cross entropy loss can effectively measure the difference between the predicted distribution and the true distribution, and is a commonly used loss function in classification problems.

[0112] It should be understood that in the application scenario of smart locks, the identity recognition task may face the problem of class imbalance, that is, the number of samples of some identities may be much larger than that of other identities. In order to alleviate this problem, this embodiment introduces class weights based on the cross entropy loss. The modified loss function can be expressed as:

[0113] where w i is the weight of the i-th class, which can be set to the inverse of the number of samples in that class, or use the dynamic weight strategy in the focal loss.

[0114] Furthermore, for the posture estimation task, this embodiment adopts the smooth L1 loss L pose The smooth L1 loss is defined as:

[0115] Where K is the number of keypoints, pi is the predicted keypoint location, and gi is the actual keypoint location. L1 The function behaves as L2 loss when the input is close to zero, and behaves as L1 loss in other cases. This design takes into account the advantages of L1 loss being insensitive to outliers and L2 loss having smooth gradients near zero.

[0116] Furthermore, for the environmental perception task, this embodiment uses the loss function L env The loss function L env is defined as:

[0117] Where M is the number of environment categories, α i is the class weight, γ is the focusing parameter, and p i is the model’s predicted probability for the i-th class, y i is the true label. The loss function effectively solves the problem of class imbalance by reducing the weight of easy-to-classify samples and focusing on difficult-to-classify samples.

[0118] Furthermore, this embodiment adds an L2 regularization term L to the total loss function. reg , which is defined as:

[0119] Where Θ is the set of model parameters. L2 regularization can prevent the model from overfitting and improve the generalization ability of the model. The regularization coefficient λ controls the strength of regularization, and its value is usually determined by cross-validation.

[0120] Furthermore, in order to automatically balance the contributions of different tasks, this embodiment adopts a weight update method based on gradient normalization. Specifically, the weight calculation formula is:

[0121] Where si is the gradient norm of the loss of the ith task, and T is the temperature parameter. This method can dynamically adjust the weights according to the relative size of the gradients of each task, thereby adaptively balancing the learning of different tasks during training. It should be understood that the temperature parameter T plays a role in controlling the smoothness of the weight distribution in this weight calculation formula. Larger T values ​​will make the weight distribution more uniform, while smaller T values ​​will make the weight distribution more inclined to tasks with larger gradients. In practical applications, the value of T can be adjusted according to the needs of specific tasks.

[0122] Through the multi-task loss function design and dynamic weight adjustment strategy of this embodiment, effective joint optimization of identity recognition, posture estimation and environmental perception tasks is achieved. The combination of cross entropy loss, smooth L1 loss and loss function takes into account the characteristics and difficulties of different tasks; the introduction of L2 regularization improves the generalization ability of the model. This design not only improves the effect of multi-task learning, but also enhances the model's adaptability to complex scenes, providing a more comprehensive solution for the image feature recognition task of smart locks.

[0123] Embodiment 7

[0124] In order to solve the problem of insufficient model generalization ability in smart lock image feature recognition, especially the problem of reduced recognition performance when facing complex and changeable actual application scenarios, this embodiment designs a comprehensive data enhancement method, including random erasing, image mixing, contrast enhancement and color adjustment techniques, to improve the model's adaptability to various lighting conditions, occlusion situations and environmental changes.

[0125] like Figure 3 As shown, the process of using the framework to train the model and learn identity recognition, pose estimation, and environment perception tasks includes data enhancement steps:

[0126] S701, randomly erasing a rectangular area from the image with probability p=0.5, the area of ​​the erased area is randomly selected between 0.02 and 0.2 of the original image, the aspect ratio is randomly selected between 0.3 and 3, and random noise is used for filling.

[0127] S702, randomly select image A and image B, randomly select the area of ​​the cropped area between 0.3 and 1 of the original image, mix the cropped area of ​​image A with the corresponding area of ​​image B in a ratio of λ:(1-λ), and λ is randomly selected between 0.2 and 0.8.

[0128] S703 , perform CLAHE processing on the Y channel of the YUV space of the image, with a grid size of 8x8 and a contrast limit threshold of 3.

[0129] S704, randomly adjusting the hue, saturation and brightness in the HSV space, with an adjustment range of ±10%.

[0130] Specifically, this embodiment adopts random erasing technology. A random rectangular area is erased on the input image with a probability of 0.5. The area of ​​the erased area is randomly selected between 2% and 20% of the original image area, and the aspect ratio is randomly selected between 0.3 and 3. This setting takes into account possible occlusions of different sizes and shapes. The erased area is filled with random noise instead of a simple fixed color filling, which helps the model learn the stability of local information loss.

[0131] It is important to understand that the random erasing technique simulates partial occlusions that may occur in real-life scenarios, such as a face being blocked by a mask, a gesture being blocked by an object, etc. By introducing this artificial "occlusion" during the training process, the model can be prompted to learn to use other features in the image for recognition, thereby improving the model's performance in the face of actual occlusions.

[0132] Furthermore, this embodiment introduces an image mixing technique. Two images A and B are randomly selected, and a region is randomly cropped from each image, where the area of ​​the cropped region is randomly selected between 30% and 100% of the original image area. Then, the cropped region of image A is mixed with the corresponding region of image B at a ratio of λ:(1-λ), where λ is randomly selected between 0.2 and 0.8. This mixing strategy not only increases the diversity of training data, but also helps the model learn more robust feature representations.

[0133] It is understandable that the image blending technology creates a "virtual" scene that may not exist in reality, but contains the features of multiple real scenes. This technology can help the model learn more abstract and generalized feature representations, improving the performance of the model when dealing with unseen complex scenes.

[0134] Furthermore, this embodiment also adopts contrast limited adaptive histogram equalization CLAHE technology. Specifically, CLAHE processing is performed on the Y channel of the YUV space of the image, the grid size is set to 8x8, and the contrast limit threshold is set to 3. CLAHE technology can effectively enhance the local contrast of the image while avoiding the noise amplification problem caused by excessive enhancement.

[0135] It is important to understand that CLAHE technology is particularly suitable for smart lock application scenarios. In actual use, smart locks may face various complex lighting conditions, such as strong light, weak light, backlight, etc. CLAHE technology can enhance the local details of the image without excessively amplifying the noise, so that the model can better extract useful feature information.

[0136] Furthermore, this embodiment performs random color adjustment on the image in the HSV color space. Specifically, the hue, saturation, and brightness are randomly adjusted within a range of ±10%. This color adjustment strategy can simulate different lighting conditions and color changes, and enhance the robustness of the model to color changes.

[0137] Furthermore, this embodiment can also introduce additional data enhancement techniques. For example, random rotation of ±10 degrees and scaling of 0.9 to 1.1 times can be used to simulate different shooting angles and distances; random Gaussian noise and salt and pepper noise can be added to simulate image inputs of different qualities; random blurring can be used to simulate blurring effects caused by camera shake or target motion. These additional enhancement techniques further increase the diversity of training data and help improve the generalization ability of the model.

[0138] Through the comprehensive data enhancement strategy of this embodiment, a high degree of diversity in training data is achieved, effectively simulating various complex situations that smart locks may encounter in actual applications. Random erasing technology improves the model's ability to recognize local occlusions; image blending technology enhances the model's ability to learn abstract features; CLAHE processing improves the model's performance under different lighting conditions; and color space adjustment enhances the model's ability to adapt to color changes.

[0139] Embodiment 8

[0140] In order to solve the accuracy and efficiency problems of smart lock image feature recognition in practical applications, this embodiment further optimizes the recognition process and refines the implementation methods of each recognition step. This embodiment designs a complete recognition process, including image preprocessing, feature extraction, identity recognition, posture estimation, and environmental perception steps to achieve comprehensive analysis and recognition of the input image.

[0141] like Figure 4 As shown, the steps of using the trained framework training model to identify the comprehensive feature representation include:

[0142] S801, image preprocessing, applying the same preprocessing steps as the training stage.

[0143] S802: Feature extraction: use the trained multi-branch deep neural network model to extract comprehensive feature representation.

[0144] S803, identity recognition, using the nearest neighbor classifier for identity matching, using cosine similarity as the distance metric.

[0145] S804, posture estimation, outputting key point coordinates through regression network, and using RANSAC algorithm for posture optimization.

[0146] S805, environmental perception, using the softmax function to output the probability distribution of environmental categories.

[0147] Specifically, image preprocessing is performed first. This step applies the same preprocessing operations as the training phase to ensure that the test data has consistent distribution characteristics with the training data. The preprocessing steps include image scaling, cropping, normalization, etc. Specifically, the input image is scaled to a fixed size, such as 224x224 pixels, center cropped, and then the pixel values ​​are normalized to the range of [-1,1]. These preprocessing operations help reduce the impact of image size and brightness changes on the recognition results.

[0148] It is important to understand that consistent preprocessing steps are fundamental to ensuring model performance. Since deep learning models are very sensitive to the distribution of input data, if the preprocessing during the testing phase is inconsistent with the training phase, it may lead to a significant drop in model performance.

[0149] Furthermore, a trained multi-branch deep neural network model is used to extract comprehensive feature representation. The network model adopts a multi-scale feature fusion strategy, which can simultaneously capture local details and global semantic information of the image. Specifically, the backbone of the network uses ResNet-50 as the basic architecture, and sets feature extraction branches at different levels. The outputs of these branches are integrated through an adaptive feature fusion module to finally generate a high-dimensional comprehensive feature vector.

[0150] It is understandable that the multi-branch structure and feature fusion strategy enable the model to focus on features of different scales and abstract levels at the same time, which is very effective for handling complex recognition tasks. For example, in identity recognition, low-level features may contain useful texture information, while high-level features may capture more abstract identity features.

[0151] For the identity recognition task, this embodiment uses the nearest neighbor classifier for identity matching and uses cosine similarity as the distance metric. Specifically, the extracted feature vector is compared with the pre-stored identity feature database to find the matching item with the highest cosine similarity.

[0152] Furthermore, in the posture estimation task, the coordinates of the key points are output through the regression network. In order to improve the stability and accuracy of the posture estimation, this embodiment introduces the random consistency sampling RANSAC algorithm for posture optimization. The RANSAC algorithm can effectively remove the influence of outliers. In the specific implementation, the regression network is first used to predict the coordinates of the key points, and then the RANSAC algorithm is applied to iteratively optimize the posture parameters, and finally a more accurate posture estimation result is obtained.

[0153] Furthermore, for the environmental perception task, this embodiment uses the softmax function to output the probability distribution of environmental categories. On this basis, in order to improve the reliability of environmental perception, this embodiment also introduces a confidence threshold and a multi-frame decision mechanism. Specifically, the recognition result is considered valid only when the probability of a certain category exceeds the preset threshold; otherwise, it is marked as an "unknown" category. In addition, by voting on the recognition results of multiple consecutive frames, the stability of environmental perception can be further improved.

[0154] Through the recognition process design of this embodiment, comprehensive analysis and efficient recognition of input images are achieved. Consistent preprocessing ensures the consistency of data distribution; multi-branch deep neural network extracts rich comprehensive features; the nearest neighbor classifier combined with LSH technology achieves efficient identity matching; the RANSAC algorithm improves the accuracy of posture estimation; the softmax function with confidence threshold and multi-frame decision mechanism enhances the reliability of environmental perception.

[0155] Embodiment 9

[0156] In order to solve the decision-making problem of smart locks in complex environments, especially how to weigh and generate appropriate control instructions when faced with multiple recognition results, this embodiment further optimizes the decision-making mechanism and refines the security strategy and control instruction generation method.

[0157] like Figure 5 As shown, the steps of generating a control instruction of the smart lock according to the recognition result include:

[0158] S901. Define a security policy rule set, including authentication thresholds, posture legal ranges, and environmental security conditions.

[0159] S902, using a fuzzy logic system to comprehensively evaluate the recognition results, the fuzzy rules include:

[0160] S9021. If the identity match is high, the posture is normal, and the environment is safe, then unlock.

[0161] S9022. If the identity match is low, the posture is abnormal, or the environment is unsafe, the door will be refused to be unlocked.

[0162] S9023. If the identity match is moderate and other conditions are normal, additional verification is required.

[0163] S903. Generate corresponding control instructions according to the output of the fuzzy logic system.

[0164] This embodiment designs a comprehensive decision-making system based on fuzzy logic, including the definition of security policy rule sets, the construction of fuzzy logic systems, and the generation of control instructions. The system can effectively handle the uncertainty in the recognition results and provide more flexible and intelligent access control.

[0165] Specifically, this embodiment defines a comprehensive security policy rule set. The rule set includes multiple dimensions such as identity authentication threshold, legal range of posture, and environmental safety conditions. Specifically, the identity authentication threshold can be set to a range value of cosine similarity, such as 0.8-1.0 for high matching, 0.6-0.8 for medium matching, and less than 0.6 for low matching. The legal range of posture can be defined as the angle limit of the head and body, for example, the deflection angle of the head does not exceed 30 degrees, and the tilt angle of the body does not exceed 15 degrees. Environmental safety conditions may include factors such as light intensity, background complexity, and whether there are suspicious objects.

[0166] It is important to understand that these security policy rules are not simply fixed thresholds, but are dynamically adjusted according to specific application scenarios and security requirements. For example, in high-security venues, the authentication threshold can be increased to narrow the legal range of gestures; while in ordinary home environments, these restrictions can be appropriately relaxed.

[0167] Furthermore, this embodiment constructs a decision-making system based on fuzzy logic. The system takes the recognition result as input and generates the final control decision through three steps: fuzzification, reasoning and defuzzification. The design of fuzzy rules takes into account the combination of multiple factors, including not only the basic rules such as "unlock if the identity match is high, the posture is normal and the environment is safe", but also more complex situation processing.

[0168] Understandably, the strength of a fuzzy logic system is its ability to handle uncertainty and ambiguity in input data. For example, the concept of "high identity match" can be a continuous membership function in fuzzy logic rather than a simple yes / no judgment. This design enables the system to better handle edge cases and complex scenarios.

[0169] Furthermore, the fuzzy rules of this embodiment also include some more detailed judgment logic. For example, "if the identity match is medium, but the user is a frequent visitor and the current time is within the preset access time period, additional voice verification is required." This rule can provide more convenient access control while ensuring security.

[0170] Furthermore, this embodiment introduces an adaptive learning mechanism to optimize fuzzy rules. The system records the results of each decision and user feedback, and adjusts the parameters of the fuzzy rules through machine learning algorithms (such as reinforcement learning). This adaptive mechanism enables the system to continuously optimize its decision-making ability over time and adapt to the usage habits and security needs of different users.

[0171] Furthermore, when generating control instructions, this embodiment not only considers the two basic instructions of unlocking and refusing to unlock, but also designs a series of intermediate states and additional operations. For example, when the system determines that additional verification is required, it can generate an instruction to "activate secondary authentication", which may include operations such as sending a mobile phone verification code, requiring fingerprint recognition, or starting a face recognition camera. In addition, the system can also generate alarm instructions based on the recognition results, such as triggering a silent alarm when suspicious behavior is detected.

[0172] Through the decision-making mechanism design of this embodiment, intelligent generation of smart lock control instructions is achieved. A comprehensive set of security policy rules provides a basis for decision-making; the fuzzy logic system enables flexible processing of complex scenarios; the adaptive learning mechanism enables continuous optimization of the system; and the diversified control instructions and feedback mechanisms further improve the intelligence of the system.

[0173] Embodiment 10

[0174] In order to solve the problem of image feature recognition of smart locks in complex environments and improve the accuracy, real-time and reliability of recognition, this embodiment further optimizes the overall architecture of the smart lock image feature recognition system and refines the implementation method of each functional module.

[0175] like Figure 6 As shown, the smart lock image feature recognition system includes:

[0176] An image acquisition module 100 is used to acquire images of the surrounding environment of the smart lock;

[0177] A preprocessing module 200, used for preprocessing the collected images;

[0178] A deep neural network module 300 is used to construct a multi-branch deep neural network model and extract features;

[0179] A feature fusion module 400 is used to fuse local features and global features;

[0180] A multi-task learning module 500, for optimizing multiple tasks;

[0181] A feature recognition module 600 is used to perform feature recognition on real-time images;

[0182] The control instruction generation module 700 is used to generate control instructions for the smart lock.

[0183] Specifically, this embodiment designs a modular smart lock image feature recognition system, including an image acquisition module 100, a preprocessing module 200, a deep neural network module 300, a feature fusion module 400, a multi-task learning module 500, a feature recognition module 600 and a control instruction generation module 700. The modules are closely connected through data flow and control flow to form an end-to-end recognition system.

[0184] Furthermore, the image acquisition module 100 uses a high-performance CMOS image sensor with a 1 / 2.8-inch optical format and an effective pixel of 1920x1080. The sensor supports wide dynamic range WDR technology with a dynamic range of up to 120dB, which can effectively cope with complex lighting environments such as backlighting. At the same time, the minimum illumination of the sensor can reach 0.01l ux, ensuring that a clear image can still be obtained under low light conditions. The module is also equipped with an autofocus lens with a focal length range of 2.8-12mm, which can cover shooting needs at different distances.

[0185] It is important to understand that high-quality image acquisition is the basis for subsequent recognition. The design of this module not only takes into account image quality, but also power consumption and cost factors, making it suitable for integration into embedded devices such as smart locks.

[0186] Further, the preprocessing module 200 receives the raw data from the image acquisition module 100 and performs a series of processing to improve image quality and standardization. First, a denoising algorithm based on deep learning, such as DnCNN, is applied to effectively remove image noise while retaining detail information. Then, the image contrast is enhanced using the adaptive histogram equalization CLAHE technology, with the parameters set to a block size of 8x8 and a contrast limit threshold of 3.0. Next, an image standardization operation is performed, including resizing and pixel value normalization.

[0187] It is understandable that the design of the preprocessing step needs to strike a balance between computational complexity and effectiveness. The processing flow of this module has been optimized to run in real time on an embedded platform with processing latency controlled within 20 milliseconds.

[0188] Furthermore, the deep neural network module 300 constructs a multi-branch feature extraction network. The backbone network adopts the ResNet-50 architecture, but is optimized in a targeted manner: some convolutional layers are replaced with depthwise separable convolutions to reduce the number of parameters while maintaining model performance; feature extraction branches are set at different levels, and each branch contains a combination of 1x1 convolution, 3x3 convolution, and 1x1 convolution, forming a "bottleneck" structure to effectively extract multi-scale features.

[0189] Furthermore, the feature fusion module 400 adopts an adaptive feature fusion strategy to effectively integrate features from different branches. In specific implementation, a 1x1 convolutional layer is used to learn the weights of each feature, and then the fused feature is obtained by weighted summation. In order to capture the long-range dependency between features, non-local operations are introduced, in which the similarity calculation uses an embedded Gaussian function and the aggregation operation uses weighted summation. This design can effectively fuse local details and global semantic information.

[0190] Furthermore, the multi-task learning module 500 adopts a hard parameter sharing strategy, where the underlying feature extraction network is shared among the three tasks of identity recognition, posture estimation, and environmental perception, while the high-level network is specialized for different tasks. In order to balance the learning process of each task, a dynamic weight adjustment mechanism based on uncertainty is introduced. Specifically, the loss weight of each task is modeled as a learnable parameter and automatically adjusted through back propagation. In addition, a curriculum learning strategy is implemented to gradually introduce different tasks in increasing order of task difficulty to improve the overall learning effect.

[0191] Furthermore, the feature recognition module 600 is responsible for feature recognition of real-time images. For the identity recognition task, the cosine similarity is used to measure the distance between feature vectors, and the K nearest neighbor classifier is combined for identity matching. The posture estimation task outputs the coordinates of 17 key points of the human body through the regression network, and uses the RANSAC algorithm for posture optimization. The number of iterations is set to 100 and the internal point threshold is 2 pixels. The environmental perception task outputs the probability distribution of 10 predefined environmental categories through the softmax function. In order to improve the real-time performance of recognition, the model quantization technology is applied to convert floating-point operations into 8-bit fixed-point operations, which significantly improves the inference speed while ensuring accuracy.

[0192] Furthermore, the control instruction generation module 700 implements a decision-making system based on fuzzy logic. The input variables include identity matching, posture legitimacy and environmental safety, and each variable defines 3-5 fuzzy sets. The reasoning process adopts the Mamdan i model, uses the minimum-maximum synthesis method for fuzzy reasoning, and finally defuzzifies through the centroid method. The module also includes an adaptive learning mechanism, which continuously optimizes the parameters of the decision rules through a reinforcement learning algorithm. In addition, a multi-level security strategy is implemented, and the decision threshold can be dynamically adjusted according to contextual information such as time and location.

[0193] Through the system design of this embodiment, full-process intelligent processing from image acquisition to control instruction generation is realized. High-quality image acquisition and effective preprocessing provide reliable input for subsequent recognition; multi-branch deep neural network and adaptive feature fusion extract rich image features; multi-task learning framework realizes simultaneous optimization of multiple tasks; feature recognition module 600 combines multiple algorithms to achieve efficient and accurate recognition; and control instruction generation module 700 based on fuzzy logic provides a flexible solution for decision-making in complex scenarios.

[0194] The above embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying image features of a smart lock based on deep learning, characterized in that: The steps of the identification method include: Constructing a multi-branch deep neural network model, wherein the multi-branch deep neural network model includes a backbone network, a feature extraction branch, and a feature fusion branch; Collect and preprocess the real-time image of the environment around the smart lock, and input the preprocessed real-time image into the backbone network to extract the multi-scale feature map; Processing the multi-scale feature map through the feature extraction branch to generate local features and global features; The local features and the global features are fused by the feature fusion branch to generate a comprehensive feature representation; Use the framework to train models to learn identity recognition, pose estimation, and environment perception tasks; The trained framework training model is used to identify the comprehensive feature representation, and the control instructions of the smart lock are generated according to the identification results.

2. The method for identifying smart lock image features based on deep learning as claimed in claim 1, characterized in that: The steps of constructing a multi-branch deep neural network model include: The structure of the backbone network is constructed based on the improved ResNeXt structure; Adding the feature extraction branches at different levels of the backbone network to capture local information of different scales; Design a feature fusion branch to integrate global context information; The output form and fusion method of each of the feature extraction branches and the feature fusion branch are defined.

3. The method for identifying smart lock image features based on deep learning as claimed in claim 2, characterized in that: The improved ResNeXt structure includes the following modifications: Introducing dilated convolution into the standard ResNeXt block to increase the receptive field, the calculation formula of the dilated rate r is: Where layer index is the index of the current layer in the network; Use the Swish activation function instead of ReLU. The Swish function is defined as: Where β is a learnable weight parameter; An attention mechanism module is added to the last ResNeXt block in each stage, and the compression ratio of the attention mechanism module is 16.

4. The method for identifying smart lock image features based on deep learning as claimed in claim 1, characterized in that: The step of processing the multi-scale feature map by the feature extraction branch comprises: Generate candidate regions using a region proposal network, where the anchor scale of the region proposal network is [32, 64, 128] and the aspect ratio is [0.5, 1, 2]; Apply the RoIAlign operation to each candidate region and extract a 7x7 feature map; The extracted feature map is input into three parallel sub-networks, each of which contains two convolutional layers with a convolution kernel size of 3x3 and a step size of 1 and a fully connected layer, which are used to extract facial features, gesture features, and environmental object features respectively.

5. The method for identifying smart lock image features based on deep learning as claimed in claim 1, characterized in that: The process of fusing the local features and the global features through the feature fusion branch adopts the following weighted summation formula: Among them, F i is the i-th local feature, F g is the global feature, α i and β are learnable weight parameters and satisfy N is the number of local features.

6. The method for identifying smart lock image features based on deep learning as claimed in claim 1, characterized in that: The framework is used to train the model and learn the following loss functions for identity recognition, pose estimation, and environment perception tasks: L total =w1·L id +w2·L pose +w3·L env +λ·L reg ; Among them, L id is the cross entropy loss for identity recognition: C is the number of identity categories, y i is the true label, pi is the predicted probability; L pose is the smooth L1 loss for pose estimation: K is the number of key points, pi is the predicted position, and gi is the actual position; L env The loss function for environment perception is: M is the number of environmental categories, α i is the category weight, γ is the focusing parameter; L reg is the L2 regularization term: Θ is the set of model parameters; w1, w2, w3 and λ are task weights, which are automatically balanced by gradient normalization: where s i is the gradient norm of each task loss, s j Represents the jth element in the input value set, and T is the temperature parameter.

7. The method for identifying image features of a smart lock based on deep learning as claimed in claim 1, characterized in that: The process of using the framework to train models to learn identity recognition, pose estimation, and environment perception tasks includes the following data augmentation steps: Randomly erase a rectangular area from the image with probability p = 0.

5. The area of ​​the erased area is randomly selected between 0.02 and 0.2 of the original image, the aspect ratio is randomly selected between 0.3 and 3, and filled with random noise; Randomly select image A and image B, randomly select the cropped area between 0.3 and 1 of the original image, mix the cropped area of ​​image A with the corresponding area of ​​image B in a ratio of λ:(1-λ), and λ is randomly selected between 0.2 and 0.8; Perform CLAHE processing on the Y channel of the YUV space of the image, with a grid size of 8x8 and a contrast limit threshold of 3; In the HSV space, the hue, saturation and brightness are randomly adjusted with an adjustment range of ±10%.

8. The method for identifying smart lock image features based on deep learning as claimed in claim 1, characterized in that: The step of using the trained framework training model to identify the comprehensive feature representation includes: Apply the same preprocessing steps as in the training phase; Use the trained multi-branch deep neural network model to extract comprehensive feature representation; The nearest neighbor classifier is used for identity matching, and cosine similarity is used as the distance metric; Output key point coordinates through regression network and use RANSAC algorithm for posture optimization; Use the softmax function to output the probability distribution of environment categories.

9. The method for identifying smart lock image features based on deep learning as claimed in claim 1, characterized in that: The step of generating a control instruction for the smart lock according to the recognition result comprises: Define security policy rule sets, including authentication thresholds, posture legal ranges, and environmental security conditions; A fuzzy logic system is used to comprehensively evaluate the recognition results. The fuzzy rules include: If the identity matches well, the posture is normal, and the environment is safe, the lock will be unlocked; If the identity match is low, the posture is abnormal, or the environment is unsafe, the lock will be refused; If the identity match is moderate and other conditions are normal, additional verification is required; According to the output of the fuzzy logic system, the corresponding control instructions are generated.

10. A smart lock image feature recognition system, characterized in that: The identification system comprises: An image acquisition module, used to acquire images of the environment surrounding the smart lock; A preprocessing module, used for preprocessing the collected images; Deep neural network module, used to build multi-branch deep neural network models and extract features; Feature fusion module, used to fuse local features and global features; Multi-task learning module for optimizing multiple tasks; A feature recognition module is used to perform feature recognition on real-time images; The control instruction generation module is used to generate control instructions for the smart lock.

Images