Door control unlocking method based on plantar pressure

Abstract

The invention discloses an entrance guard unlocking method based on plantar pressure, and belongs to the technical field of identity recognition. The method comprises the following steps: firstly, acquiring pressure data of each point of a foot sole in a walking process of a user in real time to form a dynamic pressure distribution diagram; then, an image processing module carries out image preprocessing on the obtained dynamic pressure distribution diagram and sorts the dynamic pressure distribution diagram according to a time sequence; the MCTBA-Net neural network model converts the processed dynamic pressure distribution map into gait features; the obtained gait features are compared with gait feature data sets of N users pre-stored in a storage unit for verification, and a verification result is obtained; and the execution unit judges whether to unlock according to the verification result. According to the invention, the identification accuracy is higher, the anti-interference capability is stronger, and a safe and reliable technical scheme is provided for intelligent access control management and personalized identity authentication.

Description

Technical Field

[0001] The present invention belongs to the technical field of identity recognition, and more specifically, relates to a door unlocking method based on plantar pressure. Background Art

[0002] With the rapid development of smart homes, the demand for smart locks in terms of security and convenience is becoming increasingly prominent. Currently, common identity recognition technologies include fingerprint recognition, facial recognition, and iris recognition, but these technologies still have shortcomings in environmental adaptability, privacy protection, and anti-counterfeiting capabilities. In recent years, identity recognition technologies based on gait and plantar pressure have gradually attracted attention and have been applied to access control systems and identity verification. Among them, gait recognition has great application potential in identity verification due to its non-contact nature and high difficulty in counterfeiting. However, since gait recognition currently relies on taking photos, its recognition accuracy is easily affected by factors such as lighting and hand position. Plantar pressure data itself has strong time series characteristics and is easily affected by factors such as footwear, weight, and gait changes, making accurate and efficient modeling and processing of it a technical difficulty.

[0003] Existing research has attempted to use machine learning and deep learning methods to identify plantar pressure data, such as plantar pressure image recognition methods based on SVM+CNN or time series feature modeling based on LSTM, to improve the accuracy of gait phase recognition. However, these methods still have shortcomings in feature extraction, such as spatiotemporal correlation modeling, data augmentation, and model generalization capabilities. This results in limited recognition accuracy and affects the reliability and practicality of smart lock systems in practical applications.

[0004] A search revealed Chinese patent application number 201610167596.3, published on August 10, 2016, which discloses an access control device, system, and method for identifying individuals using gait information. The control method comprises the following steps: 1. capturing a video stream containing gait information within a set area; 2. extracting gait features of individuals from the captured video stream and transmitting them to a cloud server; 3. upon receiving a touch signal, generating a trigger instruction and wirelessly transmitting it to a handheld device; 4. upon receiving the trigger instruction, obtaining the gait features from the cloud server and comparing and analyzing them with all pre-stored gait features of individuals in a gait database to determine the similarity between the gait features and the gait features of each individual in the gait database; 5. comparing the obtained similarity with a preset threshold; and 6. generating a corresponding access control action based on the comparison result. However, gait-based video recognition methods require external video cameras and are sensitive to ambient lighting and occlusion, which can affect recognition stability.

[0005] The patent document with Chinese patent application number 201810416943.0 and publication date of September 28, 2018 discloses a plantar pressure image recognition method based on SVM+CNN. The control method is performed in the following steps: 1. Read the plantar pressure image and perform filtering to construct the plantar pressure image dataset; 2. Construct a convolutional neural network (CNN) and bring the data into it for training, and save the trained model; 3. Bring the data into the trained model to obtain the feature vector extracted by the CNN model; 4. Extract the features of the plantar pressure image with clear feature significance; 5. Fuse the features extracted in step_3 and step_4 respectively, and normalize to obtain the feature dataset of the plantar pressure image, and make training set and test set according to the ratio of 80% and 20%; 6. Send the feature data of the training set to SVM for training to obtain the SVM model; 7. Send the feature data of the test set to the trained SVM model for prediction to obtain the prediction result. However, the plantar pressure image recognition method based on SVM+CNN can extract plantar pressure features, but has limited adaptability to shoe differences, weight changes and other factors, making it difficult to ensure the robustness of recognition.

[0006] The patent document with Chinese patent application number 202410117453.6 and publication date of May 03, 2024 discloses a cross-gait phase recognition method based on multi-source perception fusion. The recognition method is performed in the following steps: 1. Plantar pressure data and foot movement data acquisition; 2. Data preprocessing; 3. According to the double-foot plantar pressure cloud map, the double-foot gait is divided into 5 phases; 4. Plantar pressure data and foot movement data feature extraction; 5. Feature data fusion and PCA dimension reduction; 6. LSTM classifies and recognizes the feature data. However, the cross-gait phase recognition method based on multi-source perception fusion mainly focuses on gait phase division, and there is still room for improvement in modeling the spatiotemporal features of plantar pressure, resulting in insufficient identity matching accuracy.

[0007] The gait recognition, plantar pressure image recognition and identity recognition based on gait phase in the above-mentioned prior art can achieve identity verification, but still have certain limitations in actual application. The recognition accuracy and generalization ability are still limited in complex environments, and false recognition or rejection may occur in actual use, affecting the security and user experience of the intelligent lock system.

[0008] Therefore, how to more accurately model the spatiotemporal features of plantar pressure data and improve the adaptability to individual gait differences has become an important research direction for improving the recognition accuracy and convenience of intelligent locks. SUMMARY

[0009] 1. Problems to be solved

[0010] The application provides a door access unlocking method based on plantar pressure, which converts the plantar pressure into gait for identity recognition to achieve the purpose of door access unlocking.

[0011] 2. Technical solutions

[0012] To solve the above problems, the application adopts the following technical solutions.

[0013] A door access unlocking method based on plantar pressure, which adopts a door access system for unlocking, the door access system comprising a collection unit, a data processing unit, a storage unit and an execution unit, the data processing unit comprising an image processing module and an MCTBA-Net neural network model, the unlocking method comprising the following steps:

[0014] S1, the user steps in place multiple times at the collection unit, and the collection unit collects plantar pressure data of each point in real time and draws a dynamic pressure distribution map in real time;

[0015] S2, the image processing module performs image preprocessing on the dynamic pressure distribution map obtained in step S1 and sorts them in time sequence;

[0016] S3, a multi-scale spatiotemporal feature fusion double-channel attention (MCTBA-Net) neural network model is established, which integrates and optimizes a multi-scale convolution (Multi-scale CNN) module, a time series convolution (TCN) module, a bidirectional long short-term memory (BiLSTM) module and a multi-head attention mechanism (MHA) module, to extract discriminative features in spatial and temporal dimensions from the dynamic pressure distribution map processed in step S2 to convert them into gait features; the MCTBA-Net neural network model can efficiently extract and strengthen the spatial and temporal discriminative features in the plantar pressure image, and realize high-precision classification and discrimination of the user's identity;

[0017] S4, compare and verify the gait features obtained in step S3 with the pre-stored gait feature dataset of N users in the storage unit to obtain a verification result;

[0018] S5, the execution unit determines whether to unlock according to the verification result of step S4: the system automatically sends a control instruction to unlock or keep the lock, and performs a complete closed-loop identity verification and door lock control.

[0019] In a possible implementation of the application, in step S3, the method for converting the dynamic pressure distribution map into gait features by the MCTBA-Net neural network model is as follows:

[0020] S31, the multi-scale convolution (Multi-scale CNN) module is combined with the ResNet-50 network model through an ablation experiment, and an optimized multi-scale convolution (Multi-scale CNN) module is obtained, which is used for high-precision extraction of different foot bottom area multi-scale textures (heel, forefoot, arch) and pressure distribution in the dynamic pressure distribution map processed in step S2;

[0021] S32, in the time convolution (TCN) module, one-dimensional convolution (all kernel_size=3, dilation=1, padding=1) with two layers of weight_norm normalization is used, and residual connection is arranged, the input and output channels are both latent_dim, the parameter combination is verified through repeated ablation experiments, and an optimized time convolution (TCN) module is obtained, which is used to model the image processed in step S31, and the inter-frame dependence is preliminarily established; the short-range dynamic change between adjacent frames can be captured for training;

[0022] S33, in the bidirectional long short-term memory (BiLSTM) module, a bidirectional LSTM (hidden_size=H) with lstm_layers layers is adopted, the input dimension is latent_dim, the output dimension is 2*H, and the forward and backward information is fused through residual connection, the number of layers and the hidden_size parameter are determined after grid search and convergence speed evaluation, and an optimized bidirectional long short-term memory (BiLSTM) module is obtained, which is used to train the image processed in step S32 again to realize efficient and stable long-range dependence learning;

[0023] S34, in the multi-head attention mechanism (Multi-Head Attention, referred to as MHA) module, the fixed 4-head configuration is optimized, and Tanh activation and time average pooling are added after attention output, an optimized MHA module is obtained, which is used for dynamic focusing of key frames of the image trained in step S33, and realizes conversion of foot pressure data into gait features of the user.

[0024] As a possible embodiment of the application, in step S31, the multi-scale convolution (Multi-scale CNN) module is truncated and reconstructed on the basis of the original backbone of the ResNet-50 network model, the 256, 512 and 2048 channel features of the first, second and third stages are extracted in parallel, and adaptive average pooling and full connection mapping are introduced after each branch, which breaks the limitation of the original network single path, realizes more fine multi-scale spatial feature fusion, realizes high-precision extraction of different foot bottom area multi-scale textures and pressure distribution, and each channel number and convolution kernel configuration is optimized through strict ablation experiment.

[0025] As a possible implementation manner of the present application, in step S1, the acquisition unit is an array pressure sensor, and the array pressure sensor is arranged by a plurality of sensing nodes.

[0026] As a possible implementation manner of the present application, in step S1, the dynamic pressure distribution map includes at least one complete step cycle.

[0027] As a possible implementation manner of the present application, in step S2, the image processing module reads the first N frames of the dynamic pressure distribution map obtained in step S1 as a standard sample for image preprocessing, wherein N is at least 7 frames; the image processing module represents each frame of the pressure distribution map as a CxHxW format, and stacks the continuous N frames intercepted in the time sequence of acquisition into a high-dimensional tensor with a shape of TxCxHxW, so as to completely retain the spatial and time sequence dynamic characteristics of the gait.

[0028] As a possible implementation manner of the present application, in step S2, the image preprocessing includes image noise filtering, image size unification, spatial enhancement and pixel normalization, wherein the spatial enhancement includes random cropping and horizontal flipping.

[0029] As a possible implementation manner of the present application, in step S4, the establishment process of the pre-stored gait feature data set of N users in the storage unit is as follows:

[0030] (1) Under at least three different weight scenarios, the user steps in place on the acquisition unit multiple times, the acquisition unit real-time acquires the pressure data of each point of the foot bottom, and real-time draws a dynamic pressure distribution map; wherein the three different weight scenarios are three weights of 0, a and b, and 0

[0031] (2) The image processing module performs image preprocessing on the dynamic pressure distribution maps obtained in step (1) for each scenario, and sorts them in time sequence;

[0032] (3) The MCTBA-Net neural network model converts the dynamic pressure distribution maps processed in step (2) into gait features, and uploads them to the storage unit as the gait feature data set of the user.

[0033] As a possible implementation manner of the present application, in step S5, whether to unlock is determined according to the verification result output by the MCTBA-Net neural network model in S4.

[0034] S51, when the verification result indicates that the user identity matches, the execution unit executes the unlocking instruction to drive the lock core to unlock;

[0035] S52, when the verification result indicates that the user identity does not match, the execution unit keeps the intelligent lock core in a locked state.

[0036] As a possible implementation of the application, it also includes: S6, updating the gait features that pass the verification to the gait feature dataset of the corresponding user in the storage unit in real time.

[0037] Through the above technical solution, the application not only realizes fine parameter optimization and collaborative cooperation of each sub-module, but also significantly improves the recognition accuracy under cross-shoe type, cross-load and multi-environment conditions, which can reach 100%, and the system robustness and safety are also greatly enhanced.

[0038] 3、Beneficial effects

[0039] The multi-component space-time fusion optimization network (MCTBA-Net) identity recognition method based on plantar pressure distribution has the following significant beneficial effects in view of the low recognition accuracy, poor robustness and insufficient safety of the prior art in complex environments:

[0040] Firstly, the spatial features of the forefoot, midfoot and heel regions of the plantar pressure image are extracted in parallel through the multi-scale convolution module, and the local and global structural information in the plantar pressure image is fully retained; secondly, the time series convolution network (TCN) is introduced to model the causal convolution and residual of multiple frames of pressure data, effectively filtering out high-frequency noise and capturing short-term dynamic changes, and improving the system's ability to resist environmental interference; then, the bidirectional long short-term memory network (BiLSTM) is used to model the forward and backward time series of gait data, fully mining the long-range dependence features in the complete gait cycle; finally, the multi-head self-attention mechanism (MHA) is integrated, and the attention weight is dynamically allocated according to the inter-frame correlation, highlighting the discriminative key frames such as heel landing and swing peak, and further enhancing the discriminability and generalization ability of the model.

[0041] In addition, through pressure normalization and gait correction processing, the dependence on absolute pressure values can be effectively reduced, and the adaptability of the model in complex scenes such as cross-shoe type and load change can be improved; at the same time, the system writes the result into the storage unit after completing the identity matching, dynamically updates the identity feature library, and facilitates subsequent data management and security audit. The overall system realizes end-to-end learning, has the advantages of high recognition accuracy, strong anti-interference ability, natural user experience, etc., and is suitable for various practical application scenarios such as multi-user access control and personalized identity authentication.

[0042] The access control unlocking method based on plantar pressure of the application converts the plantar pressure distribution in the user's walking process into gait features through analysis, and has good privacy protection characteristics. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The physical map of the array type pressure sensor of the acquisition unit in the access control unlocking method based on plantar pressure of the application is shown in the figure;

[0044] Figure 2 Flow chart of the foot pressure-based access control unlocking method of the present application;

[0045] Figure 3 Foot pressure distribution map in time sequence for the foot pressure-based access control unlocking method of the present application;

[0046] Figure 4 Structural schematic diagram of the access control system in the foot pressure-based access control unlocking method of the present application;

[0047] Figure 5 Structural schematic diagram of the MCTBA-Net neural network model in the foot pressure-based access control unlocking method of the present application;

[0048] Figure 6 Schematic diagram of 20-person identity recognition results of the foot pressure-based access control unlocking method of the present application. DETAILED DESCRIPTION

[0049] The existing foot pressure intelligent lock identity recognition technology faces multiple challenges in actual application: first, environmental factors (such as light, temperature and vibration) are easy to introduce noise, significantly weakening signal quality and leading to a decline in recognition performance; second, differences in shoe sole material and structure, changes in user load state and individual gait differences can cause instability in foot pressure distribution, further reducing the robustness of the algorithm in multiple scenarios; in addition, existing solutions also have defects in privacy protection and anti-fake ability, making it difficult to balance user data security and system attack resistance needs. In view of the above bottlenecks, the present application proposes the MCTBA-Net method. Through pressure distribution normalization and gait adaptive correction preprocessing, and deep improvement and optimization of multi-scale spatial convolution, temporal convolution network (TCN), bidirectional long short-term memory network (BiLSTM) and multi-head self-attention (MHA) mechanism and fusion, end-to-end spatiotemporal feature modeling of foot pressure data is realized. In the spatial domain, MCTBA-Net parallelly intercepts multi-scale features of three major regions of forefoot, midfoot and heel, accurately capturing the texture and pressure distribution of each part; in the time domain, the TCN module effectively filters out high-frequency noise and extracts short-term dynamics, BiLSTM learns forward and backward long-range dependencies in parallel, ensuring complete representation of the full-cycle gait signal; at the critical moment, MHA dynamically allocates weights according to inter-frame correlation to highlight discriminative features such as heel landing and swing peak. Experimental results show that MCTBA-Net can maintain high recognition rate under cross-shoe type and different load states, not only greatly improving the security and convenience of the intelligent lock system, but also taking into account user privacy protection and anti-fake needs.

[0050] Specifically, the application provides a door access unlocking method based on plantar pressure, which collects plantar pressure data of a user, and uses a multi-component space-time feature fusion optimization network (Multi-scale Convolutional Temporal BiLSTM Attention Network, MCTBA-Net) proposed by the application for end-to-end modeling and inference, realizes deep joint extraction of spatial and time sequence features of the plantar pressure distribution image, and significantly improves the accuracy and robustness of identity recognition. The application can achieve 100% recognition accuracy in an experimental environment, and supports real-time updating of data storage unit information after each unlocking, so that the system can dynamically adapt to the daily changes of the user's plantar pressure distribution and continuously optimize the recognition performance.

[0051] In the specific implementation process of the application, first, the array pressure sensor in the acquisition unit acquires the plantar dynamic pressure map when the user walks, and normalizes the original pressure map to suppress environmental noise such as light, temperature and vibration and reduce the influence of individual differences on the recognition result; then, the image processing module in the data processing unit inputs the normalized time sequence tensor into the multi-scale convolution module, the time sequence convolution network (TCN), the bidirectional long short-term memory network (BiLSTM) and the multi-head self-attention mechanism (MHA) in turn, wherein the multi-scale convolution extracts the spatial features of the forefoot, midfoot and heel region in parallel, the TCN filters out high-frequency noise and captures short-term dynamics through causal dilated convolution and residual structure, the BiLSTM models the forward and backward long-range dependencies in parallel to fuse complete gait cycle information, and the MHA dynamically weights the key space-time information according to the inter-frame correlation and reduces the dependence on absolute pressure values, thereby efficiently extracting discriminative space-time features; finally, the control unit executes unlocking or keeps locking according to the identity matching result output by the model, and writes the recognition result and the corresponding plantar pressure feature information into the storage unit to continuously update the user identity feature library, improve the system recognition accuracy and facilitate safety supervision.

[0052] The intelligent lock system of the application analyzes the user's gait and plantar pressure features based on the above method through the deep learning model MCTBA-Net, realizes non-contact, high concealment and difficult to counterfeit identity verification, and can be widely used in the fields of intelligent access control and safety control. The specific implementation mode of the application will be further described below in combination with specific implementation cases.

[0053] I. In the registration phase, the system guides the indoor members to stand on the like Figure 1The user stands on the array pressure sensor in front of the door, and performs the in-place stepping operation. The array pressure sensor detects the pressure changes of different areas of the foot in real time, forms the original data stream, and forms the corresponding foot pressure distribution map. Because the sensor array contains a large number of sensing nodes, spatial resolution pressure distribution measurement can be achieved, fully reflecting the detailed features of each area of the foot, and improving the discrimination ability of the subsequent identity recognition algorithm. The collected multi-scene data is preprocessed, transmitted through a high-speed bus, and stored in a dedicated information library as the registration feature set of the member.

[0054] II. In the use stage, as shown in the figure, Figure 2 The foot pressure-based access control unlocking method of the embodiment has the following steps:

[0055] S1, the user stands on the array pressure sensor arranged with multiple resistance type pressure sensing nodes in front of the door, and performs the in-place stepping operation. The array pressure sensor detects the pressure changes of different areas of the foot in real time, and obtains the foot dynamic pressure distribution image generated by the user in the gait process, as shown in the figure, to form a continuous original foot pressure data stream. The data contains complete spatial distribution information and time sequence change characteristics, and is the basic data source for identity recognition analysis. Figure 3

[0056] S2, referring to the figure, Figure 4 The image processing module in the data processing unit is used for structured conversion of the foot pressure data stream obtained in step S1, so as to adapt to the input format of the subsequent deep learning model. Specifically, the image processing module in the data processing unit first continuously reads the original foot pressure image frame from the cache area of the array pressure sensor. In order to ensure the consistency and integrity of the gait sample, the system in this embodiment defaults to intercept the first N frames of the current sequence as the standard input sample, where N≥7, so as to cover at least one complete stepping period; if the number of frames in the cache area is less than N frames, then it is filled by cycle. Then, each frame of pressure image is encoded in C×H×W format (C is the number of channels, H is the height, and W is the width), and stacked into a four-dimensional tensor (T=N) in time order, so as to completely retain the change characteristics of the foot pressure in space and time dimensions, and provide a standardized and unified input data structure for the subsequent image preprocessing and feature extraction module.

[0057] In this embodiment, the image processing module in the data processing unit further performs image preprocessing and data enhancement operations on the original foot pressure image tensor after it is constructed, so as to improve the generalization ability and environmental robustness of the subsequent deep learning model to diversified input data. The processing procedure includes the following four sub-steps.

[0058] S21, image noise filtering (enhance signal stability) ​

[0059] The image processing module in the data processing unit first performs noise suppression processing on each frame of image x(t) e R C×H×W Specifically including:

[0060] 1. Gaussian filtering

[0061]

[0062] Where: "*" represents convolution operation, G σ (i,j) is the weight of the Gaussian kernel at the offset (i,j), σ is the standard deviation, which controls the filtering strength, and i and j are the row and column offsets relative to the kernel center.

[0063] This operation can smooth random noise, preserve the overall trend of the plantar pressure distribution, and improve the stability of subsequent feature extraction.

[0064] 2. Median filtering

[0065]

[0066] Where: is the neighborhood centered at pixel (u,v) with size k x k; k is an odd window size.

[0067] By suppressing extreme value points, impulse noise is effectively filtered out, ensuring the reliability of local details of the plantar pressure image.

[0068] After completing the suppression of image noise and the preservation of edge details, to ensure that all samples have uniform spatial dimensions when inputting the model, the image processing module in the data processing unit performs size normalization operation on each frame of image.

[0069] S22, image size unification (standardized input dimension)

[0070]

[0071] Where: the height and width of the input and output are adjusted to 224 to match the input layer size of the deep learning model, and Resize(·) can be implemented based on bilinear interpolation or nearest neighbor interpolation.

[0072] Uniform scaling to 224 x 224 pixels to match the fixed size of the input layer of the deep learning model, avoiding feature offset and dimension mismatch errors caused by inconsistent sizes.

[0073] Subsequently, to improve the robustness of the model to user posture disturbances, shoe type differences and other factors in actual scenarios, the system further performs spatial enhancement operation on the images with standardized size.

[0074] S23, spatial enhancement (improve model robustness)

[0075] The graph processing module in the data processing unit performs a spatial disturbance operation on the uniformly sized image to simulate different gait scenes in real use, thereby enhancing the discrimination ability and environmental adaptability of the model. The spatial disturbance operation includes:

[0076] Random clipping: a sub-region Ω is randomly selected in the range of [0, 224]x[0, 224] and clipped as a new input image;

[0077] Horizontal flipping: the image is subjected to left-right mirror transformation, and the corresponding pixel coordinates satisfy:

[0078]

[0079] By introducing spatial disturbance and mirror asymmetry, the robustness of the model to factors such as posture change, left-right difference, and stride offset is improved.

[0080] After completing the spatial disturbance processing, in order to improve the numerical stability and gradient update efficiency in the training process of the deep learning model, the system performs pixel normalization on the enhanced image.

[0081] S24, pixel normalization (improve numerical stability)

[0082]

[0083] wherein: represents the enhanced t-th frame of the plantar pressure image tensor, C is the number of channels, 224 is the uniform height and width, represents the minimum value of all pixel values in the image tensor, represents the maximum value of all pixel values in the image tensor, is the normalized image tensor, and all pixel values satisfy

[0084] The normalization process has the following technical effects: eliminating the dimensional differences between different input samples, improving the stability of gradient calculation, preventing gradient explosion or disappearance in the training process, accelerating the training convergence process of the neural network, and ensuring that the model has strong consistent response under different sample distributions.

[0085] At this point, all time series frame images have completed standardization processing, and a high-dimensional tensor with uniform size, enhanced characteristics, and numerical stability is finally constructed:

[0086]

[0087] The tensor completely retains the expression characteristics of the plantar pressure data in the time and space dimensions, and serves as the standard input for the subsequent identity recognition deep model.

[0088] S3、After completing the image preprocessing described in step S2, the resulting plantar pressure image sequence is organized as a five-dimensional tensor,

[0089]

[0090] wherein, B = 16 represents the batch size, T = N represents the length of the time frame, C = 3, H = 224, W = 224 represent the number of image channels, height and width respectively. The tensor is input into the MCTBA-Net model in the data processing unit (see FIG. 2), sequentially passing through the multi-scale feature extraction, time series modeling, attention focusing and classification modules, to complete user identity recognition. Figure 5

[0091] S31、The Multi-scale CNN model is improved and optimized based on the ResNet-50 network model combined with ablation experiments, and an optimized Multi-scale CNN model is obtained, which is used to capture the rich local texture (such as plantar pressure distribution) and macroscopic morphology (such as foot contour, walking posture) in gait images. Multi-scale CNN can capture these information at the same time, and improve the sensitivity to the small differences of different individuals.

[0092] The image sequence is first input into the multi-scale convolution front end Multi-Scale CNN module to extract multi-level spatial features of the gait image. Specifically, the module is constructed based on the optimized ResNet-50 framework, and the output results of layer1, layer2 and layer3 are intercepted respectively to obtain: shallow features (channel number about 256), reflecting local texture and edge morphology, middle features (channel number about 512), representing structure contour and regional distribution, and deep features (channel number about 2048), extracting abstract global information.

[0093] The above three types of features are compressed to 1x1 after adaptive average pooling, concatenated in the channel dimension to form a fusion vector (total dimension 2816). Then it is compressed to the specified latent dimension latent_dim = 256 through a linear mapping layer, and the output is a 256-dimensional spatial feature for each frame:

[0094]

[0095] wherein: z (t) ∈R 256 : represents the compressed feature vector of the t-th image, Linear(2816→256): represents a fully connected linear layer, which compresses the concatenated multi-scale spatial features (channel number 2816) to a unified dimension of 256 to match the input dimension of the subsequent time series modeling module.

[0096] The final feature sequence tensor is z∈R​B×T×256

[0097] The module can extract multi-scale spatial feature information from the plantar pressure image, retain local details such as pressure peak distribution, capture global contour shape, and improve the perception ability of the network to individual gait differences of users.

[0098] In the TCN module, one-dimensional convolution with two layers of weight_norm normalization (both kernel_size=3, dilation=1, padding=1) is used, and residual connection is arranged, and the input and output channels are both latent_dim. The parameter combination is verified through repeated ablation experiments, and an optimized TCN module is obtained. The processed image in step S31 is used for short-term local modeling in time sequence, which can quickly extract local motion patterns (such as short-term dynamics of heel landing to toe lifting), and lay a foundation for subsequent long-time dependent modeling.

[0099] The feature sequence Z is input to the optimized time convolution module TCN_Block for modeling short-range dynamic features between adjacent frames. The module includes two one-dimensional convolution operations, both using kernel_size=3, dilation=1, padding=1 settings, and adding Weight_Norm normalization and ReLU activation function after each convolution, while integrating residual connection to keep the input and output dimensions consistent:

[0100]

[0101] Where: Z∈R B×T×256 Multi-scale feature sequence, Two one-dimensional convolution operations, using WeightNorm weight normalization to enhance training stability, ReLU(·) activation function to introduce nonlinearity, "+" for residual connection to keep the input and output dimensions consistent and enhance gradient propagation; Z' represents the output sequence after short-term dynamic modeling.

[0102] The module can efficiently capture short-term transition patterns between consecutive frames (such as heel landing-arch support-toe lifting), improve the modeling ability of the model for local gait dynamics, and has parallel computing advantages to accelerate model training convergence.

[0103] In step S33, a bidirectional LSTM (hidden_size = H) with lstm_layers layers is used in the BiLSTM module, with an input dimension of latent_dim and an output dimension of 2 × H. Forward and backward information are fused via residual connections. The number of layers and the hidden_size parameter are determined after grid search and convergence rate evaluation. The images processed in step S32 are then retrained to achieve efficient and stable long-range dependency learning. (A complete walking cycle often requires multiple frames to describe. BiLSTM can integrate the dynamic changes of the entire cycle, improving the ability to distinguish individual gait rhythms and posture sequences.)

[0104] The TCN output feature sequence Z′ is then input into the BiLSTM module to capture the long-term dependency features throughout the gait cycle. In this example, a two-layer bidirectional LSTM is used with H hidden units and an output dimension of 2H per layer. By concatenating the forward and backward states, the sequence output is formed:

[0105] H (t) =BiLSTM(Z′ (t) ),H∈R B×T×2H

[0106] Where: Z′ (y) : the feature vector of the t-th frame after TCN processing; Bilstm(·): bidirectional LSTM module, extracting sequence features from forward and backward directions respectively; H (t) : The hidden state after fusing the previous and next context information at this moment; H: The complete sequence output tensor, which contains the temporal features of all frames, and the output dimension is 2H, that is, the concatenation of the forward and backward hidden states.

[0107] The forward LSTM extracts dynamic information from the past to the present, while the backward LSTM extracts supplementary information from the future to the present. After fusion, they achieve a global representation of the entire time series. Gait recognition relies on complete gait cycle information. The BiLSTM module integrates historical and future state models to model long-range dynamic features across time steps, improving the model's ability to capture individual rhythms and movement patterns.

[0108] S34, the MHA module is optimized to a fixed 4-head configuration on the native multi-head self-attention framework, and a Tanh activation and time average pooling are appended after the attention output to obtain an optimized MHA module, which is used to calculate the correlation of any two frames of features in the calculation sequence of the image trained in step S33, assign dynamic weights to each frame, strengthen the influence of key frames (such as the moment of maximum plantar pressure), suppress redundant or noisy frames, and adaptively mine the most discriminative moment features, thereby further improving the recognition accuracy and robustness. (Not every frame in the gait sequence is equally important, and self-attention helps the network to adaptively mine the most discriminative moment features, thereby further improving the recognition accuracy and robustness)

[0109] To further enhance the contribution of the key gait segment to the recognition result, the model introduces a multi-head self-attention mechanism (MHA). In this embodiment, the number of heads is set to 4, and the embedding dimension is 2H. This module generates a time series weighted representation by calculating the similarity between any two frames of features. After the attention output is activated by Tanh, it is averaged and pooled in the time dimension:

[0110] g = MeanPool(tanh(MHA(H))), g ∈ R B×2H

[0111] Where: MHA(H): multi-head self-attention mechanism for weighted representation of time series, MHA structure can adaptively assign key frame weights; tanh(·): hyperbolic tangent function as an activation operation, used to compress the attention output range and improve the nonlinear representation ability; MeanPool(·): average pooling on the time dimension to obtain a global gait aggregation vector for each sample; g: the final discriminative feature vector of each sample, with a dimension of 2H.

[0112] The self-attention module can adaptively enhance the influence of key frames (such as extreme pressure moments) on the overall recognition result and suppress the interference of redundant frames, thereby improving the discrimination accuracy and model robustness.

[0113] S35, the aggregated feature vector g is input to the final fully connected layer for identity classification prediction:

[0114]

[0115] Where: The predicted label is used to compare with the identity library to determine whether it matches; n is the total number of preset identity categories (indoor registered users). The system determines whether it matches the user identity information library according to the output result, and outputs "Y" to indicate identity matching and "N" to indicate mismatch.

[0116] The step extracts multi-dimensional space-time features in the plantar image by the multi-module combination of the MCTBA-Net, has advantages of structure optimization, controllable parameters, rich feature expression, and significantly improves the accuracy and real-time performance of the identity recognition.

[0117] S4, the pre-processed plantar pressure image sequence is input into the MCTBA-Net deep learning model to obtain real-time gait features of the user, compared with the pre-stored gait feature dataset of N users in the storage unit, a verification result is obtained, and the MCTBA-Net deep learning model directly outputs the identity recognition result Y or N of the user, which is used to determine whether the current user is a legal registered user. "Y" means matching or "N" means not matching.

[0118] The recognition result is directly transmitted to the execution unit of the door lock system, such as Figure 4 The execution unit automatically completes the corresponding operation according to the result: if the recognition result is "matching", the execution unit drives the door lock mechanism to complete the unlocking operation; if the recognition result is "not matching", the execution unit maintains the closed state of the door lock and does not unlock. This mechanism establishes an efficient and direct response path between recognition and door lock control, and realizes fast and safe identity verification and access control.

[0119] In order to improve the recognition accuracy and long-term adaptability of the system, the recognition result and related pressure data are transmitted to the storage unit for recording and updating. Specifically, the following two aspects are included:

[0120] On the one hand, for the legal user who successfully recognizes, the current plantar pressure image sequence, recognition time, and wearing state (such as shoe sole material change or whether to carry weight) are automatically stored in the historical sample set of the user. With the gradual enrichment of data samples, the system can more accurately model the gait features of the user under the change of daily state, thereby continuously optimizing the recognition effect.

[0121] On the other hand, for the case of recognition failure but actual unlocking behavior (such as the homeowner remotely authorizing the visitor to open the door), the system also records the plantar pressure data and unlocking time of this time to form a complete use log. The log information can be queried by the homeowner at any time, and can also be provided to the public security system as auxiliary evidence data for access control behavior.

[0122] Through the above steps, the embodiment constructs a three-in-one identity recognition closed-loop system of "recognition output - door lock execution - data recording", which not only ensures the timeliness and accuracy of the door lock control, but also guarantees the sustainable optimization ability and traceability of the system, and has good practicability and safety.

[0123] To further verify the practicability and identification accuracy of the present application, the inventors conducted 20 pedestrian sample collection and empirical tests on a school experimental platform. The experiment covered a variety of real use scenarios, including users wearing different types of shoe soles, changes in weight bearing state, weight fluctuations, and identification needs in different time periods. First, 20 subjects were collected and entered into the system, then each person was tested 6 times. After processing by the MCTBA-Net model, the system's identification accuracy in the standard test set reached 100% without false positives or false negatives. The results, as shown in Table 1, indicate that the present application has high robustness and accuracy, and can adapt to individual differences and fluctuations in the use environment in daily applications. The technical solution has practical feasibility and engineering promotion value. Figure 6

[0124] The examples described in the present application are merely to describe the preferred embodiments of the present application, and do not limit the concept and scope of the present application. Various modifications and improvements to the technical solutions of the present application made by those skilled in the art without departing from the design idea of the present application shall fall within the scope of protection of the present application.​

Claims

1. A method for unlocking an access control system based on plantar pressure, which is characterized by: The access control system includes an acquisition unit, a data processing unit, a storage unit, and an execution unit. The data processing unit includes an image processing module and an MCTBA-Net neural network model. The unlocking method steps are as follows: S1. The user steps on the acquisition unit multiple times. The acquisition unit collects pressure data at each point on the sole of the foot in real time and plots it as a dynamic pressure distribution map in real time. S2, the image processing module performs image preprocessing on the dynamic pressure distribution map obtained in step S1 and sorts it in chronological order; The S3 and MCTBA-Net neural network models include optimized multi-scale convolution modules, temporal convolution modules, bidirectional long short-term memory modules, and multi-head attention mechanism modules, which are used to extract discriminative features in spatial and temporal dimensions from the dynamic pressure distribution map processed in step S2 and convert them into gait features. S4, comparing and verifying the gait feature obtained in step S3 with the gait feature dataset of N users pre-stored in the storage unit to obtain a verification result; S5. The execution unit determines whether to unlock the system according to the verification result of step S4.

2. The door unlocking method based on plantar pressure according to claim 1, characterized in that: In step S3, the method by which the MCTBA-Net neural network model converts the dynamic pressure distribution map into gait features is: S31, optimizing the multi-scale convolution module using the ResNet-50 network model combined with ablation experiments to obtain an optimized multi-scale convolution module, which is used to extract high-precision multi-scale textures and pressure distributions of different plantar regions in the dynamic pressure distribution map processed in step S2; S32. In the temporal convolution module, two layers of one-dimensional convolution normalized by weight_norm are used with residual connections, and the input and output channels are both latent_dim. This parameter combination is verified by repeated ablation experiments to obtain an optimized temporal convolution module, which is used to model the image processed in step S31 and preliminarily establish inter-frame dependencies; it can capture short-range dynamic changes between adjacent frames for training; S33. In the bidirectional long short-term memory module, a bidirectional LSTM with lstm_layers layers is used, with an input dimension of latent_dim and an output dimension of 2×H, and forward and backward information are fused through residual connections. The number of layers and hidden_size parameters are determined after grid search and convergence speed evaluation to optimize the bidirectional long short-term memory module, which is used to retrain the image processed in step S32 to achieve efficient and stable long-range dependency learning; S34. In the multi-head attention mechanism module, it is optimized to a fixed 4-head configuration, and Tanh activation and time average pooling are added after the attention output to obtain the optimized multi-head attention mechanism module, which is used to dynamically focus the key frames of the image trained in step S33, and realize the conversion of plantar pressure data into the user's gait characteristics.

3. The door unlocking method based on plantar pressure according to claim 2, characterized in that: In step S31, the multi-scale convolution module is truncated and reconstructed based on the original backbone of the ResNet-50 network model. By parallel extraction of 256, 512 and 2048 channel features in the first, second and third stages, and introducing adaptive average pooling and fully connected mapping after each branch, it breaks the limitation of the single path of the original network, realizes more refined multi-scale spatial feature fusion, and achieves high-precision extraction of multi-scale texture and pressure distribution in different plantar regions. The number of channels and convolution kernel configuration are optimized through rigorous ablation experiments.

4. The door unlocking method based on plantar pressure according to claim 1, characterized in that: In step S1, the acquisition unit is an array pressure sensor, and the array pressure sensor is composed of a plurality of sensing nodes arranged.

5. The door unlocking method based on plantar pressure according to claim 4, characterized in that: In step S1 , the dynamic pressure distribution diagram includes at least one complete stepping cycle.

6. The door unlocking method based on plantar pressure according to claim 5, characterized in that: In step S2, the image processing module reads the first N frames of the dynamic pressure distribution map obtained in step S1 as standard samples for image preprocessing, where N is at least 7 frames; The image processing module represents the pressure distribution map of each frame in the C×H×W format, and stacks the captured N consecutive frames in the order of acquisition time into a high-dimensional tensor with a shape of T×C×H×W, so as to fully preserve the spatial and temporal dynamic characteristics of gait.

7. The door unlocking method based on plantar pressure according to claim 6, characterized in that: In step S2, the image preprocessing includes image noise filtering, image size unification, spatial enhancement and pixel normalization, wherein the spatial enhancement includes random cropping and horizontal flipping.

8. The door unlocking method based on plantar pressure according to claim 1, characterized in that: The process of establishing the gait feature dataset of N users pre-stored in the storage unit in step S4 is as follows: (1) In at least three different load scenarios, the user steps on the acquisition unit multiple times. The acquisition unit collects pressure data at each point on the sole of the foot in real time and plots it as a dynamic pressure distribution map in real time. The three different load scenarios are respectively the loads of objects of three weights: 0, a, and b, where 0 < a < b. (2) the image processing module performs image preprocessing on the dynamic pressure distribution maps obtained for each scene in step (1) and sorts them in chronological order; (3) The MCTBA-Net neural network model converts the dynamic pressure distribution map processed in step (2) into gait features and uploads it to the storage unit as the gait feature dataset of the user.

9. The door unlocking method based on plantar pressure according to claim 1, characterized in that: In step S5, the unlocking is determined based on the verification result output by the MCTBA-Net neural network model in step S4: S51. When the verification result indicates that the user identity matches, the execution unit executes the unlocking instruction to drive the lock cylinder to unlock; S52. When the verification result indicates that the user identity does not match, the execution unit keeps the smart lock core in a locked state.

10. The access control unlocking method based on plantar pressure according to any one of claims 1 to 9, characterized in that: Also includes: S6. The verification result and the corresponding gait feature are transmitted to the storage unit, and the gait feature data set of the corresponding user in the storage unit is dynamically updated to improve the subsequent recognition accuracy of the system and facilitate security audit and user management.

Images