Intelligent access control management method and system based on multimodal recognition and Internet of Things technology

Abstract

The present invention discloses an intelligent access control management method and system based on multimodal recognition and Internet of Things technology. The method collects multimodal data through distributed Internet of Things devices, performs synchronous alignment and adaptive preprocessing, extracts multi-scale features and dynamically fuses them. Nonlinear dimensionality reduction, multi-order tensor decomposition, graph convolutional network and multi-head attention mechanism are used for deep feature fusion. Combined with dynamically updated identity templates, multi-factor identity authentication is achieved, including biometric matching, behavior pattern analysis and environmental context verification. The verification result is determined by an adaptive decision threshold, and dynamic access control, adaptive crowd control and real-time abnormal event detection are performed. This method improves the accuracy of identity recognition, system security and adaptability, and is suitable for intelligent access control management in complex environments such as construction sites, parks and shopping malls.

Description

Technical Field

[0001] The present invention relates to an artificial intelligence application method, in particular to an intelligent access control management method and system based on multimodal recognition and Internet of Things technologies. Background Art

[0002] In today's rapidly developing era of smart cities and the Internet of Things, smart access control systems are receiving more and more attention as a key component to ensure public safety and improve user experience. The smart access control management method based on multimodal recognition and Internet of Things technology can not only improve security and reliability, but also achieve more intelligent and personalized access control. It provides all-round security for modern buildings, corporate parks, schools and public places. At the same time, it can also adapt to the needs of different scenarios, such as crowd management in large-scale events, rapid evacuation in emergencies, etc., thereby significantly improving the overall security management efficiency and user satisfaction.

[0003] At present, the research on intelligent access control systems mainly focuses on the application of single or dual biometric recognition technology, such as fingerprint recognition, face recognition or iris recognition. Although security has been improved to a certain extent, there are still some limitations. For example, some systems use simple threshold judgment methods for identity authentication, which cannot effectively deal with complex spoofing attacks. In terms of crowd management, some systems use simple counting methods, which cannot achieve accurate crowd prediction and intelligent guidance. At the same time, the existing systems have limited capabilities in abnormal event detection and response, and can often only identify a few predefined abnormal situations, making it difficult to deal with complex and changing security threats.

[0004] In short, although the intelligent access control system is constantly developing, there are still some technical problems that need to be solved. First of all, the data is scattered. Different devices from different manufacturers have different principles and data formats. In the existing park, a variety of devices have been deployed. How to fuse the above data, especially how to quickly extract and fuse complex features, needs to be solved. In terms of feature extraction and matching algorithms, existing methods often have problems of high computational complexity and insufficient generalization ability when processing large-scale and high-dimensional data. In addition, the adaptive threshold adjustment mechanism in a dynamic environment is not yet perfect, and it is difficult to balance the security and availability of the system. In terms of abnormal behavior detection, how to effectively use time series information and context information to improve the accuracy and real-time performance of detection is also an important issue. In terms of crowd flow prediction and management, the existing methods have insufficient response capabilities to emergencies and nonlinear changes, and it is difficult to achieve accurate short-term and medium- and long-term predictions. In terms of system scalability and interoperability, how to design a flexible architecture to adapt to application scenarios of different scales and types and reduce deployment and promotion costs also needs to be solved. Therefore, research and innovation are needed. Summary of the invention

[0005] The purpose of the invention is to provide an intelligent access control management method and system based on multimodal recognition and Internet of Things technology, in order to solve the above-mentioned problems existing in the prior art.

[0006] Technical solution, according to one aspect of the present application, an intelligent access control management method based on multimodal recognition and Internet of Things technology includes the following steps:

[0007] Step S1: Collect multimodal data through distributed IoT devices, perform synchronization and alignment processing, and adaptive preprocessing, obtain preprocessed data and extract multiscale features from it, dynamically fuse the multiscale features to obtain fused features, compress and preliminarily reduce the dimensionality of the fused features and preprocessed data, and transmit them to the central processing unit for stream processing, store the processed data in a time series database and update the feature index;

[0008] Step S2, obtaining the processed data and feature index, performing adaptive enhancement processing on each modal feature, then performing feature selection based on mutual information, using a nonlinear dimensionality reduction method for dimensionality reduction, and performing deep fusion through multi-order tensor decomposition, graph convolutional network and multi-head attention mechanism to obtain the final fused feature vector;

[0009] Step S3, obtaining the final fused feature vector and the pre-built feature distribution model, dynamically updating the stored identity template and performing multi-factor authentication; using an adaptive decision threshold to determine the final authentication result, generating an authentication result including the identified identity ID, confidence score and verification level; multi-factor authentication includes biometric matching, behavior pattern analysis and environmental context verification;

[0010] Step S4, obtain the identity authentication result, perform dynamic access control based on the user identity and the authentication result, perform adaptive crowd control, and perform real-time abnormal event detection and response to comprehensively arrive at the final access control decision; wherein, the adaptive crowd control includes crowd prediction and congestion detection.

[0011] According to another aspect of the present application, there is also provided an intelligent access control management system based on multimodal recognition and Internet of Things technology, including:

[0012] at least one processor; and,

[0013] a memory communicatively connected to at least one of the processors; wherein,

[0014] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent access control management method based on multimodal recognition and Internet of Things technology described in any of the above technical solutions.

[0015] Beneficial effects: By integrating distributed IoT devices to collect multimodal data and perform synchronous alignment and adaptive preprocessing, the quality and consistency of the original data are greatly improved, and the existing equipment is fully utilized without the need for equipment replacement. Multi-scale feature extraction and dynamic fusion technology effectively capture the multi-dimensional information of the data and enhance the expressiveness of the features. The combination of nonlinear dimensionality reduction, multi-order tensor decomposition, graph convolutional network and multi-head attention mechanism realizes deep feature fusion and significantly improves the distinguishability and robustness of the features. In addition, the dynamically updated identity template and multi-factor authentication method, including biometric matching, behavioral pattern analysis and environmental context verification, build a multi-level security protection system to reduce the risk of identity fraud and deception. The application of adaptive decision thresholds enables the system to dynamically adjust the verification criteria according to different scenarios, balancing security and user experience. Through the combination of dynamic access control, adaptive crowd control and real-time abnormal event detection, the system can intelligently respond to complex and changing actual scenarios, optimize traffic efficiency and improve the level of security management. It is suitable for use in complex environments with high security requirements and large-scale crowd flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the present invention.

[0017] Figure 2 It is a flow chart of step S1 of the present invention.

[0018] Figure 3 It is a flow chart of step S2 of the present invention.

[0019] Figure 4 It is a flow chart of step S3 of the present invention.

[0020] Figure 5 It is a flow chart of step S4 of the present invention. DETAILED DESCRIPTION

[0021] like Figure 1 As shown, according to one aspect of the present application, an intelligent access control management method based on multimodal recognition and Internet of Things technology includes the following steps:

[0022] Step S1: Collect multimodal data through distributed IoT devices, perform synchronization and alignment processing, and adaptive preprocessing, obtain preprocessed data and extract multiscale features from it, dynamically fuse the multiscale features to obtain fused features, compress and preliminarily reduce the dimensionality of the fused features and preprocessed data, and transmit them to the central processing unit for stream processing, store the processed data in a time series database and update the feature index;

[0023] Step S2, obtaining the processed data and feature index, performing adaptive enhancement processing on each modal feature, then performing feature selection based on mutual information, using a nonlinear dimensionality reduction method for dimensionality reduction, and performing deep fusion through multi-order tensor decomposition, graph convolutional network and multi-head attention mechanism to obtain the final fused feature vector;

[0024] Step S3, obtaining the final fused feature vector and the pre-built feature distribution model, dynamically updating the stored identity template and performing multi-factor authentication; using an adaptive decision threshold to determine the final authentication result, generating an authentication result including the identified identity ID, confidence score and verification level; multi-factor authentication includes biometric matching, behavior pattern analysis and environmental context verification;

[0025] Step S4, obtain the identity authentication result, perform dynamic access control based on the user identity and the authentication result, perform adaptive crowd control, and perform real-time abnormal event detection and response to comprehensively arrive at the final access control decision; wherein, the adaptive crowd control includes crowd prediction and congestion detection.

[0026] This application builds a comprehensive, intelligent and secure access control management system by integrating multimodal data acquisition, efficient feature extraction, intelligent identity authentication and dynamic access control. First of all, the application of distributed IoT devices realizes all-round and multi-angle data acquisition, including multiple modalities such as vision, audio, thermal imaging, motion and environment, which greatly enhances the system's perception of complex scenes. The use of precision time protocol servers for data synchronization solves the problem of time alignment of multi-source heterogeneous data and lays the foundation for subsequent feature fusion. The introduction of adaptive preprocessing technologies, such as image enhancement, audio noise reduction and thermal map super-resolution reconstruction, significantly improves the quality of raw data, especially in complex environments (such as low light and high noise).

[0027] In terms of feature processing, multi-scale feature extraction and dynamic fusion methods capture the key information of data at different scales and enhance the expressiveness of features. The deep fusion method using high-order tensor decomposition, graph convolutional network and multi-head attention mechanism fully utilizes the high-order correlation and spatial structure information between features to generate more expressive fused feature vectors. This multi-level and multi-angle feature processing strategy greatly improves the system's adaptability to complex scenarios and diverse users. Especially when processing large-scale, high-dimensional multimodal data, this method shows significant advantages and can better capture the complementary information between different modalities.

[0028] The dynamic template update and multi-factor verification mechanism are introduced in the identity verification process, which greatly improves the security and adaptability of the system. The pre-built feature distribution model provides a statistical basis for identity verification and enhances the system's ability to identify abnormal situations. The use of adaptive decision thresholds enables the system to dynamically adjust the verification criteria according to different scenarios and user characteristics, balancing security and user experience. The introduction of multi-factor authentication, including biometric matching, behavioral pattern analysis, and environmental context verification, builds a multi-level security protection system, significantly reducing the risk of identity impersonation and deception.

[0029] In terms of access control, the system implements dynamic permission management based on user identity and verification results, providing refined and personalized access control. The introduction of adaptive crowd control, including crowd prediction and congestion detection, enables the system to actively adjust traffic strategies, optimize space utilization efficiency, and improve the overall traffic experience. The implementation of real-time abnormal event detection and response mechanism has greatly improved the system's security and ability to respond to emergencies.

[0030] In summary, through the deep fusion and intelligent analysis of multimodal data, high-precision and high-reliability identity recognition and verification are achieved. The system can effectively cope with various complex environments and abnormal situations, such as changes in lighting, posture changes, background noise, etc., showing strong robustness and adaptability. At the same time, through adaptive learning and dynamic update mechanisms, the system can continuously optimize its performance and adapt to changing user behaviors and environmental conditions. This comprehensive and intelligent access control management method not only improves security, but also greatly improves user experience. It can provide reliable, efficient and intelligent access control management services in various scenarios with high security requirements, such as government agencies, financial institutions, large enterprises, etc. Especially in large public places or high-security areas, this method can effectively balance security requirements and traffic efficiency, reduce congestion and safety hazards, and provide strong technical support for the development of modern intelligent buildings and smart cities.

[0031] like Figure 2 As shown, according to one aspect of the present application, step S1 is specifically:

[0032] Step S11: collect multi-angle visual data, spatialized audio signals, heat map sequences, environmental parameters, and the number of people entering and leaving through distributed IoT devices; call the precision time protocol server to synchronize the clock of each IoT device, add a microsecond timestamp to each data point, and obtain a data set with a timestamp;

[0033] Step S12: Based on the data set with timestamps, calculate the complexity of each modal data, and dynamically adjust the sliding window size according to the complexity; apply the dynamic time warping algorithm to the data of each modality, and calculate the alignment matrix between different modalities; resample and interpolate the data set with timestamps according to the alignment matrix to obtain the aligned multimodal time series;

[0034] Step S13, reading and adaptively preprocessing the aligned multimodal time series to obtain each modal feature; the adaptive preprocessing includes enhancement and deblurring of images and depth maps, enhancement of audio signals, super-resolution reconstruction of thermal map sequences, smoothing of millimeter-wave radar signals, and correction of outliers of environmental parameters; each modal feature includes visual features, audio features, thermal features, motion features, and environmental features;

[0035] Step S14, organizing the extracted visual features, audio features, thermal features, motion features and environmental features into a fifth-order tensor; calculating the pattern correlation matrix of the fifth-order tensor; dynamically adjusting the rank of the core tensor of Tucker decomposition according to the correlation matrix; applying the adaptive Tucker decomposition algorithm to decompose the tensor into a core tensor and a factor matrix; and obtaining fusion features by reconstructing the core tensor and the factor matrix;

[0036] Step S15, for the fused features, calculate the local complexity and global redundancy; dynamically adjust the quantization parameter and the coding parameter according to the complexity and redundancy; use adaptive quantization and entropy coding to compress the fused features to obtain compressed features; calculate the covariance matrix for the preprocessed raw data; perform feature decomposition on the covariance matrix, select the eigenvectors corresponding to the largest eigenvalues ​​to form a projection matrix; project the preprocessed raw data onto the projection matrix to obtain the data after dimensionality reduction;

[0037] Step S16, merging the compressed features and the reduced-dimensional data into a data stream; dynamically calculating the optimal shard size and transmission priority according to the network status and data importance; sharding the data stream according to the shard size to obtain a data shard set; determining the transmission order of each shard according to the transmission priority; and transmitting the data shards to the central processing unit using the improved MQTT protocol;

[0038] Step S17, receiving the transmitted data fragments, and recombining to obtain the original data stream; calculating the quality index of each data point in the data stream; dynamically adjusting the cleaning threshold according to the quality index; applying the adaptive cleaning algorithm to the data stream, removing low-quality data points, and obtaining a cleaned data stream; analyzing the feature distribution of the cleaned data stream, and selecting a suitable feature transformation function according to the distribution; applying the feature transformation function to perform feature transformation on the cleaned data stream to obtain a transformed data stream;

[0039] Step S18: Apply an incremental clustering algorithm to the converted data stream, dynamically update the cluster center and cluster radius, and obtain clustering results; write the clustering results into a time series database; calculate the importance score and access frequency of each data point; dynamically adjust the data retention period according to the importance score and access frequency; apply an adaptive data retention strategy to archive or delete expired data;

[0040] Step S19, call the constructed incremental LSM tree structure to insert new data into the memory table; when the memory table reaches a preset threshold, merge it into a lower level on the disk; dynamically adjust the number of levels of the LSM tree and the size of each level to optimize read and write performance; finally obtain the stored processed data and updated feature index.

[0041] This step collects multimodal data through distributed IoT devices, and performs synchronization and alignment processing to achieve efficient fusion of multi-source heterogeneous data. Adaptive preprocessing technology is used to dynamically adjust processing parameters according to the characteristics of different modal data, thereby improving data quality and the accuracy of feature extraction. The method of multi-scale feature extraction and dynamic fusion captures the key information of data at different scales and enhances the expressiveness of features. By compressing and preliminarily reducing the dimensionality of fused features and preprocessed data, the amount of data transmission is greatly reduced, and the real-time performance of the system is improved. Stream processing technology and time series database storage are used to achieve efficient processing and rapid retrieval of large-scale data. These technical innovations combined have significantly improved the system's ability to process multimodal data, laying a solid data foundation for subsequent identity recognition and verification. Especially in complex environments, such as lighting changes, noise interference, etc., this method shows stronger robustness and effectively reduces the misrecognition rate.

[0042] like Figure 3 As shown, according to one aspect of the present application, step S2 is specifically:

[0043] Step S21, reading processed data and feature indexes from a time series database; extracting visual features, audio features, thermal features, motion features, and environmental features according to the feature indexes; calculating statistical properties of each feature; determining enhancement parameters for each feature according to the statistical properties; the statistical properties include mean, variance, skewness, and kurtosis;

[0044] Step S22, calculate the spatial attention weight and channel attention weight of the visual feature; multiply the visual feature with the spatial attention weight and the channel attention weight to obtain the enhanced visual feature; calculate the time-frequency spectrum of the audio feature; estimate the noise power spectrum; use spectral subtraction to enhance the time-frequency spectrum; re-extract the Mel-frequency cepstral coefficient feature from the enhanced time-frequency spectrum to obtain the enhanced audio feature; apply adaptive histogram equalization to the thermal feature; adjust the histogram distribution of each subband to obtain the enhanced thermal feature; use wavelet threshold denoising to the motion feature; adaptively select the threshold to obtain the denoised motion feature; apply exponential weighted moving average to the environmental feature; calculate the smoothing coefficient to obtain the smoothed environmental feature; obtain the enhanced features of each modality;

[0045] Step S23, combining the enhanced modal features with the pre-stored identity tags; calculating the mutual information between each feature and the identity tag; setting a mutual information threshold, and selecting a feature subset whose mutual information is greater than the threshold;

[0046] Step S24, constructing a feature dictionary and a sparse coding matrix; optimizing the objective function, using the alternating direction multiplier method to find the optimal solution, and obtaining a sparse representation of the feature;

[0047] Step S25, constructing a k-nearest neighbor graph for the sparse representation; calculating the Laplacian matrix of the graph; solving the generalized eigenvalue problem; selecting eigenvectors corresponding to the smallest number of non-zero eigenvalues ​​to form a projection matrix; projecting the sparse representation onto the projection matrix to obtain features after dimensionality reduction;

[0048] Step S26, organizing the reduced-dimensional features into high-order tensors; calculating the correlation matrix between the various-order modes of the tensor; determining the rank vector of the tensor decomposition according to the correlation matrix; applying the tensor train decomposition algorithm to decompose the tensor into a series of core tensors and factor matrices; and reconstructing the core tensors and factor matrices to obtain tensor fusion features;

[0049] Step S27, constructing a feature graph based on the k-nearest neighbor graph; calculating the adjacency matrix and degree matrix of the feature graph; constructing a graph convolutional network layer; inputting the reduced-dimensional features into the multi-layer graph convolutional network to obtain graph convolutional features;

[0050] Step S28, connecting the tensor fusion features and the graph convolution features to form a multimodal feature matrix; calculating the self-attention matrix of the multimodal feature matrix; calculating the multi-head attention output; connecting the outputs of all attention heads; processing the connected outputs through a feedforward neural network to obtain the final fused feature vector;

[0051] Step S29, write the final fused feature vector into the time series database and update the feature index; calculate the statistical characteristics of the fused feature vector; update the feature distribution model based on the statistical characteristics; store the updated feature distribution model in the model library for subsequent identity recognition and verification; the statistical characteristics include mean, variance and covariance matrix.

[0052] This step significantly improves the quality and discrimination of features by adaptively enhancing the features of each modality. The feature selection method based on mutual information effectively removes redundant and irrelevant features, improving the efficiency and accuracy of subsequent processing. The use of nonlinear dimensionality reduction methods, such as manifold learning, retains the intrinsic structure of the data and overcomes the limitations of traditional linear dimensionality reduction methods. The deep fusion method of multi-order tensor decomposition, graph convolutional network and multi-head attention mechanism makes full use of the high-order correlation and spatial structure information between features to generate a more expressive fused feature vector. This multi-level and multi-angle feature processing and fusion strategy greatly enhances the system's adaptability to complex scenarios and diverse users. Especially when processing large-scale, high-dimensional multimodal data, this method shows significant advantages, can better capture the complementary information between different modalities, and improve the accuracy and reliability of identity recognition.

[0053] like Figure 4 As shown, according to one aspect of the present application, step S3 is specifically:

[0054] Step S31, read the stored identity template set and fused feature vector from the time series database; calculate the Euclidean distance between the fused feature vector and each template, and select several templates with the smallest distance as candidate templates; calculate the similarity score of the candidate templates and update them using the online stochastic gradient descent algorithm; calculate the quality score of the updated template, and determine the update weight according to the score; perform weighted average of the updated template and the original template to obtain the final updated template; write the updated template back to the time series database and update the corresponding template index;

[0055] Step S32, construct a local sensitive hash function set and apply it to all updated templates and input fused feature vectors; calculate the Hamming distance between hash codes, and select several templates with the smallest distance as candidate sets; calculate the cosine similarity and likelihood probability between the fused feature vector and each template in the candidate set, and select the best matching template; retrieve the historical behavior sequence related to the best matching template from the user behavior database, use the hidden Markov model to model and calculate the likelihood probability of the current behavior sequence; retrieve the current environment parameters and historical association data from the environment database, and use the Bayesian network model to calculate the posterior probability of the user appearing; output the best matching template, behavior consistency probability and environment context probability;

[0056] Step S33, calculate the global false alarm rate and missed alarm rate based on historical verification data; use adaptive dichotomy to search for the optimal decision threshold; calculate the confidence score of the current verification result, including the best cosine similarity, behavior sequence likelihood probability and environmental posterior probability; compare the confidence score with the optimal decision threshold; build a decision tree model, input the biometric matching result, behavior consistency score and environmental context probability into the decision tree; obtain the result of whether the verification is passed and the final verification level;

[0057] Step S34: Based on the outputs of step S32 and step S33, generate an identity authentication result including the identified identity, confidence score and verification level; write the verification result into a verification result database; update the user's verification history record; calculate and update the user's average verification performance index; adjust the user's verification strategy parameters based on the updated performance index; output the final identity authentication result and the updated user verification strategy.

[0058] This step greatly improves the security and adaptability of the system by dynamically updating the stored identity templates and multi-factor authentication. The pre-built feature distribution model provides a statistical basis for identity authentication and enhances the system's ability to identify abnormal situations. The use of adaptive decision thresholds enables the system to dynamically adjust the verification criteria according to different scenarios and user characteristics, balancing security and user experience. The introduction of multi-factor authentication, including biometric matching, behavioral pattern analysis, and environmental context verification, builds a multi-level security protection system, significantly reducing the risk of identity fraud and deception. In particular, the introduction of behavioral pattern analysis and environmental context verification enables the system to identify abnormal access behaviors and environmental factors that do not conform to normal patterns, further improving security. This comprehensive verification method not only improves the accuracy of recognition, but also can cope with complex and changeable practical application scenarios, such as access control in different time periods and under different environmental conditions, providing more comprehensive and reliable technical support for intelligent access control management.

[0059] like Figure 5 As shown, according to one aspect of the present application, in step S4, dynamic access control is performed based on the user identity and the verification result, specifically:

[0060] Step S41: retrieve the historical behavior data corresponding to the identity ID from the user database, extract the feature vector of each record, including access time, location, duration and other information, and construct a feature matrix; calculate the covariance matrix of the feature matrix, perform feature decomposition on the covariance matrix, obtain the first k principal components, and construct a projection matrix; project the feature matrix onto the projection matrix to obtain the behavior features after dimensionality reduction; k is a natural number greater than 1;

[0061] Step S42, obtaining the reduced-dimensional behavior features in step S41, and processing them using the fuzzy c-means clustering algorithm; initializing the cluster center and the membership matrix, and iteratively updating the membership and cluster center until the objective function converges or reaches the maximum number of iterations; assigning role labels to users according to the final clustering results;

[0062] Step S43: retrieve the predefined role-authority mapping relationship from the authority database, and combine the user role label obtained in step S42 and the verification level and confidence score obtained in step S37 to build an ontology model containing concepts such as users, roles, resources and operations and their relationships; use the description logic reasoning engine to reason on the ontology model to obtain the set of resources accessible to the user and the set of allowed operations.

[0063] According to one aspect of the present application, in step S4, adaptive crowd flow control is performed, and the process is specifically as follows:

[0064] Step S44: retrieve historical crowd flow data from the crowd flow database, construct a time series, and use a sliding window method to process the time series data; divide the processed data into a training set and a test set, use a pre-built and trained LSTM model to predict the crowd flow in the next m time steps, and obtain a predicted sequence, which is recorded as crowd flow data, where m is a natural number;

[0065] Step S45: obtain the real-time crowd distribution data and crowd data of the current site, calculate the local density of each location point; calculate the minimum distance from each point to a point with higher density, and identify the density peak point as the congestion center according to the product of density and distance; for each congestion center, calculate the congestion situation, including the impact range and severity;

[0066] Step S46, construct a topological map of the site, where nodes represent locations and edges represent accessible paths; calculate the cost function for each edge based on the congestion situation; set the current location as the starting point and the target location as the end point, and use the A algorithm to find the optimal path from the starting point to the end point; wherein the heuristic function of the A algorithm combines the actual cost from the starting point to the current node and the estimated cost from the current node to the end point.

[0067] According to one aspect of the present application, in step S4, real-time abnormal event detection and response are performed to comprehensively obtain the final access control decision, specifically:

[0068] Step S47, retrieve the normal behavior pattern set from the behavior pattern database, extract the behavior sequence of the current user; calculate the deviation between each behavior in the current user behavior sequence and each pattern in the normal pattern set; construct an isolation forest model, including multiple isolated trees; for each tree, randomly select features and split points, recursively split the data until all samples are isolated or the maximum depth is reached; calculate the average path length of each behavior in the current user behavior sequence in the isolation forest, and then calculate the anomaly score; compare the anomaly score with the preset threshold to determine whether it is an abnormal behavior;

[0069] Step S48: deploy multiple intelligent agents, each of which is responsible for monitoring and responding to a specific area or function, maintaining local status and decision-making rule sets; when abnormal behavior is detected, the relevant agent broadcasts event information to other agents; the agent that receives the event information generates a response decision based on its own status and rule set; the Raft consensus algorithm is used to reach a global consensus decision among agents, first electing a leader agent, the leader collects the decisions of all agents, merges them into a global decision, and synchronizes the global decision to all agents;

[0070] Step S49, comprehensively process the access permission reasoning results, role assignment results, optimal path, anomaly detection results and global consensus decision to generate the final access control decision, including whether to grant access permission, access level, routing information and alarm level; write the generated decision into the decision log database, and update the access control policy library at the same time; wherein the access control decision includes whether to grant access permission, access level, routing information and alarm level.

[0071] This step achieves refined and personalized permission management by dynamically controlling access based on user identity and verification results. The introduction of adaptive crowd control, including crowd prediction and congestion detection, enables the system to actively adjust traffic strategies, optimize space utilization efficiency, and improve the overall traffic experience. The implementation of real-time abnormal event detection and response mechanism has greatly improved the security of the system and its ability to respond to emergencies. This method of comprehensively considering identity authentication, crowd control, and abnormal event processing enables the intelligent access control system to not only accurately identify individuals, but also make intelligent decisions based on the overall situation. Especially in large public places or high-security areas, this method can effectively balance security requirements and traffic efficiency, reduce congestion and safety hazards. By dynamically adjusting access strategies and traffic paths, the system can maximize traffic efficiency while ensuring safety, providing more intelligent and humane access control management services.

[0072] According to another aspect of the present application, step S13 is specifically:

[0073] Step S131, calculating the local complexity of the received image sequence and depth map sequence; adaptively adjusting the contrast limit parameter according to the calculated local complexity, applying the adjusted parameter to the contrast limited adaptive histogram equalization algorithm, processing the image sequence and depth map sequence, and obtaining an enhanced image sequence and depth map sequence;

[0074] Step S132, obtaining the enhanced image sequence in the previous step, calculating the optical flow field between adjacent frames in the sequence; estimating the motion blur kernel according to the calculated optical flow field, inputting the estimated blur kernel into the Wiener filter, performing deblurring processing on each frame image in the enhanced image sequence, and obtaining a deblurred image sequence;

[0075] Step S133, obtaining the deblurred image sequence and enhanced depth map sequence from the first two steps, calculating the image gradient and local direction distribution; adaptively adjusting the scale parameter and direction parameter of the scale-invariant feature transformation algorithm according to the calculated gradient and direction distribution; applying the adjusted parameters to the scale-invariant feature transformation algorithm to extract visual features from the deblurred image sequence and enhanced depth map sequence;

[0076] Step S134: receiving the audio signal aligned in step S13, estimating the signal-to-noise ratio at each time point, azimuth angle, and elevation angle; dynamically adjusting the beamforming weight according to the estimated signal-to-noise ratio, applying the adjusted weight to the beamforming algorithm, processing the original audio signal, and obtaining an enhanced audio signal;

[0077] Step S135, receiving the audio signal enhanced in the previous step, and calculating its short-time spectrum; dynamically adjusting the center frequency and bandwidth of the Mel filter bank according to the time-varying characteristics of the calculated short-time spectrum; applying the adjusted Mel filter bank to the short-time spectrum, calculating the Mel frequency cepstral coefficients, and obtaining dynamic Mel frequency cepstral coefficients as audio features;

[0078] Step S136, obtaining the heat map sequence aligned in step S13, constructing a residual network including multiple residual blocks and upsampling layers; using a high-resolution heat map data set to train the residual network to minimize the reconstruction error; applying the trained residual network to each frame in the original heat map sequence to perform super-resolution reconstruction to obtain a high-resolution heat map sequence;

[0079] Step S137, obtaining the high-resolution thermal image sequence in the previous step, applying the empirical mode decomposition algorithm to it, and obtaining a series of intrinsic mode functions; performing Hilbert transform on each intrinsic mode function, calculating the instantaneous frequency and instantaneous amplitude; combining the calculated instantaneous frequency and instantaneous amplitude into a thermal signature;

[0080] Step S138, receiving the millimeter-wave radar signal aligned in step S13, calculating the dynamic characteristics of the signal, including the signal strength change rate and the phase change rate; adaptively adjusting the process noise covariance matrix and the measurement noise covariance matrix of the Kalman filter according to the calculated dynamic characteristics; inputting the adjusted covariance matrix into the Kalman filter, filtering the original signal, and obtaining a smoothed signal;

[0081] Step S139, receiving the millimeter-wave radar signal smoothed in the previous step, calculating its power spectrum density and autocorrelation function; combining the calculated power spectrum density and autocorrelation function into a motion feature;

[0082] Step S1310, obtaining the environmental parameters aligned in step S13, including temperature, humidity, light, carbon dioxide concentration and fine particulate matter index; calculating the local anomaly score for each parameter, using the sliding window method to calculate the local mean and standard deviation of each parameter, and comparing the current value with the local statistic to obtain the anomaly score; identifying the outliers according to the calculated anomaly scores, and correcting the outliers using the interpolation method or the local regression method to obtain the corrected environmental parameters;

[0083] Step S1311, obtaining the corrected environmental parameters in the previous step, calculating the mutual information matrix between the parameters; dynamically adjusting the separation matrix of the independent component analysis algorithm according to the calculated mutual information matrix; applying the adjusted separation matrix to the independent component analysis algorithm to extract the main environmental features from the corrected environmental parameters;

[0084] Step S1312: Integrate the various modal features obtained in the previous steps, including visual features, audio features, thermal features, motion features and environmental features, to form a multimodal feature set; output this multimodal feature set for use in subsequent processing steps.

[0085] By collecting multi-angle visual data, spatialized audio signals, heat map sequences, environmental parameters and the number of people entering and leaving, all-round scene perception is achieved. The use of precision time protocol servers ensures accurate synchronization of data between different devices, laying the foundation for subsequent multimodal fusion. The application of dynamic time warping algorithm solves the problem of time alignment of data of different modalities and improves the accuracy of data fusion. Adaptive preprocessing techniques, such as image enhancement, audio enhancement and heat map super-resolution reconstruction, significantly improve the quality of raw data. Especially in complex environments, such as low light and high noise, these preprocessing techniques can effectively extract key information. The method of extracting and organizing multimodal features into fifth-order tensors fully captures the multi-dimensional information of the data and provides rich feature representation for subsequent deep fusion. The application of adaptive Tucker decomposition algorithm realizes efficient feature fusion and dimensionality reduction, which greatly reduces the amount of data while retaining key information. This comprehensive data collection and preprocessing method greatly enhances the system's adaptability to complex scenes and recognition accuracy.

[0086] According to one aspect of the present application, step S14 is specifically:

[0087] Step S141, receive the visual features, audio features, thermal features, motion features and environmental features from the previous step, and organize them into a five-order tensor T in a predefined order, where each dimension corresponds to a different feature mode, sample, time step, feature dimension and sensor channel. Use the sliding window method to process the time series data to ensure the consistency of the time dimension of the tensor. Normalize each feature and use the minimum-maximum scaling method to map the feature value to the [0,1] interval to ensure the scale consistency of different modal features.

[0088] Step S142, calculate the mode correlation matrix of the fifth-order tensor T. First, expand the fifth-order tensor T to obtain five matrices X_1, X_2, X_3, X_4, X_5, each matrix corresponding to a mode of the tensor. For each matrix X_i, calculate its covariance matrix C_i = (Xi T * X_i) / (n-1), where n is the number of samples. Normalize the covariance matrix C_i to the correlation matrix R_i, with diagonal elements equal to 1 and off-diagonal elements equal to the corresponding correlation coefficients. Construct a pattern correlation matrix M, where M_{ij} represents the average correlation between pattern i and pattern j, calculated as the average of the off-diagonal elements of R_i and R_j.

[0089] Step S143, dynamically adjust the core tensor rank of Tucker decomposition according to the pattern correlation matrix M. First, set a base rank r_base and a maximum rank r_max. For each pattern i, calculate its average correlation with other patterns a_i =Σ_j M_{ij} / (n-1), where n is the number of patterns. Define the rank adjustment function f(a_i) = r_base + (r_max - r_base) * (1 - a_i). Use this function to calculate the adjusted rank r_i = round(f(a_i)) of each pattern, where round means rounding to the nearest integer. Get the adjusted core tensor rank vector r = [r_1, r_2, r_3, r_4, r_5].

[0090] Step S144, apply the adaptive Tucker decomposition algorithm to decompose the tensor T into a core tensor and a factor matrix. First, use the high-order singular value decomposition (HOSVD) method to initialize the core tensor G and factor matrices U1, U2, U3, U4, U5. For each mode i, calculate the i-mode matrix of the tensor T, and perform truncated SVD to obtain the initial factor matrix U_i. Use these initial factor matrices to calculate the initial core tensor G. Then, apply the alternating least squares (ALS) method to optimize the decomposition results. Fix other factor matrices and update the factor matrix U of the current mode. i = T (i) * (U n @ ... @ U (i+1) @U (i-1) @ ... @U1) * G (i) + , where @ represents the Kronecker product, G_(i) represents the i-mode matrix of the core tensor G, and + represents the pseudo-inverse. Update the core tensor G = T ×1U1 T ×2U2 T ×3U3 T ×4U4 T ×5U5 T , where × i Denotes the i-mode tensor product. The ALS process is repeated until convergence or the maximum number of iterations is reached.

[0091] Step S145, reconstruct the fusion feature by using the core tensor G and the factor matrices U1, U2, U3, U4, and U5. Calculate the reconstructed tensor T_recon = G ×1U1×2U2×3U3×4U4×5U5. Rearrange the reconstructed tensor T_recon into a two-dimensional matrix F, where each row represents the fusion feature vector of a sample. Apply principal component analysis (PCA) to F for dimensionality reduction, and retain the principal components whose explained variance ratio reaches a preset threshold (such as 95%). Output the feature matrix after dimensionality reduction as the final fusion feature F_final for use in subsequent steps.

[0092] Step S146, evaluate the quality of the fused features. Calculate the mutual information between the original features and the fused features, and use the nearest entropy estimation method to estimate the high-dimensional mutual information. Calculate the feature separation before and after fusion, using the Fisher discriminant ratio as a metric. Compare the performance improvement on downstream tasks (such as classification or regression) before and after fusion. According to the evaluation results, dynamically adjust the parameters of Tucker decomposition, such as the kernel tensor rank and the number of iterations, to optimize the fusion effect. Output the fused feature F_final together with the evaluation report.

[0093] By organizing the extracted multimodal features into a fifth-order tensor, the multidimensional structural information of the data is fully retained. The calculation of the mode correlation matrix and the dynamic adjustment of the core tensor rank of the Tucker decomposition realize the adaptive modeling of the complex relationship between features. The application of the adaptive Tucker decomposition algorithm not only realizes efficient feature fusion, but also dynamically adjusts the decomposition parameters according to the data characteristics, improving the flexibility and adaptability of the model. The fused features obtained by reconstructing the core tensor and factor matrix integrate the advantageous information of each modality and enhance the expressive power of the features. Compared with traditional feature fusion technology, this high-order tensor decomposition method can better capture the high-order correlation between multimodal data and significantly improve the quality and discrimination of the fused features. Especially when processing complex multi-source heterogeneous data, this method shows superior performance and can effectively mine the complementary information between different modalities, providing a more reliable feature representation for subsequent identity recognition and verification.

[0094] According to another aspect of the present application, step S14 may also be:

[0095] Step S141, obtaining the pre-processed multimodal data from step S13, including visual data, audio data, heat map data, motion data and environmental data.

[0096] Step S142: Apply a deep convolutional neural network to the visual data to extract multi-level visual features. Use the pre-trained ResNet model as a feature extractor to extract the feature maps of the last three convolutional layers, and obtain a visual feature vector of fixed dimension through global average pooling.

[0097] Step S143: Calculate the short-time Fourier transform of the audio data to obtain a time-frequency spectrum. Apply a convolutional neural network to process the time-frequency spectrum and extract audio features. Use the pre-trained VGGish model as a feature extractor and obtain the output of the last fully connected layer as an audio feature vector.

[0098] Step S144: Apply a three-dimensional convolutional neural network to the heat map data to extract spatiotemporal thermal features. Use a custom 3D-CNN model, including three 3D convolutional layers and two fully connected layers, to extract thermal feature vectors.

[0099] Step S145: Apply a long short-term memory network (LSTM) to the motion data to extract temporal motion features. Construct a bidirectional LSTM network, including two layers of LSTM units and one fully connected layer, to extract motion feature vectors.

[0100] Step S146: Apply a multi-layer perceptron to the environmental data to extract environmental features. Construct a three-layer fully connected network and use the ReLU activation function to extract environmental feature vectors. Connect all extracted feature vectors to form a multimodal feature vector, which is output to the next step.

[0101] According to one aspect of the present application, step S15 is specifically:

[0102] Step S151: Receive the fusion feature F_final outputted in the previous step. Calculate the local complexity C_l using the sliding window method. For each feature dimension in the fusion feature F_final, calculate the local variance. Define the window size w and step size s, and calculate the variance σ for the data in each window. 2 i The local complexity C_l is defined as the average value of all window variances C_l = (1 / N) * Σ σ 2 i , where N is the total number of windows.

[0103] Step S152, calculate the global redundancy R_g. First, calculate the covariance matrix Σ of F_final. Perform eigenvalue decomposition on Σ to obtain eigenvalues ​​λ_1, λ_2, ..., λ_n. Define the global redundancy R_g as the ratio of the sum of the first k largest eigenvalues ​​to the sum of all eigenvalues: R_g = (Σ i=1 k λ_i) / (Σ i=1 n λ i ), where k is selected so that the cumulative explained variance reaches a preset threshold (such as 95%).

[0104] Step S153, dynamically adjust the quantization parameter Q and the encoding parameter E according to C_l and R_g. Define the quantization level adjustment function f_Q(C_l, R_g) = Q_base * (1 + α * C_l - β * R_g), where Q_base is the basic quantization level, and α and β are weight coefficients. The coding bit adjustment function f_E(C_l, R_g) = E_base * (1 + γ * C_l - δ * R_g), where E_base is the basic coding bit, and γ and δ are weight coefficients. Calculate the adjusted quantization parameter Q = round(f_Q(C_l, R_g)) and encoding parameter E = round(f_E(C_l, R_g)).

[0105] Step S154: Compress the fused feature F_final using adaptive quantization. Apply the K-means clustering algorithm to divide the feature space into Q regions. Calculate the centroid of each region as the quantization codebook. For each data point in F_final, find the nearest centroid and replace the original value with the index of the centroid. Obtain the quantized feature F_quantized.

[0106] Step S155: Apply entropy coding to the quantized feature F_quantized. Calculate the frequency of occurrence of each symbol (quantization index) in the quantized feature F_quantized and construct a Huffman tree. Assign a variable-length binary code to each symbol according to the Huffman tree. Encode the quantized feature F_quantized using the assigned code to obtain a compressed feature F_compressed. Record the Huffman coding table for subsequent decoding.

[0107] Step S156, obtaining preprocessed raw data D_preprocessed, calculating the covariance matrix Σ_raw of D_preprocessed, performing eigenvalue decomposition on Σ_raw, and obtaining eigenvalues ​​λ_1, λ_2, ..., λ_m and corresponding eigenvectors v_1, v_2, ..., v_m.

[0108] Step S157: According to the eigenvalue size, select the eigenvectors corresponding to the first p largest eigenvalues ​​to form a projection matrix P = [v_1, v_2, ..., v_p]. The selection of p is based on the cumulative explained variance ratio, so that (Σ i= p λ_i) / (Σ i=1 m λ_i) ≥ threshold, where threshold is the preset threshold (such as 0.95).

[0109] Step S158: Project the preprocessed original data D_preprocessed onto P to obtain the dimension-reduced data D_reduced = D_preprocessed * P. Calculate the reconstruction error of the preprocessed original data D_reduced to ensure that the dimension-reduced data can retain the main information of the original data.

[0110] Step S159: Combine the compressed features F_compressed and the preprocessed raw data D_reduced into a data stream S. Use data identifiers to distinguish compressed features from reduced-dimensional data. Construct a metadata header, including data type, dimension information, compression parameters, and encoding tables. Add the metadata header to the beginning of the data stream S to form a complete data stream.

[0111] By calculating local complexity and global redundancy, accurate quantification of data characteristics is achieved. Based on these quantitative indicators, the quantization parameters and encoding parameters are dynamically adjusted, so that the compression process can adaptively balance data fidelity and compression rate. The application of adaptive quantization and entropy coding greatly improves data compression efficiency while retaining key information. Covariance matrix analysis and feature decomposition are performed on the pre-processed raw data to achieve effective dimensionality reduction and reduce data redundancy. This adaptive compression and dimensionality reduction method based on data characteristics not only significantly reduces data transmission and storage requirements, but also ensures the retention of important information. Especially when processing high-dimensional, large-scale multimodal data, this method can effectively balance computational efficiency and information retention, making real-time processing and decision-making possible. Through this intelligent data compression and dimensionality reduction strategy, the system can process larger-scale and more complex access control scene data with limited computing and storage resources, improving the response speed and scalability of the overall system.

[0112] Step S161: Receive data stream S. Obtain current network status, including bandwidth B, delay L, and packet loss rate P. Calculate network quality index NQ = f(B, L, P), where f is a weighting function. Evaluate data importance and assign importance score I according to data type (compressed features or reduced dimensionality data) and application scenario.

[0113] Step S162: Dynamically calculate the optimal fragment size s. Define the fragment size calculation function g(NQ, I) = s_base* (1 + μ * NQ + ν * I), where s_base is the base fragment size, and μ and ν are adjustment coefficients. Calculate s = round(g(NQ, I)) to ensure that s does not exceed the maximum transmission unit (MTU) limit.

[0114] Step S163: Calculate the transmission priority p. Define the priority calculation function h(NQ, I) = w_1 * NQ + w_2* I, where w_1 and w_2 are weight coefficients. According to the value of h(NQ, I), assign priority p to different parts of the data stream S. The higher the priority, the smaller the p value.

[0115] Step S164: Slice the data stream S according to the calculated slicing size s. From the beginning of S, create a slicing for every s bytes. For the last slicing that may be less than s bytes, fill it with padding data. Add a sequence number and a checksum to each slicing to form a slicing set {S_1, S_2, ..., S_m}.

[0116] Step S165: Sort the shard set {S_1, S_2, ..., S_m} according to the priority p calculated in step S163. Build a transmission queue Q, and insert the shards into Q in descending order of priority. For shards with the same priority, keep their relative order in the original data stream.

[0117] Step S166: Improve the MQTT protocol to support reliable large data transmission. Implement an adaptive QoS mechanism to dynamically adjust the QoS level according to network conditions and data importance. Introduce a data compression mechanism to compress the MQTT payload in real time. Implement a breakpoint resume function to save the transmission status when the connection is interrupted and resume the transmission from the breakpoint after reconnection.

[0118] Step S167: Use the improved MQTT protocol to transmit the data slices to the central processing unit. Establish a secure connection with the MQTT agent and use TLS / SSL to encrypt the data transmission. Take out the slices one by one from the transmission queue Q and publish them to the predefined MQTT topic. Implement a confirmation mechanism to ensure that each slice is received correctly. Monitor the network status during the transmission process and dynamically adjust the transmission parameters when necessary.

[0119] Step S168: Receive data fragments at the central processing unit. Reassemble the fragments according to the sequence number to restore the complete data stream. Verify data integrity and handle any lost or damaged fragments. Parse the metadata header to separate the compressed features and reduced dimensionality data. Decode and dequantize the compressed features to restore the original feature space. Store the received data in a temporary buffer for subsequent processing.

[0120] By dynamically calculating the optimal shard size and transmission priority, efficient use of network resources is achieved. The adaptive transmission strategy based on network conditions and data importance greatly improves the reliability and efficiency of data transmission. The application of the improved MQTT protocol enhances the system's data transmission capability in an unstable network environment. The combination of data sharding and priority transmission ensures the timely communication of key information while improving the overall transmission efficiency. In the data reorganization and quality assessment process of the central processing unit, the use of the adaptive cleaning algorithm ensures the high quality and consistency of the data. The implementation of incremental clustering and dynamic update strategies enables the system to adapt to changes in data distribution in real time, improving the real-time and accuracy of data processing. Especially in large-scale, distributed access control systems, this intelligent data transmission and processing method can effectively cope with network fluctuations and data anomalies, ensuring the stability and reliability of the system. Through this comprehensive data management strategy, the system can provide continuous and stable high-quality data support in complex and changeable practical application environments, providing a solid data foundation for intelligent access control management.

[0121] According to one aspect of the present application, step S15 and step S16 may also be:

[0122] S15, receiving the multimodal feature vector output by step S14. First, the attention mechanism is applied to weight the features of different modalities. The self-attention weight of each modal feature is calculated, and the scaled dot product attention mechanism is used. According to the calculated attention weight, the features of each modality are weighted and summed to obtain the preliminary fusion feature. Then, a cross-modal converter network is used for deep feature fusion. A converter model including a multi-head self-attention layer and a feedforward neural network layer is constructed, and the preliminary fusion feature is input to obtain the cross-modal fusion feature. Next, tensor decomposition is applied for high-order feature interaction. The cross-modal fusion feature is reshaped into a third-order tensor, and the tensor is decomposed using the Tucker decomposition method to obtain the core tensor and the factor matrix. The high-order interaction features are reconstructed based on the core tensor and the factor matrix. Finally, the gated recurrent unit (GRU) network is used to capture the temporal dependency of the features. A bidirectional GRU network is constructed, and the high-order interaction features are input to extract the temporal correlation. The hidden state of the GRU network is used as the final fusion feature vector and output for use in the next step.

[0123] S16. Receive the fused feature vector from step S15. First, apply principal component analysis (PCA) for preliminary dimensionality reduction. Calculate the feature covariance matrix, perform eigenvalue decomposition, and select the eigenvectors corresponding to the first k largest eigenvalues ​​to construct a projection matrix. Use the projection matrix to perform linear dimensionality reduction on the fused features to obtain PCA dimensionality reduction features. Then, use the t-distributed random neighbor embedding (t-SNE) algorithm for nonlinear dimensionality reduction. Calculate the high-dimensional space distance matrix of the PCA dimensionality reduction features and construct the conditional probability distribution. Initialize the data points in the low-dimensional space, optimize the Kullback-Leibler divergence by gradient descent, and obtain the t-SNE dimensionality reduction features. Next, apply the autoencoder for feature compression. Construct a symmetric multi-layer autoencoder network, including an encoder and a decoder. Use the encoder to map the t-SNE dimensionality reduction features to a lower-dimensional latent space to obtain compressed features. Finally, use Huffman coding to losslessly compress the compressed features. Calculate the probability distribution of each dimension of the compressed features, construct a Huffman tree, and assign binary codes to each eigenvalue. The generated Huffman code is used to encode the compressed features to obtain the final compressed data. The compressed data and the Huffman coding dictionary are output together for use in subsequent steps.

[0124] According to one aspect of the present application, steps S21 to S26 are specifically:

[0125] Step S211: Read the processed data D_processed and feature index I_feature from the time series database. Use the index I_feature to locate and extract the visual feature K_v, audio feature M_a, thermal feature F_h, motion feature M_v, and environmental feature E_main. Verify the integrity of the extracted features to ensure that all necessary features have been correctly acquired.

[0126] Step S212: Calculate the statistical properties of each modal feature (K_v, M_a, F_h, M_v, E_main). Calculate the mean μ = (1 / N) * Σ x_i, where N is the number of samples. Calculate the variance σ^2 = (1 / N) * Σ (x_i - μ) 2 . Calculate the skewness S = (1 / N) * Σ ((x_i - μ) / σ) 3 Calculate the kurtosis K = (1 / N) * Σ ((x_i - μ) / σ) 4 -3.

[0127] Step S213: Determine an enhancement parameter α for each feature based on the calculated statistical properties. Define an enhancement parameter calculation function f(μ, σ 2, S, K) = α_base * (1 + w_1 * |S| + w_2 * |K - 3|), where α_base is the base enhancement parameter, w_1 and w_2 are weight coefficients. Calculate α = f(μ, σ 2 , S, K), ensuring that α is within the predefined range.

[0128] By extracting multimodal features from the time series database and calculating their statistical properties, a comprehensive understanding of the data characteristics is achieved. The method of dynamically determining enhancement parameters based on statistical properties enables the feature enhancement process to be adaptively adjusted to adapt to the characteristics of different data. This data-driven adaptive enhancement strategy significantly improves the quality and discrimination of features, especially when dealing with diverse and dynamically changing data. By considering high-order statistics such as mean, variance, skewness, and kurtosis, the system can more comprehensively capture the distribution characteristics of the data, thereby making more precise adjustments in subsequent feature processing. This method not only improves the effect of feature enhancement, but also enhances the system's ability to handle abnormal data and edge cases. Especially in complex access control environments, such as changes in lighting and posture, this adaptive enhancement method can effectively improve the robustness of features and provide more reliable feature representation for subsequent identity recognition and verification.

[0129] Step S221: Calculate the spatial attention weight W_s for the visual feature K_v. Use a convolutional neural network to extract the spatial feature map F. Calculate the average pooling and maximum pooling results of F, and concatenate them through a 1x1 convolutional layer to obtain the spatial attention map A_s. Apply the sigmoid activation function to obtain the final spatial attention weight W_s = sigmoid(A_s).

[0130] Step S222, calculate the channel attention weight W_c of the visual feature K_v. Perform global average pooling and global maximum pooling on K_v to obtain two channel descriptors. Input these two descriptors into the shared multi-layer perceptron respectively to obtain two channel attention vectors. After adding these two vectors, apply the sigmoid activation function to obtain the channel attention weight W_c.

[0131] Step S223: multiply the visual feature K_v by the spatial attention weight W_s and the channel attention weight W_c. K_v_enhanced = K_v * W_s * W_c, where * represents element-by-element multiplication. In this way, the enhanced visual feature K_v_enhanced is obtained.

[0132] Step S224: Calculate the short-time Fourier transform (STFT) of the audio feature M_a to obtain the time-frequency spectrum S(t,f). Use overlapping Hanning windows with a window size of 25ms and a step size of 10ms. S(t,f) = STFT(M_a).

[0133] Step S225, estimate the noise power spectrum N(f). Use the minimum statistic method to estimate the noise level in the speech inactive interval. For each frequency bin, track the minimum value over multiple time frames and use these minimum values ​​to estimate the noise power spectrum.

[0134] Step S226: Use spectral subtraction to enhance the time-frequency spectrum S(t,f). S_enhanced(t,f) = max(|S(t,f)| 2 - α * N(f), β * |S(t,f)| 2 ), where α is the over-subtraction factor and β is the spectrum lower limit. The enhanced power spectrum is taken square root and the enhanced time-frequency spectrum is reconstructed in combination with the original phase information.

[0135] Step S227: Extract Mel frequency cepstral coefficient (MFCC) features from the enhanced time-frequency spectrum S_enhanced(t,f). Apply the Mel filter bank to calculate the logarithmic Mel power spectrum. Perform discrete cosine transform on the logarithmic Mel power spectrum and take the first 13 coefficients as MFCC features. Calculate the first and second order differences of MFCC and combine them to obtain the enhanced audio feature M_a_enhanced.

[0136] Step S228: Apply adaptive histogram equalization to the thermal feature F_h. Divide F_h into multiple overlapping blocks. Calculate the cumulative distribution function (CDF) for each block. Use bilinear interpolation to fuse the CDFs of adjacent blocks to obtain a mapping function for each pixel. Apply the mapping function to transform F_h to obtain an enhanced thermal feature F_h_enhanced.

[0137] Step S229, use wavelet threshold to denoise the motion feature M_v. Perform discrete wavelet transform on M_v to obtain wavelet coefficients. Use SURE (Stein's Unbiased Risk Estimate) method to adaptively select threshold λ. Apply soft threshold function to wavelet coefficients: if |coef|>λ, then sign(coef)(|coef| - λ), else 0. Perform inverse wavelet transform to obtain the denoised motion feature M_v_enhanced.

[0138] Step S2210: Apply exponential weighted moving average (EWMA) to the environmental feature E_main. Calculate the smoothing coefficient α = 2 / (N + 1), where N is the selected time window size. For each time series x_t in E_main, calculate EWMA: y_t = α * x_t + (1 - α) * y_{t-1}. Obtain the smoothed environmental feature E_main_smoothed.

[0139] Through targeted enhancement of visual, audio, thermal, motion and environmental features, the quality and information content of each modal data are greatly improved. The spatial and channel attention mechanism of visual features effectively highlights the key areas and important feature channels in the image and improves the expressiveness of the features. The spectral subtraction and MFCC extraction of audio features effectively remove background noise and enhance the recognizability of speech signals. The adaptive histogram equalization of thermal features improves the contrast and detail performance of thermal maps. The wavelet threshold denoising of motion features effectively suppresses noise interference and retains key motion information. The exponentially weighted moving average processing of environmental features smoothes the fluctuations of environmental parameters and provides more stable environmental context information. This multi-modal and multi-angle feature enhancement method not only improves the quality of each modal data, but also enhances the system's adaptability to complex environments. Especially when dealing with various interferences and changes in actual access control scenarios, such as uneven lighting, background noise, temperature fluctuations, etc., this comprehensive feature enhancement strategy can effectively extract key information and provide high-quality input data for subsequent feature fusion and identity recognition.

[0140] Step S231: Combine the enhanced modal features (K_v_enhanced, M_a_enhanced, F_h_enhanced, M_v_enhanced, E_main_smoothed) in step S22 with the pre-stored identity label Y to form a feature-label pair set F = {(f_i, y_i)}.

[0141] Step S232: Calculate the mutual information I(X;Y) between each feature and the identity label. Use the k-nearest neighbor method to estimate the mutual information: I(X;Y) ≈ ψ(k) + 1 / N * Σ[ψ(nx) + ψ(ny)] - ψ(N), where ψ is the double gamma function, nx and ny are the number of k-th nearest neighbors in the X and Y spaces, respectively, and N is the number of samples.

[0142] Step S233: Set a mutual information threshold T_I. Compare the mutual information value of each feature with T_I, select features that satisfy I(X;Y)>T_I, and form a feature subset F_selected.

[0143] By calculating the mutual information between features and identity tags, effective feature selection is achieved. This feature selection method based on information theory can identify the feature subset most relevant to identity recognition, greatly reducing the dimension of the feature space while retaining the most recognizable information. By setting the mutual information threshold, the system can flexibly balance the number of features and recognition performance to adapt to different application requirements. This method not only improves the efficiency of subsequent processing, but also effectively reduces the risk of overfitting and enhances the generalization ability of the model. Especially when processing high-dimensional and multimodal access control data, this feature selection method can effectively remove redundant and irrelevant features and extract the most representative feature subset. This not only greatly reduces the computational complexity, but also improves the accuracy of recognition. In practical applications, this method can effectively cope with feature changes under different users and different environmental conditions, automatically select the most recognizable feature combination, and improve the adaptability and robustness of the system. Through this intelligent feature selection strategy, the system can greatly improve processing efficiency while ensuring recognition performance, making real-time identity authentication possible.

[0144] Step S241: Construct a feature dictionary D and a sparse coding matrix A based on F_selected. Initialize D as a random orthogonal matrix and A as a zero matrix. Define the objective function J(D,A) = ||F_selected - DA||2 2 + λ||A||1, where λ is the regularization parameter.

[0145] Step S242: Use the alternating direction multiplier method (ADMM) to solve the optimization problem. Convert the problem into augmented Lagrangian form: L(D,A,Z,Y) = ||F_selected - DA||2 2 + λ||Z||_1 + Y T (AZ) + (ρ / 2)||AZ||2 2 .

[0146] Step S243: Update D, A, Z and Y alternately. Fix other variables and update D: D = F_selected * A T * (A* A T ) -1 . Update A: A = (D T * D + ρI) -1 * (D T * F_selected + ρZ - Y). Update Z: Z = S_λ / ρ(A +Y / ρ), where S_α is the soft threshold operator. Update Y: Y = Y + ρ(A - Z).

[0147] Step S244: Repeat step S243 until convergence or the maximum number of iterations is reached. Obtain the final feature dictionary D and sparse coding matrix A. Calculate the sparse representation of the feature F_sparse = D * A.

[0148] By constructing a feature dictionary and a sparse coding matrix, the high-dimensional features are effectively represented and compressed. The use of the alternating direction multiplier method (ADMM) to solve the optimization problem not only ensures the efficiency of the solution, but also can handle large-scale, non-smooth optimization problems. This method based on sparse representation can capture the intrinsic structure of the data and effectively reduce the redundancy of the features. Especially when dealing with high-dimensional, multimodal access control data, sparse representation can provide a more compact and effective feature representation method. By adjusting the regularization parameters, the system can flexibly balance the feature reconstruction error and sparsity to adapt to different application requirements. This method not only greatly reduces the storage and computing requirements, but also improves the recognition and robustness of the features. In practical applications, this sparse representation method can effectively handle noise and outliers and improve the system's adaptability to complex environments. Through the learned feature dictionary, the system can better understand and represent the features of different users and scenarios, providing a more reliable and efficient feature representation for subsequent identity recognition.

[0149] Step S251: construct a k-nearest neighbor graph G = (V, E) based on F_sparse. Use the KD tree algorithm to efficiently find the k nearest neighbors of each data point. For each pair of points that are k-nearest neighbors to each other, add an edge to G.

[0150] Step S252: Calculate the Laplacian matrix L of the graph G. First, construct the adjacency matrix W, W ij = 1 if nodes i and j are connected, otherwise 0. Calculate the degree matrix D, D ii = Σ j W ij . The Laplace matrix L = D - W.

[0151] Step S253: Solve the generalized eigenvalue problem Lf = λDf. Use the Lanczos algorithm to efficiently calculate the first m smallest non-zero generalized eigenvalues ​​of L and D and their corresponding eigenvectors.

[0152] Step S254: Select the eigenvectors corresponding to the smallest m non-zero eigenvalues ​​to form a projection matrix P = [f_1, f_2, ..., f_m]. The selection of m is based on the cumulative explained variance ratio, so that Σ i=1 m λ_i / Σ i=1 n λ i ≥threshold, where threshold is the preset threshold.

[0153] Step S255, project F_sparse onto the projection matrix P to obtain the reduced-dimensional feature F_reduced = F_sparse * P.

[0154] By constructing a k-nearest neighbor graph of features and applying Laplace matrix analysis, the intrinsic structure of the data is effectively captured. This graph-based nonlinear dimensionality reduction method can retain the local structure and global relationship of the data, overcoming the limitations of traditional linear dimensionality reduction methods. By solving the generalized eigenvalue problem, the system can find the optimal low-dimensional embedding, effectively reducing the feature dimension while retaining the key information of the data. Especially when processing high-dimensional, nonlinear access control data, this method can better capture the intrinsic structure and relationship of the data. By dynamically selecting the embedding dimension, the system can flexibly balance information retention and dimensionality reduction to adapt to different application requirements. This dimensionality reduction method not only greatly reduces the computational complexity of subsequent processing, but also improves the recognition and generalization ability of features. In practical applications, this graph-based dimensionality reduction method can effectively process different types of features, such as a mixture of continuous and discrete features, and improves the system's ability to process complex data. By retaining the intrinsic structure of the data, the system can better identify and distinguish different users and access patterns, providing a more accurate and reliable feature representation for intelligent access control management.

[0155] Step S261: Organize F_reduced into a high-order tensor T. The order of the tensor and the size of each dimension are determined according to the natural structure of the data (such as time, space, channel, etc.). For example, it can be organized into a fourth-order tensor T ∈ R^{I×J×K×L}, where I, J, K, and L represent the number of samples, time step, spatial position, and feature channel, respectively.

[0156] Step S262: Calculate the correlation matrix R between the modes of each order of the tensor T. For each pair of modes (i, j), calculate the matrix T_(i) and T_(j) of mode i and mode j. Calculate their correlation coefficient matrix C ij = corr(T i , T j ). ij C ij The average value of .

[0157] Step S263: Determine the rank vector r = [r1, r2, ..., r n]. Use heuristic methods such as r_i = round(rank_ratio * size(T, i) * (1 - avg(R_i))), where rank_ratio is the preset rank scaling factor and R_i is the average correlation between mode i and other modes.

[0158] Step S264: Apply the tensor train (TT) decomposition algorithm to reshape T into a high-order tensor T' ∈ R I _1×I_2×...×I_d Initialize TT cores G_1, G_2, ..., G_d, each G_k has a size of r k-1 × I_k × r_k.

[0159] Step S265, use the alternating least squares (ALS) method to optimize the TT core. Fix all cores except G_k, convert the problem into a matrix form, and use SVD to solve the optimal G_k. Optimize each core repeatedly until convergence or the maximum number of iterations is reached.

[0160] Step S266: Reconstruct the tensor fusion feature T_fused through the optimized TT cores G_1, G_2, ..., G_d. Calculate T_fused = G1×1G2×2... × d-1 G d , where ×_i represents the i-module tensor product.

[0161] Step S267, evaluate the fusion effect. Calculate the relative reconstruction error between the original tensor T and the fused tensor T_fused. Use Tucker decomposition to calculate the multilinear rank of T_fused and compare it with the original tensor T to evaluate the information compression effect. Reshape T_fused back to the original data form as the final fusion feature output, i.e., tensor fusion feature, for use in subsequent processing steps.

[0162] By organizing the reduced-dimensional features into high-order tensors and applying the tensor train decomposition algorithm, the deep fusion of multimodal features is achieved. This tensor-based method can effectively capture the high-order correlations between different modal features and retain the multidimensional structural information of the data. By calculating the mode correlation matrix and dynamically adjusting the decomposition rank, the system can adaptively balance the computational complexity and information retention to adapt to different data characteristics. The application of tensor train decomposition not only greatly reduces the number of parameters, but also effectively processes high-dimensional data and overcomes the dimensionality curse problem. Especially when processing multi-source heterogeneous access control data, this method can better fuse information from different modalities and extract more expressive features. By optimizing the TT core by alternating least squares, the system can efficiently learn the potential representation of the data. This deep fusion method not only improves the expressiveness of the features, but also enhances the system's robustness to missing or noisy data in different modalities. In practical applications, this tensor-based fusion method can effectively handle complex multimodal interactions, such as joint analysis of vision, audio, and behavior, providing a more comprehensive and accurate identity feature representation. Through this efficient feature fusion strategy, the system can significantly improve processing efficiency while ensuring recognition accuracy, providing technical support for real-time multimodal identity authentication.

[0163] According to another aspect of the present application, steps S21 to S24 may also be:

[0164] S21. Obtain the processed data and updated feature index stored in step S16 from the database. Apply contrast-limited adaptive histogram equalization (CLAHE) to visual features. Divide the feature map into multiple small blocks, calculate the histogram for each small block and perform equalization, and limit the contrast to prevent noise amplification. Perform spectral subtraction enhancement on audio features. Estimate the noise power spectrum, subtract the weighted noise power spectrum from the original speech power spectrum, and reconstruct the enhanced speech signal. Apply adaptive median filtering to thermal features. Use a median filter with a variable window size to adaptively adjust the window size according to the local noise level to remove salt and pepper noise while retaining edge information. Perform wavelet threshold denoising on motion features. Perform discrete wavelet transform on the features, apply a soft threshold function in the wavelet domain to suppress the noise coefficient, and perform inverse transform to reconstruct the denoised features. Apply exponentially weighted moving average (EWMA) smoothing to environmental features. Calculate the smoothing factor based on historical data, perform exponentially weighted smoothing on time series data, and reduce the impact of short-term fluctuations. Combine the enhanced features of each modality to form an enhanced feature set, which is output for use in the next step.

[0165] S22, receiving the enhanced feature set outputted by step S21. First, calculate the mutual information value between each feature and the target variable. Use the histogram method to estimate the joint probability distribution and the marginal probability distribution, and calculate the mutual information. Sort the features according to the calculated mutual information value. Then, apply the minimum redundancy maximum correlation (mRMR) algorithm for feature selection. Iteratively select features to maximize the correlation between the features and the target while minimizing the redundancy between the selected features. Calculate the redundancy between the candidate features and the selected feature set, and select the features with the largest difference (correlation minus redundancy). Next, use the stability selection method to improve the robustness of feature selection. Subsample the data multiple times, and apply Lasso regression on each subsample. Count the frequency of occurrence of each feature in multiple selections, and select features whose frequency exceeds the threshold. Finally, apply the recursive feature elimination (RFE) algorithm for final screening. Train the support vector machine classifier and calculate the feature importance score. Iteratively remove the least important features until the predetermined number of features is reached. Output the selected feature subset for use in subsequent steps.

[0166] S23. Obtain the selected feature subset from step S22. First, apply the t-SNE algorithm for preliminary dimensionality reduction. Calculate the conditional probability distribution of data points in high-dimensional space and initialize data points in low-dimensional space. Use gradient descent to minimize KL divergence and optimize low-dimensional embedding. Then, use the UMAP (Uniform Manifold Approximation and Projection) algorithm for further dimensionality reduction. Construct a k-nearest neighbor graph of the data and calculate the fuzzy topological representation. Optimize the low-dimensional embedding to maintain local and global structures. Next, apply the Isomap algorithm to maintain the geodesic distance of the data. Construct a neighbor graph of data points and calculate the shortest path distance between all pairs of points. Perform eigenvalue decomposition on the geodesic distance matrix and select the first few eigenvectors as low-dimensional embeddings. Then, use the LLE (Locally Linear Embedding) algorithm to maintain the local linear structure. Find k nearest neighbors for each data point and calculate the reconstruction weight. Minimize the reconstruction error in low-dimensional space and obtain the embedding coordinates. Finally, apply the Multi-View Spectral Embedding (MSE) algorithm to integrate multiple dimensionality reduction results. The results of different dimensionality reduction methods are treated as multiple views, and the affinity matrix of each view is constructed. The optimization results in a shared low-dimensional embedding that takes into account the information of all views. The final low-dimensional embedding result is output for use in subsequent processing steps.

[0167] S24, receiving the low-dimensional embedding result output by step S23. First, apply CP decomposition (CANDECOMP / PARAFAC decomposition). Organize the data into a third-order tensor and select an appropriate rank R. Use alternating least squares to iteratively optimize and obtain a factor matrix. Calculate the outer product of the factor matrix and reconstruct the original data. Then, use Tucker decomposition for high-order feature extraction. Determine the dimension of the core tensor and initialize the core tensor and factor matrix. Apply high-order singular value decomposition (HOSVD) to obtain the initial solution, and then use alternating least squares to optimize the decomposition result. Next, apply Tensor Train to decompose and process high-dimensional data. Reshape the original tensor into a high-order tensor and specify the TT rank. Use SVD to recursively calculate the TT core to obtain a series of low-rank matrices. Then, use hierarchical Tucker decomposition to process large-scale data. Divide the data into multiple sub-tensors and apply Tucker decomposition to each sub-tensor. Recursively perform Tucker decomposition on the obtained core tensor to form a hierarchical representation. Finally, apply tensor network decomposition to capture complex feature interactions. Design a tensor network structure, including multiple tensor nodes and connecting edges. Use an alternating optimization algorithm to solve the tensor of each node and obtain a compact network representation. Combine the results of all decomposition methods to form a multi-view tensor representation and output it for use in the next step.

[0168] According to one aspect of the present application, step S33 is specifically:

[0169] Step S331: retrieve historical verification records H = {(x_i, y_i)} from the verification history database, where x_i is the verification feature vector and y_i is the true label (1 indicates a valid user, 0 indicates an invalid user). Randomly divide H into a training set H_train and a test set H_test, with a ratio of 8:2.

[0170] Step S332: Train a binary classification model M (such as logistic regression or support vector machine) on H_train. Use k-fold cross validation to optimize model hyperparameters. For each sample x_i in H_test, use M to predict the probability p_i that it is a legitimate user.

[0171] Step S333: Set the threshold range [τ_min, τ_max]. For the threshold τ ∈ [τ_min, τ_max], calculate the global false alarm rate FAR(τ) = (Σ I(p_i ≥ τ and y_i = 0)) / (Σ I(y_i = 0)) and the false alarm rate FRR(τ) = (Σ I(p_i< τ and y_i = 1)) / (Σ I(y_i = 1)), where I is the indicator function.

[0172] Step S334: Use adaptive binary search to search for the optimal decision threshold τ*. Initialize the left boundary l = τ_min, and the right boundary r = τ_max. While r - l> ε (ε is the preset accuracy):

[0173] Calculate the midpoint m = (l + r) / 2;

[0174] Calculate FAR(m) and FRR(m);

[0175] If |FAR(m) - FRR(m)|< δ (δ is the preset tolerance), then τ* = m and the search ends;

[0176] Otherwise, if FAR(m)>FRR(m), then l = m, otherwise r = m;

[0177] Repeat until τ* is found or the maximum number of iterations is reached.

[0178] Step S335: Get the feature vector f_current of the current verification sample. Calculate the cosine similarity between f_current and each template in the stored identity template set T = {t_1, t_2, ..., t_n}. s_cos_i = (f_current· t_i) / (||f_current|| * ||t_i||). Select the maximum cosine similarity s_cos_best = max(s_cos_i).

[0179] Step S336: retrieve the historical behavior sequence B = {b_1, b_2, ..., b_m} related to the current identity ID from the user behavior database. Use the hidden Markov model (HMM) λ = (A, B, π) to model B, where A is the state transition matrix, B is the observation probability matrix, and π is the initial state distribution.

[0180] Step S337: Estimate HMM parameter λ using the Baum-Welch algorithm. Obtain the observed behavior sequence O ={o_1, o_2, ..., o_k} of the current user. Use the forward algorithm to calculate the likelihood probability P(O|λ) of O under the model λ.

[0181] Step S338: retrieve the current environment parameter E_current and the historical environment-user association data D_eu = {(E_i, U_i)} from the environment database. Construct a Bayesian network model BN to represent the conditional dependency relationship between the environment, user identity and time.

[0182] Step S339: Use maximum likelihood estimation or Bayesian estimation to learn the parameters of BN. Calculate the prior probability P(U) of the user appearing, the conditional probability P(E|U) of the environmental parameters, and the conditional probability P(T|U,E) of time. Calculate the posterior probability using Bayesian theorem: P(U|E,T) = P(E|U) * P(T|U,E) * P(U) / P(E,T).

[0183] Step S3310: Calculate the confidence score C = f(s_cos_best, P(O|λ), P(U|E,T)) of the current verification result. Define f as a weighted sum function: C = w1 * s_cos_best + w2 * log(P(O|λ)) + w3 * P(U|E,T), where w1, w2, and w3 are weight coefficients, which are determined by verification performance optimization.

[0184] Step S3311: Compare the calculated confidence score C with the optimal decision threshold τ in step S334. If C ≥ τ, the preliminary determination verification is passed; otherwise, the preliminary determination verification fails.

[0185] Step S3312: Construct a decision tree model D. Use historical validation data to train the decision tree, with features including biometric matching results (such as s_cos_best), behavior consistency scores (such as P(O|λ)), and environmental context probabilities (such as P(U|E,T)). Use the information gain ratio criterion to select the optimal split feature, and use pessimistic pruning to prevent overfitting.

[0186] Step S3313: Input various indicators of the current verification case (s_cos_best, P(O|λ), P(U|E,T)) into the decision tree D. Traverse the decision tree until the leaf node to obtain the final verification result (pass / fail) and verification level L. The verification level L can be a discrete (such as high, medium, low) or continuous confidence score.

[0187] Step S3314: Evaluate the performance of the decision tree model. Use k-fold cross validation to calculate the accuracy, precision, recall, and F1 score. Draw the ROC curve and PR curve, and calculate the AUC value. If the performance does not meet the preset standards, consider using an ensemble method (such as random forest or gradient boosted decision tree) to improve the model performance.

[0188] Step S3315: Implement an online update mechanism for the decision tree model. Maintain a sliding window W containing the most recent N verification results. When the number of samples in W reaches a threshold, use the data in W to update the decision tree model. Use an incremental learning algorithm, such as the Hoeffding tree, to achieve efficient online updates.

[0189] Step S3316: Generate a verification report R, which includes the following contents: verification result (pass / fail), verification level L, confidence score C, detailed values ​​of various indicators (s_cos_best, P(O|λ), P(U|E,T)), decision path analysis and recommended follow-up operations. R is stored in the verification result database and transmitted to the relevant system components for subsequent processing.

[0190] By comprehensively applying machine learning, probability models, and decision theory, a highly intelligent and personalized identity authentication process is achieved. First, by analyzing historical verification data to train a binary classification model and using adaptive binary search for the optimal decision threshold, the system is able to dynamically adjust the verification criteria and balance security and usability. This approach not only improves the accuracy of verification, but also adapts to the needs of different scenarios and user groups. Secondly, by calculating the cosine similarity between feature vectors and stored templates, analyzing user behavior sequences in combination with hidden Markov models, and evaluating environmental contexts using Bayesian network models, the system achieves multi-factor, multi-dimensional identity authentication. This comprehensive verification method greatly improves the security of the system and effectively prevents identity fraud and deception. Especially in complex and changeable actual access control environments, this method can effectively deal with various abnormal situations, such as unusual access patterns or environmental changes. By building and updating the decision tree model, the system is able to make the final verification decision based on multiple factors and provide detailed verification level and confidence evaluation. This not only improves the transparency and explainability of verification, but also provides valuable decision support information for administrators. Finally, by implementing an online update mechanism, the system is able to continuously learn and adapt to new verification modes and maintain long-term high performance. This adaptive, multi-dimensional verification method greatly improves the security, flexibility and user experience of the smart access control system, and provides strong technical support for various scenarios with high security requirements.

[0191] According to another aspect of the present application, step S33 may also be:

[0192] S33. Retrieve historical behavior sequences associated with identity IDs from the user behavior database. First, apply the hidden Markov model (HMM) to model user behavior. Define the observed state and hidden state, and use the Baum-Welch algorithm to estimate the model parameters. Use the Viterbi algorithm to infer the most likely state sequence and calculate the likelihood probability of the behavior sequence. Then, use the long short-term memory (LSTM) network to capture long-term dependencies. Encode the behavior sequence into a vector and input it into the LSTM network. Train the network to predict the next behavior, using the cross entropy loss function. Calculate the perplexity of the test sequence as the anomaly score. Next, apply the isolation forest algorithm to detect abnormal behavior. Construct multiple isolated trees and calculate the average path length of the behavior. Convert the average path length to anomaly scores to identify potential abnormal behaviors. Then, use a high-order Markov chain model to analyze the behavior pattern. Consider longer historical behavior sequences and construct a high-order transition probability matrix. Use the sliding window method to calculate the probability of the current behavior sequence and detect low-probability events. Finally, apply the dynamic time warping (DTW) algorithm to compare the behavior sequences. Align the current behavior sequence with the historical typical sequence and calculate the warping distance. The abnormal degree of behavior is determined based on the regular distance. The results of multiple methods are combined to generate a behavior anomaly detection report, which is output for use in subsequent steps.

[0193] In another embodiment of the present application, it also includes S35, receiving the abnormal behavior detection report of step S33 and other verification results. First, the adaptive boosting (AdaBoost) algorithm is applied to integrate multiple weak classifiers. Using a decision tree as a base classifier, multiple classifiers are trained on different feature subsets. The sample weights are dynamically adjusted according to the performance of each classifier to obtain the final strong classifier. Then, the receiver operating characteristic (ROC) curve analysis is used to determine the initial threshold. The true positive rate and false positive rate under different thresholds are calculated, and the ROC curve is drawn. The point that maximizes Youden's Index is selected as the initial threshold. Next, the adaptive weighted moving average (AWMA) method is applied to dynamically adjust the threshold. The weighted moving average is calculated based on the historical verification results, and the threshold is dynamically adjusted to adapt to the changes in the system state. Then, the Bayesian optimization method is used to fine-tune the threshold. Define the objective function (such as F1 score) and search for the optimal threshold within a given range. Use Gaussian process regression to establish a probability model of the objective function, and find the optimal threshold through iterative optimization. Finally, a multi-objective optimization algorithm is applied to balance multiple performance indicators. Consider multiple objectives such as false positive rate, false negative rate, and system response time. The NSGA-II algorithm is used to obtain the Pareto optimal solution set, from which the best threshold configuration is selected. The optimized decision threshold and its adaptation strategy are output for use in the subsequent verification process.

[0194] According to another aspect of the present application, steps S42 to S45 may also be:

[0195] S42. Retrieve historical passenger flow data from the passenger flow database. First, apply the autoregressive integrated moving average (ARIMA) model for basic forecasting. Determine the difference order to make the time series stationary and estimate the autoregressive and moving average parameters. Use maximum likelihood estimation to fit the model and generate short-term passenger flow forecasts. Then, use long short-term memory (LSTM) networks to capture long-term patterns. Convert the time series into a supervised learning problem and build a multi-layer LSTM network. Use historical data to train the network and predict the passenger flow for multiple time steps in the future. Next, apply the Prophet model to handle seasonal and holiday effects. Decompose the time series into trend, seasonal, and holiday components. Use the Bayesian method to fit the model parameters and generate forecasts that take into account the impact of special events. Then, use the wavelet neural network (WNN) model to handle multi-scale features. Perform wavelet decomposition on the original time series to obtain different frequency components. Build a neural network model for each frequency component and combine the prediction results of each frequency to obtain the final forecast. Next, apply Gaussian process regression (GPR) to quantify forecast uncertainty. Define the kernel function to capture the correlation structure of the time series and use maximum marginal likelihood to estimate the kernel parameters. Generate a prediction distribution, provide confidence intervals and uncertainty estimates. Finally, use ensemble learning methods to fuse multiple prediction results. Give dynamic weights to each model prediction, and the weights are dynamically adjusted based on the historical performance of the model. Use weighted average or stacked generalization methods to get the final crowd flow prediction results. Output the prediction results and their uncertainty estimates for use in subsequent steps.

[0196] S43, receiving the crowd flow prediction results and real-time crowd flow distribution data of step S42. First, the density clustering algorithm DBSCAN is applied to identify high-density areas. The density threshold and the minimum number of samples are set to cluster the spatial points. The high-density areas obtained by clustering are marked as potential congestion points. Then, the crowd movement is analyzed using the optical flow method. The pixel displacement field between consecutive frames is calculated to estimate the movement direction and speed of the crowd. The area where the movement speed is significantly reduced is detected as the basis for congestion judgment. Next, the social force model is applied to simulate crowd dynamics. The expected speed and direction are defined for each individual, considering the interaction force between individuals and the influence of environmental obstacles. The crowd distribution in the short term is predicted by numerical integration to identify the congestion area that may be formed. Then, the graph convolutional neural network (GCN) is used to model spatial correlation. The scene is represented as a graph structure, the node represents the area, and the edge represents the adjacent relationship. Use GCN to learn the mutual influence between regions and predict the propagation trend of congestion. Finally, the fuzzy logic system is applied to define the scene-specific congestion criteria. Fuzzy rule sets are defined for different types of scenes based on expert knowledge and historical data. Input real-time crowd density, speed and forecast data, and use fuzzy reasoning to obtain a qualitative description of the degree of congestion. Combine the results of multiple methods to generate a congestion detection report, including the location, degree and forecast trend of congestion, and output it for subsequent steps.

[0197] S44. Obtain real-time data from multiple data sources, including the congestion detection report, video surveillance data, sensor data, and system logs in step S43. First, apply a convolutional long short-term memory network (ConvLSTM) to process the video sequence. Input the video frame into the ConvLSTM network to capture spatiotemporal features. Train the network to detect abnormal events, such as sudden crowd gatherings or abnormal behavior patterns. Then, use a variational autoencoder (VAE) to build a normal behavior model. Train the VAE with normal data to learn the potential representation of the data. Calculate the reconstruction error of the real-time data, and treat events with high reconstruction errors as potential anomalies. Next, apply ensemble empirical mode decomposition (EEMD) to analyze the sensor signal. Decompose the signal into a series of intrinsic mode functions (IMFs) and analyze the statistical characteristics of each IMF. Detect IMFs that deviate from the normal range and identify abnormal patterns. Then, use the random forest algorithm to integrate multi-source data for anomaly detection. Construct a decision tree for each data source and combine the prediction results of multiple trees. Calculate the anomaly score and determine the global anomaly threshold. Finally, apply a dynamic Bayesian network model to capture the temporal dependency of events. Construct a Bayesian network to represent the system state and observed variables, and use the particle filter algorithm for online inference. Detect event sequences that violate conditional probability relationships and identify complex abnormal patterns. Combine the results of multiple methods to generate a real-time abnormal event detection report, including the type, location, severity, and possible cause of the abnormality, and output it for subsequent steps.

[0198] According to another aspect of the present application, there is also provided an intelligent access control management system based on multimodal recognition and Internet of Things technology, including:

[0199] at least one processor; and,

[0200] a memory communicatively connected to at least one of the processors; wherein,

[0201] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent access control management method based on multimodal recognition and Internet of Things technology described in any of the above technical solutions.

[0202] In another embodiment of the present application, specifically:

[0203] Step S1, collect multimodal data through distributed IoT devices, synchronize and align the collected data, and then perform adaptive preprocessing; extract multi-scale features from the preprocessed data, and dynamically fuse the extracted features; compress and preliminarily reduce the dimension of the fused features and preprocessed raw data on the edge device, and transmit the compressed and reduced-dimensional data to the central processing unit; perform stream processing on the data in the central processing unit, and finally store the processed data in the time series database and update the index;

[0204] S11. Collect data through at least two IoT devices, including: collect multi-angle visual data through a high-speed camera array to obtain image sequence I={I_1, I_2, ..., I_n} and depth information Z={Z_1, Z_2, ..., Z_n}. Use a microphone array to collect spatialized audio signals A(t,θ,φ). Use a thermal imaging camera to collect heat map sequences H={H_1,H_2, ..., H_m}. Collect human motion features V(t)={V_1(t), V_2(t), ..., V_k(t)} through millimeter wave radar. Use a sensor network to collect environmental parameters P={T, H, L, CO2, PM2.5}, where T is temperature, H is humidity, L is light, CO2 is carbon dioxide concentration, and PM2.5 is fine particulate matter index. Use infrared array sensors and pressure sensors to collect the number of people entering and leaving N_in(t) and N_out(t).

[0205] S12. Use the Precision Time Protocol (PTP) server to synchronize the clocks of all devices collected in S11, add a microsecond timestamp t_i to each data point, and obtain a timestamped data set D_t={I_t, Z_t, A_t, H_t, V_t, P_t, N_in_t, N_out_t}. Calculate the complexity C_i of each modal data in D_t, and dynamically adjust the sliding window size w_i according to C_i. Apply the dynamic time warping algorithm to the data of each modality and calculate the alignment matrix M_ij between different modalities. Resample and interpolate D_t according to M_ij to obtain the aligned multimodal time series D_aligned(t).

[0206] S13. For the image sequence I_t and depth map Z_t in D_aligned(t) in S12, calculate the local complexity C_local. Adaptively adjust the contrast limit parameter α_i according to C_local, apply contrast-limited adaptive histogram equalization, and obtain enhanced I_enhanced and Z_enhanced. Calculate the optical flow field F in I_enhanced, estimate the motion blur kernel K according to F, use Wiener filtering for deblurring, and obtain I_deblurred. For the audio signal A_t(t,θ,φ), estimate the signal-to-noise ratio SNR(t,θ,φ), dynamically adjust the beamforming weights w(t,θ,φ) according to the SNR, and apply the beamforming technology to obtain the enhanced audio signal A_enhanced. For the heat map sequence H_t, construct a residual network R, train R to minimize the reconstruction error, and apply R for super-resolution reconstruction to obtain H_sr. For the millimeter-wave radar signal V_t(t), the signal dynamic characteristics D(t) are calculated, and the Kalman filter parameters Q(t) and R(t) are adaptively adjusted according to D(t), and the smooth signal V_smooth is obtained by filtering. For the environmental parameter P_t, the local anomaly score S_i of each parameter is calculated, and the outliers are identified and corrected according to S_i to obtain the corrected environmental parameter P_corrected.

[0207] S14. For I_deblurred and Z_enhanced in S13, calculate the image gradient G and local directional distribution H. Adaptively adjust the scale parameter σ and directional parameter θ of the SIFT algorithm according to G and H, and extract the visual feature K_v. For A_enhanced, calculate the short-time spectrum S(t,f), dynamically adjust the center frequency f_c and bandwidth B_w of the Mel filter bank according to the time-varying characteristics of S(t,f), and extract the dynamic Mel frequency cepstrum coefficient M_a. For H_sr, apply empirical mode decomposition to obtain the intrinsic mode function IMF_i, perform Hilbert transform on each IMF_i, calculate the instantaneous frequency IF_i and instantaneous amplitude IA_i, and extract the thermal feature F_h={IF_i, IA_i}. For V_smooth, calculate the power spectral density PSD(f) and the autocorrelation function ACF(τ), and extract the motion feature M_v={PSD, ACF}. For P_corrected, the mutual information matrix I_m between parameters is calculated, and the separation matrix W of the ICA algorithm is dynamically adjusted according to I_m to extract the main environmental features E_main.

[0208] S15. Organize the features K_v, M_a, F_h, M_v and E_main extracted in S14 into a fifth-order tensor T. Calculate the pattern correlation matrix R of T. Dynamically adjust the core tensor rank r_i of Tucker decomposition according to R. Apply the adaptive Tucker decomposition algorithm to decompose T into the core tensor G and the factor matrix U_i. The fused feature F_fused is reconstructed by G and U_i.

[0209] S16. Calculate the local complexity C_l and global redundancy R_g for F_fused obtained in S15. Dynamically adjust the quantization parameter Q and the encoding parameter E according to C_l and R_g. Compress F_fused using adaptive quantization and entropy coding to obtain the compressed feature F_compressed. Calculate the covariance matrix Σ for the preprocessed raw data D_preprocessed in S13. Perform eigendecomposition on Σ and select the eigenvectors corresponding to the largest k eigenvalues ​​to form the projection matrix P. Project D_preprocessed onto P to obtain the reduced-dimensional data D_reduced.

[0210] S17. Combine F_compressed and D_reduced of S16 into data stream S. Dynamically calculate the optimal shard size s and transmission priority p according to the network status N and data importance I. Shard S according to s to obtain a data shard set {S_1, S_2, ..., S_m}. Determine the transmission order of each shard according to p, and use the improved MQTT protocol to transmit the data shards to the central processing unit.

[0211] S18. Receive the data fragments transmitted by S17, and reassemble them to obtain the original data stream S'. Calculate the quality index Q_i of each data point in S'. Dynamically adjust the cleaning threshold T_c according to Q_i. Apply the adaptive cleaning algorithm to S' to remove low-quality data points and obtain the cleaned data stream S_clean. Analyze the feature distribution D_f of S_clean, and select the appropriate feature transformation function f_t according to D_f. Apply f_t to perform feature transformation on S_clean to obtain the transformed data stream S_transformed. Apply the incremental clustering algorithm to S_transformed, dynamically update the cluster center C_i and cluster radius r_i, and obtain the clustering result S_clustered.

[0212] S19. Write S_clustered processed by S18 into the time series database. Calculate the importance score I_s and access frequency F_a of each data point. Dynamically adjust the data retention period T_r according to I_s and F_a. Apply an adaptive data retention policy to archive or delete expired data. Build an incremental LSM tree structure and insert new data into the memory table. When the memory table reaches the threshold, merge it into a lower level on the disk. Dynamically adjust the number of levels L of the LSM tree and the size S_i of each level to optimize read and write performance. Finally, the stored processed data D_processed and the updated feature index I_feature are obtained.

[0213] S2. Get the processed data D_processed and the updated feature index I_feature stored in S1, and perform feature enhancement, selection, dimensionality reduction and deep fusion on these data; first, perform adaptive enhancement processing on each modal feature, then perform feature selection based on mutual information, then use nonlinear dimensionality reduction method to reduce feature dimensions, and finally perform deep fusion of features through multi-order tensor decomposition, graph convolution network and multi-head attention mechanism to obtain the final fused feature vector F_final;

[0214] S21. Read the processed data D_processed and feature index I_feature from the time series database. According to the index information in I_feature, extract the visual feature K_v, audio feature M_a, thermal feature F_h, motion feature M_v and environmental feature E_main from D_processed. Calculate the statistical properties of each feature, including mean μ, variance σ^2, skewness S and kurtosis K. According to these statistical properties, determine the enhancement parameter α for each feature.

[0215] S22. For the visual feature K_v in S21, calculate the spatial attention weight W_s and channel attention weight W_c of the feature map. Multiply K_v with W_s and W_c to obtain the enhanced visual feature K_v_enhanced. For the audio feature M_a, calculate the time-frequency spectrum S(t,f) and estimate the noise power spectrum N(f). Use spectral subtraction S_enhanced(t,f) = S(t,f) - α·N(f) to enhance S(t,f), and then re-extract the MFCC features from S_enhanced(t,f) to obtain M_a_enhanced. For the thermal feature F_h, apply adaptive histogram equalization to adjust the histogram distribution of each subband to obtain F_h_enhanced. For the motion feature M_v, use wavelet threshold denoising and adaptively select the threshold λ to obtain the denoised feature M_v_enhanced. For the environmental feature E_main, apply the exponential weighted moving average, calculate the smoothing coefficient α, and obtain the smoothed feature E_main_smoothed.

[0216] S23. Combine the enhanced features K_v_enhanced, M_a_enhanced, F_h_enhanced, M_v_enhanced and E_main_smoothed in S22 with the pre-stored identity label Y. Calculate the mutual information I(X;Y) between each feature and Y. Set the mutual information threshold T_I, select the feature subset with I(X;Y)>T_I, and obtain the selected feature set F_selected. Construct the feature dictionary D and the sparse coding matrix A, and optimize the objective function J(D,A) = ||F_selected - DA||_2^2 + λ||A||_1. Use the alternating direction multiplier method to solve the optimal solution of J(D,A) and obtain the sparse representation F_sparse.

[0217] S24. Construct a k-nearest neighbor graph G = (V, E) for F_sparse in S23, where V is the data point set and E is the edge set. Calculate the Laplacian matrix L of G. Solve the generalized eigenvalue problem Lf = λDf, where D is the degree matrix. Select the eigenvectors corresponding to the smallest m non-zero eigenvalues ​​to form a projection matrix P. Project F_sparse onto P to obtain the reduced feature F_reduced.

[0218] S25. Organize F_reduced of S24 into a high-order tensor T. Calculate the correlation matrix R between the order modes of T. Determine the rank vector r = [r_1, r_2, ..., r_n] of tensor decomposition based on R. Apply the tensor train decomposition algorithm to decompose T into a series of core tensors G_i and factor matrices U_i. The tensor fusion feature F_tt is obtained by reconstructing G_i and U_i.

[0219] S26. Construct the feature graph G_f = (V_f, E_f) based on the k-nearest neighbor graph G of S24, where V_f is the feature node set and E_f is the feature relationship edge set. Calculate the adjacency matrix A and degree matrix D of G_f. Construct the graph convolutional network layer GCN(H, A) = σ(D^(-1 / 2)AD^(-1 / 2)HW), where H is the input feature, W is the learnable weight matrix, and σ is the activation function. Input F_reduced into the multi-layer GCN to obtain the graph convolution feature F_gcn.

[0220] S27. Connect F_tt of S25 and F_gcn of S26 to form a multimodal feature matrix F_multi = [F_tt; F_gcn]. Calculate the self-attention matrix A_self = softmax(F_multi * F_multi^T / √d) of F_multi, where d is the feature dimension. Calculate the multi-head attention output M_head_i = A_self * F_multi * W_i, where W_i is the learnable weight matrix of the i-th attention head. Connect the outputs of all attention heads M_head = [M_head_1; M_head_2; ...; M_head_h], where h is the number of attention heads. Process M_head through a feedforward neural network to obtain the final fused feature vector F_final.

[0221] S28. Write F_final obtained in S27 into the time series database and update the feature index I_feature. Calculate the statistical characteristics of F_final, including mean, variance, covariance matrix, etc. Based on these statistical characteristics, update the feature distribution model P(F). Store the updated feature distribution model P(F) in the model library for subsequent identity recognition and verification.

[0222] S3. Obtain the final fused feature vector F_final and the updated feature distribution model P(F) in S2 for identity recognition and verification; first dynamically update the stored identity template, and then perform multi-factor authentication, including biometric matching, behavior pattern analysis, and environmental context verification; use an adaptive decision threshold to determine the final authentication result, and generate an authentication result R containing the identified identity ID, confidence score, and verification level;

[0223] S31. Read the stored identity template set T = {T_1, T_2, ..., T_n} and the fused feature vector F_final in S2 from the time series database. Calculate the Euclidean distance d_i between F_final and each template T_i. Select the k templates with the smallest distance as the candidate template set T_c. For each candidate template T_c_i, calculate its similarity score s_i with F_final. Update the candidate template using the online stochastic gradient descent algorithm: T_c_i_new = T_c_i + η * (F_final -T_c_i), where η is the learning rate. Calculate the quality score q_i = f(age_i, usage_i,accuracy_i) of the updated template, where age_i is the template age, usage_i is the usage frequency, and accuracy_i is the historical matching accuracy. Determine the update weight w_i based on q_i, and perform weighted average of the updated template and the original template: T_i_updated = w_i *T_c_i_new + (1 - w_i) * T_c_i. Write T_i_updated back to the time series database and update the corresponding template index.

[0224] S32. Construct a local sensitive hash (LSH) function set H = {h_1, h_2, ..., h_m}, where h_i(x) =sign(w_i^T * x + b_i), w_i is a random vector, and b_i is a random bias. Apply H to all templates T_i_updated updated in S31 and the input fused feature vector F_final to obtain their hash codes. Use the Hamming distance to calculate the distance between the hash code of F_final and the hash codes of all templates, and select the L templates with the smallest distance as the candidate set T_candidates. Calculate the cosine similarity s_cos_i between F_final and each template in T_candidates. For each s_cos_i, calculate its likelihood probability p_i = P(s_cos_i | F) under the feature distribution model P(F). Select the template with the highest likelihood probability as the best match T_best.

[0225] S33. Retrieve the historical behavior sequence B = {B_1, B_2,..., B_t} related to T_best from the user behavior database. Use the hidden Markov model (HMM) to model B and obtain the model parameters λ = (A, B, π), where A is the state transition matrix, B is the observation probability matrix, and π is the initial state distribution. Get the observed behavior sequence O = {O_1, O_2, ..., O_k} of the current user. Use the forward algorithm to calculate the likelihood probability P(O|λ) of O under model λ. Compare P(O|λ) with the preset threshold T_behavior to determine the consistency of behavior.

[0226] S34. Retrieve the current environment parameters E_current and the historical environment-user association data D_eu = {(E_i, U_i)} from the environment database. Use the Bayesian network model to represent the conditional dependency between the environment, user identity, and time. Calculate the prior probability P(U) of the user's appearance, the conditional probability P(E|U) of the environment parameters, and the conditional probability P(T|U,E) of time. Use Bayes' theorem to calculate the posterior probability: P(U|E,T) = P(E|U) * P(T|U,E) * P(U) / P(E,T). Compare the calculated posterior probability P(U|E,T) with the preset threshold T_context to evaluate the rationality of the environment context.

[0227] S35. Calculate the global false alarm rate FAR and false negative rate FRR based on historical verification data. Use adaptive binary search to search for the optimal decision threshold τ so that |FAR(τ) - FRR(τ)| is minimized. Calculate the confidence score C = f(s_cos_best, P(O|λ), P(U|E,T)) of the current verification result, where s_cos_best is the best cosine similarity in S32. Compare C with τ. If C ≥ τ, the verification passes, otherwise the verification fails.

[0228] S36. Based on the verification results of S32-S35, a decision tree model D is constructed. The nodes of the decision tree include biometric matching results, behavior consistency scores, and environmental context probabilities. The information gain criterion is used to select the optimal split features and construct a multi-level verification path. The various indicators of the current verification case are input into the decision tree D to obtain the final verification level L.

[0229] S37. Combining the results of S32-S36, generate the identity authentication result R = {ID, C, L}, where ID is the identified identity, C is the confidence score, and L is the verification level. Write R into the verification result database and update the user's verification history. Calculate and update the user's average verification performance indicators, including average confidence, verification success rate, etc. According to the updated performance indicators, adjust the user's verification strategy parameters, such as decision threshold, behavior model complexity, etc.

[0230] S4. Obtain the authentication result R = {ID, C, L} in S3 and perform intelligent access control and crowd flow management. First, perform dynamic access control based on user identity and authentication results, then perform adaptive crowd flow control, including crowd flow prediction and congestion detection. Finally, perform real-time abnormal event detection and response, and comprehensively derive the final access control decision D.

[0231] S41. Retrieve the historical behavior data H = {h_1, h_2, ..., h_n} corresponding to the identity ID from the user database, where h_i is a single behavior record. Extract the feature vector f_i for each record h_i, including access time, location, duration, etc. Construct a feature matrix F = [f_1, f_2, ..., f_n]. Calculate the covariance matrix Σ of F. Perform feature decomposition on Σ, obtain the first k principal components, and form a projection matrix P. Project F onto P to obtain the reduced-dimensional behavior feature F_reduced.

[0232] S42. Apply the fuzzy c-means clustering algorithm to F_reduced in S41. Initialize the cluster centers C = {c_1,c_2, ..., c_m} and the membership matrix U. Iteratively update the membership u_ij and cluster centers c_j until the objective function J =Σ_i Σ_j u_ij^m ||f_i - c_j||^2 converges or the maximum number of iterations is reached. According to the final clustering result, assign the role label Role to the user.

[0233] S43. Retrieve the predefined role-authority mapping relationship M_rp from the authority database. Construct an ontology model O, including concepts such as user, role, resource, and operation and their relationships. Input Role, verification level L, and confidence score C into O. Use the description logic reasoning engine E to reason about O and obtain the user-accessible resource set R_accessible and the allowed operation set O_allowed.

[0234] S44. Retrieve historical crowd flow data S = {s_1, s_2, ..., s_t} from the crowd flow database, where s_i is the crowd flow at a certain moment. Construct a time series X = {x_1, x_2, ..., x_t}, where x_i = [s_i, s_(i-1), ..., s_(i-w+1)] is a sliding window of length w. Divide X into a training set X_train and a test set X_test. Construct a long short-term memory network (LSTM) model, which includes an input layer, an LSTM layer, and an output layer. Use X_train to train the LSTM model and minimize the mean square error loss function L = 1 / n Σ (y_i - ŷ_i)^2. Use the trained LSTM model to predict the crowd flow in the next k time steps and obtain the predicted sequence S_pred = {s_(t+1), s_(t+2), ..., s_(t+k)}.

[0235] S45. Obtain the real-time crowd distribution data D = {(x_1, y_1, n_1), (x_2, y_2,n_2), ..., (x_m, y_m, n_m)} of the current site, where (x_i, y_i) is the location coordinate and n_i is the number of people at the location. Calculate the local density of each point ρ_i = Σ_j χ(d_ij - d_c), where d_ij is the distance between point i and point j, d_c is the cutoff distance, and χ(x) is the indicator function. Calculate the minimum distance δ_i = min_j:ρ_j>ρ_i(d_ij) from each point to a higher density point. Identify the density peak points based on the product of ρ and δ, which are the congestion centers. For each congestion center, calculate its impact range and severity.

[0236] S46. Construct the topological graph G = (V, E) of the site, where V is a set of nodes (representing locations) and E is a set of edges (representing accessible paths). For each edge e_ij∈E, calculate its cost function c(e_ij) = f(length, congestion) based on the congestion in S45. Set the starting point s (current location) and the end point t (target location). Use the A algorithm to find the optimal path P = {v_1, v_2, ..., v_k} from s to t. The heuristic function of the A algorithm is h(n) = g(n) + h'(n), where g(n) is the actual cost from the starting point to node n, and h'(n) is the estimated cost from node n to the end point.

[0237] S47. Retrieve the normal behavior pattern set N = {n_1, n_2, ..., n_m} from the behavior pattern database. Extract the current user's behavior sequence B_current = {b_1, b_2, ..., b_k}. For each b_i, calculate its deviation d_ij from each normal pattern n_j in N. Construct an isolation forest model F, containing t isolated trees. For each tree, randomly select a feature and a split point, and recursively split the data until all samples are isolated or the maximum depth is reached. For each behavior b_i in B_current, calculate its average path length l(b_i) in F. The anomaly score s(b_i) = 2^(-l(b_i) / c(F)), where c(F) is the normalization factor of F. Compare s(b_i) with the preset threshold T_anomaly to determine whether it is an abnormal behavior.

[0238] S48. Deploy multiple intelligent agents A = {a_1, a_2, ..., a_n}, each of which is responsible for monitoring and responding to a specific area or function. Each agent a_i maintains a local state s_i and a decision rule set R_i. When an abnormal event is detected, the relevant agent a_j broadcasts the event information E_j to other agents. The agent that receives E_j generates a response decision d_i based on its own state s_i and rule set R_i. The global decision D_global is reached among agents using the Raft consensus algorithm. First, a leader agent a_leader is elected. a_leader collects the decisions of all agents, merges them into a global decision D_global, and synchronizes D_global to all agents.

[0239] S49. Based on the results of S41-S48, the final access control decision D = {access_granted, access_level, routing_info, alert_level} is generated. Among them, access_granted is determined based on the permission reasoning result of S43; access_level is determined according to the role allocation of S42 and the permission level of S43; routing_info uses the optimal path P calculated by S46; alert_level is based on the anomaly detection result of S47 and the consensus decision of S48. D is written into the decision log database, and the access control policy library is updated at the same time. According to the decision result of this time, the parameters of the decision model are adjusted using the reinforcement learning algorithm to optimize the long-term decision performance.

[0240] It should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

Claims

1. An intelligent access control management method based on multimodal recognition and Internet of Things technology, characterized in that: The steps include: Step S1: Collect multimodal data through distributed IoT devices, perform synchronization and alignment processing, and adaptive preprocessing, obtain preprocessed data and extract multiscale features from it, dynamically fuse the multiscale features to obtain fused features, compress and preliminarily reduce the dimensionality of the fused features and preprocessed data, and transmit them to the central processing unit for stream processing, store the processed data in a time series database and update the feature index; Step S2, obtaining the processed data and feature index, performing adaptive enhancement processing on each modal feature, then performing feature selection based on mutual information, using a nonlinear dimensionality reduction method for dimensionality reduction, and performing deep fusion through multi-order tensor decomposition, graph convolutional network and multi-head attention mechanism to obtain the final fused feature vector; Step S3, obtaining the final fused feature vector and the pre-built feature distribution model, and dynamically updating the stored identity template and performing multi-factor identity authentication; Use adaptive decision thresholds to determine the final authentication result, generating an authentication result that includes the identified identity ID, confidence score, and verification level; multi-factor authentication includes biometric matching, behavioral pattern analysis, and environmental context verification; Step S4: Obtain identity authentication results, perform dynamic access control based on user identity and authentication results, and perform adaptive crowd control, as well as real-time abnormal event detection and response, to comprehensively arrive at the final access control decision; wherein, adaptive crowd control includes crowd prediction and congestion detection; In step S4, dynamic access control is performed based on the user identity and the verification result, specifically: Step S41: retrieve the historical behavior data corresponding to the identity ID from the user database, extract the feature vector of each record, including the access time, location, and duration information, and construct a feature matrix; calculate the covariance matrix of the feature matrix, perform feature decomposition on the covariance matrix, obtain the first k principal components, and construct a projection matrix; project the feature matrix onto the projection matrix to obtain the behavior features after dimensionality reduction; k is a natural number greater than 1; Step S42, obtaining the reduced-dimensional behavior features in step S41, and processing them using the fuzzy c-means clustering algorithm; initializing the cluster center and the membership matrix, and iteratively updating the membership and cluster center until the objective function converges or reaches the maximum number of iterations; assigning role labels to users according to the final clustering results; Step S43: Retrieve the predefined role-authority mapping relationship from the authority database, and build an ontology model including user, role, resource and operation concepts and their relationships by combining the user role label, verification level and confidence score obtained in step S42; use the description logic reasoning engine to reason on the ontology model to obtain the resource set accessible to the user and the allowed operation set; In step S4, adaptive crowd flow control is performed, and the specific process is as follows: Step S44: retrieve historical crowd flow data from the crowd flow database, construct a time series, and use a sliding window method to process the time series data; divide the processed data into a training set and a test set, use a pre-built and trained LSTM model to predict the crowd flow in the next m time steps, and obtain a predicted sequence, which is recorded as crowd flow data, where m is a natural number; Step S45: obtain the real-time crowd distribution data and crowd data of the current site, calculate the local density of each location point; calculate the minimum distance from each point to a point with higher density, and identify the density peak point as the congestion center according to the product of density and distance; for each congestion center, calculate the congestion situation, including the impact range and severity; Step S46: construct a topological map of the site, where nodes represent locations and edges represent accessible paths; calculate a cost function for each edge according to the congestion situation; set the current location as the starting point and the target location as the end point, and use the A-star algorithm to find the optimal path from the starting point to the end point; wherein the heuristic function of the A-star algorithm combines the actual cost from the starting point to the current node and the estimated cost from the current node to the end point; In step S4, real-time abnormal event detection and response are performed to comprehensively arrive at the final access control decision, specifically: Step S47, retrieve the normal behavior pattern set from the behavior pattern database, extract the behavior sequence of the current user; calculate the deviation between each behavior in the current user behavior sequence and each pattern in the normal pattern set; construct an isolation forest model, including multiple isolated trees; for each tree, randomly select features and split points, recursively split the data until all samples are isolated or the maximum depth is reached; calculate the average path length of each behavior in the current user behavior sequence in the isolation forest, and then calculate the anomaly score; compare the anomaly score with the preset threshold to determine whether it is an abnormal behavior; Step S48: deploy multiple intelligent agents, each of which is responsible for monitoring and responding to a specific area or function, maintaining local status and decision-making rule sets; when abnormal behavior is detected, the relevant agent broadcasts event information to other agents; the agent that receives the event information generates a response decision based on its own status and rule set; the Raft consensus algorithm is used to reach a global consensus decision among agents, first electing a leader agent, the leader collects the decisions of all agents, merges them into a global decision, and synchronizes the global decision to all agents; Step S49, comprehensively process the access right reasoning results, role assignment results, optimal path, anomaly detection results and global consensus decision to generate the final access control decision, including whether to grant access rights, access level, routing information and alarm level; write the generated decision into the decision log database, and update the access control policy library at the same time.

2. The intelligent access control management method based on multimodal recognition and Internet of Things technology as claimed in claim 1, characterized in that: The step S1 is specifically as follows: Step S11: collect multi-angle visual data, spatialized audio signals, heat map sequences, environmental parameters, and the number of people entering and leaving through distributed IoT devices; call the precision time protocol server to synchronize the clock of each IoT device, add a microsecond timestamp to each data point, and obtain a data set with a timestamp; Step S12: Based on the data set with timestamps, calculate the complexity of each modal data, and dynamically adjust the sliding window size according to the complexity; apply the dynamic time warping algorithm to the data of each modality, and calculate the alignment matrix between different modalities; resample and interpolate the data set with timestamps according to the alignment matrix to obtain the aligned multimodal time series; Step S13, reading and adaptively preprocessing the aligned multimodal time series to obtain features of each modality; Adaptive preprocessing includes image and depth map enhancement and deblurring, audio signal enhancement, super-resolution reconstruction of thermal map sequences, millimeter wave radar signal smoothing, and outlier correction of environmental parameters; each modal feature includes visual features, audio features, thermal features, motion features, and environmental features; Step S14, organizing the extracted visual features, audio features, thermal features, motion features and environmental features into a fifth-order tensor; calculating the pattern correlation matrix of the fifth-order tensor; dynamically adjusting the rank of the core tensor of Tucker decomposition according to the correlation matrix; applying the adaptive Tucker decomposition algorithm to decompose the tensor into a core tensor and a factor matrix; and obtaining fusion features by reconstructing the core tensor and the factor matrix; Step S15, calculating the local complexity and global redundancy of the fused features; dynamically adjusting the quantization parameters and coding parameters according to the complexity and redundancy; compressing the fused features using adaptive quantization and entropy coding to obtain compressed features; For the preprocessed raw data, calculate the covariance matrix; Perform eigendecomposition on the covariance matrix, select the eigenvectors corresponding to the largest eigenvalues ​​to form a projection matrix; project the preprocessed raw data onto the projection matrix to obtain the reduced-dimensional data; Step S16: merging the compressed features and the dimension-reduced data into a data stream; Dynamically calculate the optimal fragment size and transmission priority according to network conditions and data importance; fragment the data stream according to the fragment size to obtain a data fragment set; Determine the transmission order of each shard according to the transmission priority; use the improved MQTT protocol to transmit the data shards to the central processing unit; Step S17, receiving the transmitted data fragments, and recombining to obtain the original data stream; calculating the quality index of each data point in the data stream; dynamically adjusting the cleaning threshold according to the quality index; applying an adaptive cleaning algorithm to the data stream, removing low-quality data points, and obtaining a cleaned data stream; analyzing the feature distribution of the cleaned data stream, and selecting a suitable feature transformation function according to the distribution; Applying a feature transformation function to perform feature transformation on the cleaned data stream to obtain a transformed data stream; Step S18: Apply an incremental clustering algorithm to the converted data stream, dynamically update the cluster center and cluster radius, and obtain clustering results; write the clustering results into a time series database; calculate the importance score and access frequency of each data point; dynamically adjust the data retention period according to the importance score and access frequency; Apply adaptive data retention policies to archive or delete expired data; Step S19: Call the constructed incremental LSM tree structure to insert new data into the memory table; When the memory table reaches a preset threshold, it is merged into a lower level on disk; Dynamically adjust the number of levels and the size of each level of the LSM tree to optimize read and write performance; ultimately obtain the stored processed data and updated feature index.

3. The intelligent access control management method based on multimodal recognition and Internet of Things technology as claimed in claim 2, characterized in that: The step S2 is specifically as follows: Step S21, reading processed data and feature indexes from a time series database; extracting visual features, audio features, thermal features, motion features, and environmental features according to the feature indexes; Calculate the statistical properties of each feature; determine the enhancement parameters for each feature based on the statistical properties; the statistical properties include mean, variance, skewness and kurtosis; Step S22, calculating the spatial attention weight and channel attention weight of the visual feature; multiplying the spatial attention weight and channel attention weight of the visual feature to obtain an enhanced visual feature; Calculate the time-frequency spectrum of audio features; Estimate the noise power spectrum; enhance the time-frequency spectrum using spectral subtraction; re-extract the Mel-frequency cepstral coefficient features from the enhanced time-frequency spectrum to obtain enhanced audio features; Applying adaptive histogram equalization to the thermal signature; adjusting the histogram distribution of each sub-band to obtain enhanced thermal signatures; Use wavelet threshold to denoise the motion features; adaptively select the threshold to obtain the denoised motion features; Applying exponential weighted average to the environmental characteristics; calculating the smoothing coefficient to obtain the smoothed environmental characteristics; Obtain enhanced features of each modality; Step S23, combining the enhanced modal features with the pre-stored identity tags; calculating the mutual information between each feature and the identity tag; setting a mutual information threshold, and selecting a feature subset whose mutual information is greater than the threshold; Step S24, constructing a feature dictionary and a sparse coding matrix; Optimize the objective function and use the alternating direction multiplier method to find the optimal solution to obtain a sparse representation of the features; Step S25, constructing a k-nearest neighbor graph for the sparse representation; calculating the Laplacian matrix of the graph; solving the generalized eigenvalue problem; selecting eigenvectors corresponding to the smallest number of non-zero eigenvalues ​​to form a projection matrix; Project the sparse representation onto the projection matrix to obtain the reduced-dimensional features; Step S26, organizing the reduced-dimensional features into high-order tensors; calculating the correlation matrix between the various-order modes of the tensor; determining the rank vector of the tensor decomposition according to the correlation matrix; applying the tensor train decomposition algorithm to decompose the tensor into a series of core tensors and factor matrices; and reconstructing the core tensors and factor matrices to obtain tensor fusion features; Step S27, constructing a feature graph based on the k-nearest neighbor graph; calculating the adjacency matrix and degree matrix of the feature graph; constructing a graph convolutional network layer; inputting the reduced-dimensional features into the multi-layer graph convolutional network to obtain graph convolutional features; Step S28, connecting the tensor fusion features and the graph convolution features to form a multimodal feature matrix; calculating the self-attention matrix of the multimodal feature matrix; calculating the multi-head attention output; connecting the outputs of all attention heads; processing the connected outputs through a feedforward neural network to obtain the final fused feature vector; Step S29: write the final fused feature vector into the time series database and update the feature index; Calculate the statistical characteristics of the fused feature vector; update the feature distribution model based on the statistical characteristics; store the updated feature distribution model in the model library for subsequent identity recognition and verification; the statistical characteristics include mean, variance and covariance matrix.

4. The intelligent access control management method based on multimodal recognition and Internet of Things technology as claimed in claim 3 is characterized in that: The step S3 is specifically as follows: Step S31, read the stored identity template set and fused feature vector from the time series database; calculate the Euclidean distance between the fused feature vector and each template, and select several templates with the smallest distance as candidate templates; calculate the similarity score of the candidate templates and update them using the online stochastic gradient descent algorithm; Calculate the quality score of the updated template and determine the update weight according to the score; perform weighted average of the updated template and the original template to obtain the final updated template; Write the updated template back to the time series database and update the corresponding template index; Step S32, construct a local sensitive hash function set and apply it to all updated templates and input fused feature vectors; calculate the Hamming distance between hash codes, and select several templates with the smallest distance as candidate sets; calculate the cosine similarity and likelihood probability between the fused feature vector and each template in the candidate set, and select the best matching template; retrieve the historical behavior sequence related to the best matching template from the user behavior database, use the hidden Markov model to model and calculate the likelihood probability of the current behavior sequence; retrieve the current environment parameters and historical association data from the environment database, and use the Bayesian network model to calculate the posterior probability of the user appearing; Output the best matching template, behavior consistency probability and environmental context probability; Step S33, calculate the global false alarm rate and missed alarm rate based on historical verification data; use adaptive dichotomy to search for the optimal decision threshold; calculate the confidence score of the current verification result, including the best cosine similarity, behavior sequence likelihood probability and environmental posterior probability; compare the confidence score with the optimal decision threshold; build a decision tree model, input the biometric matching result, behavior consistency score and environmental context probability into the decision tree; obtain the result of whether the verification is passed and the final verification level; Step S34: Based on the outputs of step S32 and step S33, generate an identity authentication result including the identified identity, confidence score and verification level; write the verification result into a verification result database; update the user's verification history record; calculate and update the user's average verification performance index; adjust the user's verification strategy parameters based on the updated performance index; output the final identity authentication result and the updated user verification strategy.

5. The intelligent access control management method based on multimodal recognition and Internet of Things technology as claimed in claim 4 is characterized in that: The step S13 is specifically as follows: Step S131, calculating the local complexity of the received image sequence and depth map sequence; adaptively adjusting the contrast limit parameter according to the calculated local complexity, applying the adjusted parameter to the contrast limited adaptive histogram equalization algorithm, processing the image sequence and depth map sequence, and obtaining an enhanced image sequence and depth map sequence; Step S132, obtaining the enhanced image sequence in the previous step, and calculating the optical flow field between adjacent frames in the sequence; The motion blur kernel is estimated according to the calculated optical flow field, and the estimated blur kernel is input into the Wiener filter to deblur each frame of the enhanced image sequence to obtain a deblurred image sequence; Step S133, obtaining the deblurred image sequence and enhanced depth map sequence from the first two steps, calculating the image gradient and local direction distribution; adaptively adjusting the scale parameter and direction parameter of the scale-invariant feature transformation algorithm according to the calculated gradient and direction distribution; applying the adjusted parameters to the scale-invariant feature transformation algorithm to extract visual features from the deblurred image sequence and enhanced depth map sequence; Step S134: receiving the audio signal aligned in step S13, estimating the signal-to-noise ratio at each time point, azimuth angle, and elevation angle; dynamically adjusting the beamforming weight according to the estimated signal-to-noise ratio, applying the adjusted weight to the beamforming algorithm, processing the original audio signal, and obtaining an enhanced audio signal; Step S135, receiving the audio signal enhanced in the previous step, and calculating its short-time spectrum; dynamically adjusting the center frequency and bandwidth of the Mel filter bank according to the time-varying characteristics of the calculated short-time spectrum; applying the adjusted Mel filter bank to the short-time spectrum, calculating the Mel frequency cepstral coefficients, and obtaining dynamic Mel frequency cepstral coefficients as audio features; Step S136, obtaining the heat map sequence aligned in step S13, constructing a residual network including multiple residual blocks and upsampling layers; using a high-resolution heat map data set to train the residual network to minimize the reconstruction error; applying the trained residual network to each frame in the original heat map sequence to perform super-resolution reconstruction to obtain a high-resolution heat map sequence; Step S137, obtaining the high-resolution thermal image sequence in the previous step, applying the empirical mode decomposition algorithm to it, and obtaining a series of intrinsic mode functions; performing Hilbert transform on each intrinsic mode function, calculating the instantaneous frequency and instantaneous amplitude; combining the calculated instantaneous frequency and instantaneous amplitude into a thermal signature; Step S138, receiving the millimeter-wave radar signal aligned in step S13, calculating the dynamic characteristics of the signal, including the signal strength change rate and the phase change rate; adaptively adjusting the process noise covariance matrix and the measurement noise covariance matrix of the Kalman filter according to the calculated dynamic characteristics; inputting the adjusted covariance matrix into the Kalman filter, filtering the original signal, and obtaining a smoothed signal; Step S139, receiving the millimeter-wave radar signal smoothed in the previous step, calculating its power spectrum density and autocorrelation function; combining the calculated power spectrum density and autocorrelation function into a motion feature; Step S1310, obtaining the environmental parameters aligned in step S13, including temperature, humidity, light, carbon dioxide concentration and fine particulate matter index; calculating the local anomaly score for each parameter, using the sliding window method to calculate the local mean and standard deviation of each parameter, and comparing the current value with the local statistic to obtain the anomaly score; identifying the outliers according to the calculated anomaly scores, and correcting the outliers using the interpolation method or the local regression method to obtain the corrected environmental parameters; Step S1311, obtaining the corrected environmental parameters in the previous step, calculating the mutual information matrix between the parameters; dynamically adjusting the separation matrix of the independent component analysis algorithm according to the calculated mutual information matrix; applying the adjusted separation matrix to the independent component analysis algorithm to extract the main environmental features from the corrected environmental parameters; Step S1312: Integrate the various modal features obtained in the previous steps, including visual features, audio features, thermal features, motion features and environmental features, to form a multimodal feature set; output this multimodal feature set for use in subsequent processing steps.

6. An intelligent access control management system based on multimodal recognition and Internet of Things technology, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent access control management method based on multimodal recognition and Internet of Things technology as described in any one of claims 1 to 5.

Images