AI access control dynamic identification system based on multi-modal fusion

Abstract

The invention belongs to the technical field of access control systems, and discloses a multi-modal fusion-based AI access control dynamic identification system, which comprises a data acquisition module, a data processing module, a multi-modal fusion and decision module, a data storage module, a communication and interface module and a user interaction module. Through the arrangement of the data acquisition module, the data processing module and the multi-modal fusion and decision-making module, after the data acquisition module acquires the gait, face and voiceprint biological feature information of a user, the data processing module carries out feature extraction and analysis on the gait, face and voiceprint; then multi-modal fusion of gait, face and voiceprint independent decision results is carried out through the multi-modal fusion and decision module, and compared with a traditional access control system and mutual complementation of multi-modal features, the defects of a single biological recognition technology are overcome, the problem that identity information is prone to being copied is solved, and the safety of the access control system is improved. Therefore, the accuracy and robustness of identity authentication are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of access control systems, and specifically relates to an AI access control dynamic recognition system based on multimodal fusion. Background Art

[0002] With the rapid development of information technology, access control systems, as an important means of ensuring security, are undergoing a profound transformation from traditional mechanical locks to intelligent electronic systems. Traditional access control systems, such as password locks and card access control, have improved security to a certain extent, but there are security risks such as easy forgetting, easy loss, and easy copying. As society's requirements for security and convenience continue to increase, biometric technology has gradually become an important direction for access control system upgrades.

[0003] Existing biometric technologies mainly include face recognition, fingerprint recognition, iris recognition, etc. The security of the access control system can be improved by extracting and analyzing individual biometric features for identity authentication. However, single biometric technology still has limitations in practical applications. The recognition rate of face recognition decreases when the lighting conditions are poor or the face is obscured; fingerprint recognition is greatly affected by finger moisture and stains; although iris recognition has high accuracy, the equipment cost is high and user acceptance is limited. Moreover, fingerprints and faces, as commonly used biometric means, have the problem of easy identity information copying and low recognition accuracy, so improvements are needed. Summary of the Invention

[0004] The purpose of the present invention is to provide an AI access control dynamic recognition system based on multimodal fusion to solve the problems raised in the above background technology.

[0005] In order to achieve the above-mentioned purpose, the present invention provides the following technical solutions: an AI access control dynamic recognition system based on multimodal fusion, comprising: a data acquisition module, a data processing module, a multimodal fusion and decision module, a data storage module, a communication and interface module, and a user interaction module, wherein:

[0006] The data acquisition module is used to collect biometric information of the user's gait, face, and voiceprint;

[0007] The data processing module is used to extract features from the collected information;

[0008] The multimodal fusion and decision module is used to perform multimodal fusion and decision-making on the extracted features;

[0009] The data storage module is used to store the input biometric information and system logs;

[0010] The communication and interface module is responsible for data transmission and communication between the system and other devices or systems, and provides an external interface for third-party applications to call;

[0011] The user interaction module provides an interactive interface between the user and the system and displays the recognition results.

[0012] As a preferred technical solution of the present invention, the data acquisition module is divided into a gait acquisition unit, a face acquisition unit and a voiceprint acquisition unit. The gait acquisition unit captures the coordinates of the user's joints when walking through a camera, the face acquisition unit collects the user's face through a camera, and the voiceprint acquisition unit collects the user's voice through a microphone.

[0013] As a preferred technical solution of the present invention, the data processing module is divided into a gait feature extraction unit, a face feature extraction unit and a voiceprint feature extraction unit. The extraction method of the gait feature extraction unit is:

[0014] Normalization of joint point coordinates:

[0015]

[0016] Where, P i is the original coordinate of the i-th joint point, P root is the root joint, P hip is the hip joint coordinate, which is subtracted from the root joint coordinate and divided by the distance from the hip joint to the root joint to eliminate the influence of individual height differences on gait;

[0017] Joint angle calculation:

[0018]

[0019] Where p1, p2, and p3 are three joint points, and the angle formed by the three joint points is used to describe the degree of limb bending;

[0020] Spatiotemporal feature modeling:

[0021] Using the spatiotemporal graph convolutional network, the calculation formula is:

[0022]

[0023] In the formula, the adjacency matrix A k To describe the connection relationship between joints, the adjacency matrix D k Used to normalize the adjacency matrix to avoid numerical instability, weight It is a learnable parameter used to aggregate spatial features. Temporal convolution processes time series through 1D convolution to capture the dynamic changes of gait;

[0024] Gait sequence analysis and matching:

[0025] Dynamic time warping is used for calculation, and the calculation formula is as follows:

[0026]

[0027] Where d(i, j) is the distance between two gait sequences at time points θ and θ. Dynamic time warping aligns gait sequences through dynamic programming to solve the problem of inconsistent sequence lengths caused by different speeds or step lengths.

[0028] As a preferred technical solution of the present invention, the extraction method of the face feature extraction unit is: using the Dlib library method to detect the face position, obtain key points, extract 128-dimensional feature vectors, and then use the Euler distance formula to calculate:

[0029]

[0030] Where x i 、y i are the X and Y coordinates. By calculating the straight-line distance between the two eigenvectors, the smaller the distance, the higher the similarity.

[0031] As a preferred technical solution of the present invention, the extraction method of the voiceprint feature extraction unit is: using Mel-frequency cepstral coefficients for feature extraction, and the calculation formula is:

[0032]

[0033] Where X(k) is the spectrum amplitude of the speech signal, H m (k) Response function of the mth Mel filter (a total of M filters);

[0034] Gaussian mixture model training:

[0035] The probability density function calculation formula is:

[0036]

[0037] Where x is the eigenvector of the Mel-frequency cepstral coefficients, are model parameters (weight, mean, covariance matrix), and C is the number of Gaussian components (usually 32-256).

[0038] As a preferred technical solution of the present invention, the multimodal fusion and decision module is used to output the decision results of gait behavior, face recognition and voiceprint recognition and perform multimodal feature decision fusion:

[0039] The calculation method of gait behavior decision results is:

[0040] P gait =D(i,j)

[0041] The calculation method for face recognition decision results is:

[0042] P face =dis(X,Y)

[0043] The calculation method for voiceprint recognition decision results is:

[0044] P voice =p(x|λ)

[0045] The multimodal decision fusion formula is:

[0046] P final =(ω1P gait +ω2P face +ω3P voice )×100%

[0047] Where, P final The final result is P gait Output identity probability for gait feature extraction unit, P face The face feature extraction unit outputs the identity probability, P voice The identity probability is output by the voiceprint feature extraction unit, ω1, ω2, ω3 are weight coefficients, P final A value greater than 90% indicates successful recognition.

[0048] As a preferred technical solution of the present invention, the data storage module is divided into a user database unit, an identification record library unit and a system log unit. The user database unit is used to store the biometric information and identity information of registered users, the identification record library unit is used to store the results of each identification, and the system log unit is used to store the system's operating status, error information and maintenance records.

[0049] As a preferred technical solution of the present invention, the communication and interface module includes a network communication unit and an API interface unit. The network communication unit supports TCP / IP, HTTP, and MQTT protocols to realize data transmission between the system and a remote server or other device. The API interface unit provides a standard API interface for third-party applications to integrate and call the functions of the access control system.

[0050] As a preferred technical solution of the present invention, the user interaction module is divided into a display unit, a voice interaction unit, and an operation feedback unit. The display unit displays the recognition results through a display screen, the voice interaction unit plays voice prompts through a speaker to guide the user to perform operations or inform the recognition results, and the operation feedback unit gives corresponding feedback based on the user's operations or recognition results.

[0051] The beneficial effects of the present invention are as follows:

[0052] The present invention is provided with a data acquisition module, a data processing module and a multimodal fusion and decision-making module. After the data acquisition module collects the user's gait, face and voiceprint biometric information, the data processing module performs feature extraction and analysis of the gait, face and voiceprint, and then the multimodal fusion and decision-making module performs multimodal fusion of the independent decision results of gait, face and voiceprint. Compared with the traditional access control system, the mutual complementarity of multimodal features overcomes the shortcomings of single biometric recognition technology and solves the problem that identity information is easily copied, thereby significantly improving the accuracy and robustness of identity authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] like Figure 1 As shown, the embodiment of the present invention provides an AI access control dynamic recognition system based on multimodal fusion, including: a data acquisition module, a data processing module, a multimodal fusion and decision module, a data storage module, a communication and interface module, and a user interaction module, wherein:

[0056] The data acquisition module is used to collect biometric information such as user gait, face, and voiceprint;

[0057] The data processing module is used to extract features from the collected information;

[0058] The multimodal fusion and decision module is used to perform multimodal fusion and decision-making on the extracted features;

[0059] The data storage module is used to store the input biometric information and system logs;

[0060] The communication and interface module is responsible for data transmission and communication between the system and other devices or systems, and provides external interfaces for third-party applications to call;

[0061] The user interaction module provides an interactive interface between the user and the system and displays the recognition results.

[0062] After the data acquisition module collects the user's gait, face, and voiceprint biometric information, the data processing module can perform feature extraction and analysis on the gait, face, and voiceprint, and then perform multimodal fusion of the gait, face, and voiceprint decision results through the multimodal fusion and decision module. Compared with traditional access control systems, the recognition accuracy is higher, overcoming the shortcomings of single biometric technology, solving the problem of easy duplication of identity information, and significantly improving the security of the access control system.

[0063] Among them, the data acquisition module is divided into a gait acquisition unit, a face acquisition unit and a voiceprint acquisition unit. The gait acquisition unit captures the coordinates of the user's joints when walking through a camera, the face acquisition unit collects the user's face through a camera, and the voiceprint acquisition unit collects the user's voice through a microphone.

[0064] The cameras used in the gait acquisition unit and the face acquisition unit are high-definition infrared cameras, so that the user's gait and face can be acquired during the day and at night.

[0065] The data processing module is divided into a gait feature extraction unit, a face feature extraction unit, and a voiceprint feature extraction unit. The extraction method of the gait feature extraction unit is as follows:

[0066] Normalization of joint point coordinates:

[0067]

[0068] Where, P i is the original coordinate of the i-th joint point, P root is the root joint, P hip is the hip joint coordinate, which is subtracted from the root joint coordinate and divided by the distance from the hip joint to the root joint to eliminate the influence of individual height differences on gait;

[0069] Joint angle calculation:

[0070]

[0071] Where p1, p2, and p3 are three joint points, and the angle formed by the three joint points is used to describe the degree of limb bending;

[0072] Spatiotemporal feature modeling:

[0073] Using the spatiotemporal graph convolutional network, the calculation formula is:

[0074]

[0075] In the formula, the adjacency matrix A k To describe the connection relationship between joints, the adjacency matrix D k Used to normalize the adjacency matrix to avoid numerical instability, weight It is a learnable parameter used to aggregate spatial features. Temporal convolution processes time series through 1D convolution to capture the dynamic changes of gait;

[0076] Gait sequence analysis and matching:

[0077] Dynamic time warping is used for calculation, and the calculation formula is as follows:

[0078]

[0079] Where d(i, j) is the distance between two gait sequences at time points θ and θ. Dynamic time warping aligns gait sequences through dynamic programming to solve the problem of inconsistent sequence lengths caused by different speeds or step lengths.

[0080] By modeling the spatiotemporal features of gait characteristics and analyzing and matching gait sequences, the user's gait behavior can be accurately identified by capturing the dynamic patterns of human walking.

[0081] The facial feature extraction unit uses the following method to detect the face position using the Dlib library method, obtain key points, extract a 128-dimensional feature vector, and then calculate using the Euler distance formula:

[0082]

[0083] Where x i 、y i are the X and Y coordinates. By calculating the straight-line distance between the two eigenvectors, the smaller the distance, the higher the similarity.

[0084] The Dlib library is a powerful C++ library that provides face detection methods based on HOG and deep learning. It is widely used in machine learning, computer vision, image processing, and linear algebra.

[0085] The voiceprint feature extraction unit uses the Mel-frequency cepstral coefficients for feature extraction, and the calculation formula is:

[0086]

[0087] Where X(k) is the spectrum amplitude of the speech signal, H m (k) Response function of the mth Mel filter (a total of M filters);

[0088] Gaussian mixture model training:

[0089] The probability density function calculation formula is:

[0090]

[0091] Where x is the eigenvector of the Mel-frequency cepstral coefficients, are model parameters (weight, mean, covariance matrix), and C is the number of Gaussian components (usually 32-256).

[0092] Mel-frequency cepstral coefficients can simulate the nonlinear perception of sound frequency by the human ear and are one of the most commonly used features in voiceprint recognition. The Gaussian mixture model fits the speaker's characteristic distribution through the weighted sum of multiple Gaussian distributions.

[0093] Among them, the multimodal fusion and decision module is used to output the decision results of gait behavior, face recognition and voiceprint recognition and perform multimodal feature decision fusion:

[0094] The calculation method of gait behavior decision results is:

[0095] P gait =D(i,j)

[0096] The calculation method for face recognition decision results is:

[0097] P face =dis(X,Y)

[0098] The calculation method for voiceprint recognition decision results is:

[0099] P voice =p(x|λ)

[0100] The multimodal decision fusion formula is:

[0101] P final =(ω1P gait +ω2P face +ω3P voice )×100%

[0102] Where, P final The final result is P gait Output identity probability for gait feature extraction unit, P face The face feature extraction unit outputs the identity probability, P voice The identity probability is output by the voiceprint feature extraction unit, ω1, ω2, ω3 are weight coefficients, P final A value greater than 90% indicates successful recognition.

[0103] By outputting independent decision results for gait behavior, face recognition, and voiceprint recognition, and then performing probabilistic fusion through weighted voting, the fault tolerance of the AI ​​access control dynamic recognition system can be improved. At the same time, multimodal features complement each other, which can significantly improve the accuracy and robustness of identity authentication.

[0104] Among them, the data storage module is divided into a user database unit, an identification record library unit and a system log unit. The user database unit is used to store the biometric information and identity information of registered users, the identification record library unit is used to store the results of each identification, and the system log unit is used to store the system's operating status, error information and maintenance records.

[0105] Through the data storage module, users can query historical data, making it easier for users to understand the recognition records of the AI ​​access control dynamic recognition system. In addition, it can also facilitate troubleshooting and system optimization.

[0106] Among them, the communication and interface module includes a network communication unit and an API interface unit. The network communication unit supports TCP / IP, HTTP, and MQTT protocols to realize data transmission between the system and remote servers or other devices. The API interface unit provides a standard API interface for third-party applications to integrate and call the functions of the access control system.

[0107] Through the network communication unit and API interface unit in the communication and interface module, users can remotely access the AI ​​access control dynamic recognition system through their mobile phones, thereby improving the convenience of using the AI ​​access control dynamic recognition system.

[0108] Among them, the user interaction module is divided into a display unit, a voice interaction unit, and an operation feedback unit. The display unit displays the recognition results through the display screen, the voice interaction unit plays voice prompts through the speaker to guide the user to perform operations or inform the recognition results, and the operation feedback unit gives corresponding feedback based on the user's operations or recognition results.

[0109] Through the user interaction module, users can directly obtain the access control recognition results. At the same time, through the guidance of the user interaction module, users can perform effective recognition, thereby improving the recognition efficiency of the AI ​​access control dynamic recognition system.

[0110] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0111] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. AI access control dynamic recognition system based on multimodal fusion, characterized by: It includes: data acquisition module, data processing module, multimodal fusion and decision module, data storage module, communication and interface module, and user interaction module, among which, The data acquisition module is used to collect biometric information of the user's gait, face, and voiceprint; The data processing module is used to extract features from the collected information; The multimodal fusion and decision module is used to perform multimodal fusion and decision-making on the extracted features; The data storage module is used to store the input biometric information and system logs; The communication and interface module is responsible for data transmission and communication between the system and other devices or systems, and provides an external interface for third-party applications to call; The user interaction module provides an interactive interface between the user and the system and displays the recognition results.

2. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 1 is characterized by: The data acquisition module is divided into a gait acquisition unit, a face acquisition unit and a voiceprint acquisition unit. The gait acquisition unit captures the coordinates of the user's joints when walking through a camera, the face acquisition unit collects the user's face through a camera, and the voiceprint acquisition unit collects the user's voice through a microphone.

3. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 1 is characterized by: The data processing module is divided into a gait feature extraction unit, a face feature extraction unit and a voiceprint feature extraction unit. The extraction method of the gait feature extraction unit is: Normalization of joint point coordinates: Where, P i is the original coordinate of the i-th joint point, P root is the root joint, P hip is the hip joint coordinate, which is subtracted from the root joint coordinate and divided by the distance from the hip joint to the root joint to eliminate the influence of individual height differences on gait; Joint angle calculation: Where p1, p2, and p3 are three joint points, and the angle formed by the three joint points is used to describe the degree of limb bending; Spatiotemporal feature modeling: Using the spatiotemporal graph convolutional network, the calculation formula is: In the formula, the adjacency matrix A k To describe the connection relationship between joints, the adjacency matrix D k Used to normalize the adjacency matrix to avoid numerical instability, weight It is a learnable parameter used to aggregate spatial features. Temporal convolution processes time series through 1D convolution to capture the dynamic changes of gait; Gait sequence analysis and matching: Dynamic time warping is used for calculation, and the calculation formula is as follows: Where d(i, j) is the distance between two gait sequences at time points θ and θ. Dynamic time warping aligns gait sequences through dynamic programming to solve the problem of inconsistent sequence lengths caused by different speeds or step lengths.

4. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 3 is characterized by: The facial feature extraction unit uses the following extraction method: using the Dlib library method to detect the face position, obtain key points, extract 128-dimensional feature vectors, and then use the Euler distance formula to calculate: Where x i 、y i are the X and Y coordinates. By calculating the straight-line distance between the two eigenvectors, the smaller the distance, the higher the similarity.

5. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 3 is characterized by: The extraction method of the voiceprint feature extraction unit is: using Mel frequency cepstral coefficients for feature extraction, and the calculation formula is: Where X(k) is the spectrum amplitude of the speech signal, H m (k) Response function of the mth Mel filter (a total of M filters); Gaussian mixture model training: The probability density function calculation formula is: Where x is the eigenvector of the Mel-frequency cepstral coefficients, are model parameters (weight, mean, covariance matrix), and C is the number of Gaussian components (usually 32-256).

6. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 1 is characterized by: The multimodal fusion and decision module is used to output the decision results of gait behavior, face recognition and voiceprint recognition and perform multimodal feature decision fusion: The calculation method of gait behavior decision results is: P gait =D(i,j) The calculation method for face recognition decision results is: P face =dis(X,Y) The calculation method for voiceprint recognition decision results is: P voice =p(x|λ) The multimodal decision fusion formula is: P final =(ω1P gait +ω2P face +ω3P voice )×100% Where, P final The final result is P gait Output identity probability for gait feature extraction unit, P face The face feature extraction unit outputs the identity probability, P voice The identity probability is output by the voiceprint feature extraction unit, ω1, ω2, ω3 are weight coefficients, P final A value greater than 90% indicates successful recognition.

7. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 1 is characterized by: The data storage module is divided into a user database unit, an identification record library unit and a system log unit. The user database unit is used to store the biometric information and identity information of registered users, the identification record library unit is used to store the results of each identification, and the system log unit is used to store the system's operating status, error information and maintenance records.

8. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 1 is characterized by: The communication and interface module includes a network communication unit and an API interface unit. The network communication unit supports TCP / IP, HTTP, and MQTT protocols to realize data transmission between the system and remote servers or other devices. The API interface unit provides a standard API interface for third-party applications to integrate and call the functions of the access control system.

9. The AI ​​access control dynamic recognition system based on multimodal fusion according to claim 1 is characterized by: The user interaction module is divided into a display unit, a voice interaction unit, and an operation feedback unit. The display unit displays the recognition results through a display screen. The voice interaction unit plays voice prompts through a speaker to guide the user to perform operations or inform the recognition results. The operation feedback unit gives corresponding feedback based on the user's operations or recognition results.

Images