Multi-factor security authentication system and method based on voiceprint recognition and privacy calculation

Through voiceprint recognition and privacy computing technology, combined with thin-ResNet-34 network and ASPE homomorphic encryption, efficient and secure access authentication and data interaction between intelligent interactive devices and virtual digital humans in the metaverse are realized, and the existing technology is solved The problems of privacy data leakage and inefficient access authentication improve the security and computing efficiency of authentication.

CN119030720AInactive Publication Date: 2024-11-26BEIJING UNIV OF POSTS & TELECOMM
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410964205.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing identity authentication technology cannot meet the needs of efficient and secure access authentication and data interaction between intelligent interactive devices and virtual digital humans in the metaverse, and there is a risk of privacy data leakage.

Method used

Using a multi-factor security authentication system based on voiceprint recognition and privacy computing, voiceprint recognition is performed through thin-ResNet-34 network architecture, voiceprint features are chaotic and chaotic encryption using ASPE homomorphic encryption algorithm, and dual-fold combination with cryptocode module Multi-factor certification for insurance.

Benefits of technology

It improves the access authentication security between smart interactive devices and virtual digital humans and the privacy of data interaction, reduces the consumption of computing resources, and ensures the legitimacy of user identities and the security of private information in the metaverse.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119030720A_ABST
    Figure CN119030720A_ABST
Patent Text Reader

Abstract

The invention provides a multi-factor security authentication system and method based on voiceprint recognition and privacy calculation. The multi-factor security authentication system comprises a voiceprint recognition module, a privacy calculation module and a multi-factor authentication module, a thin-ResNet-34 network architecture is adopted, and the method comprises the following steps: carrying out The method comprises the following steps of: firstly, performing chaotic scrambling on the scrambling of voice by using a cosine similarity as an evaluation index and adopting an ASPE homomorphic encryption algorithm, and then encrypting the scrambled scrambling by adopting the ASPE and storing the scrambled scrambling in a database; according to the method, the biological characteristics of the intelligent interaction device and the virtual digital human are extracted, and the legality of the identities of the intelligent interaction device and the virtual digital human when the intelligent interaction device and the virtual digital human access the meta universal world is ensured; the ASPE encryption technology is used, identities of the two parties and database entries are matched in a ciphertext state, then attacks of malicious users are resisted, and leakage of privacy information of intelligent interaction equipment and virtual digital humans in the meta universe is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of voiceprint recognition, and in particular to a multi-factor security authentication system and method based on voiceprint recognition and privacy computing. Background Art

[0002] In recent years, with the support of new-generation artificial intelligence technologies such as digital twins and blockchain, the metaverse has broken the barriers between reality and virtuality, broken through the constraints of time and space, and gradually entered the public eye. The concept of the metaverse describes a huge virtual space that can map the real world but is independent of the real world. Subsequently, the metaverse continued to evolve and has gradually become a new Internet form that is "multi-dimensional, full-sensory, and immersive." Users can use interactive technologies such as virtual reality, augmented reality, and machine learning to interact socially with the virtual environment of the metaverse.

[0003] However, as a virtual social platform, the Metaverse collects a large amount of personal information, threatening user privacy. At the same time, in the construction of the Metaverse, communication between intelligent interactive devices and virtual digital people becomes the key. However, due to the high latency of existing rendering equipment and the requirements of immersion, low latency and other factors, the basic equipment can no longer meet the needs of the development of the Metaverse. Therefore, how to efficiently and securely realize the access authentication and data interaction between intelligent interactive devices and virtual digital people has become a key issue that needs to be solved urgently.

[0004] At present, the protection of Metaverse data privacy is mainly focused on the research of basic technologies, including identity authentication and other technologies. At present, there are many studies on identity authentication in academia, such as:

[0005] 1. Identity authentication based on static password:

[0006] ①Liu et al. proposed a new two-factor authentication scheme based on one-time password (OTP);

[0007] The scheme uses mutual authentication, which is highly secure and has low computational cost. It extends key authentication by using an infinite forward hash chain and uses two one-way hash functions, one for message encryption and the other for creating a core hash chain.

[0008] However, this solution is vulnerable to man-in-the-middle attacks and social engineering attacks, which may intercept or fraudulently obtain the OTP, affecting the user experience and increasing inconvenience.

[0009] ②NYang et al. proposed two visual authentication protocols:

[0010] One is a one-time password protocol, and the other is a password-based authentication protocol.

[0011] The system model consists of four different entities (or actors), which are the user, the smartphone, the user terminal, and the server. The system appears to be very secure and can prevent attacks from keyloggers;

[0012] However, scanning using terminals and smartphones may cause inconvenience to users; smartphones storing private keys may not be protected.

[0013] ③Bicakci et al. proposed a mobile authentication protocol called mobile-id to prevent man-in-the-middle attacks.

[0014] The success of the protocol depends on the assumption that it runs on a mobile medium like an ID card, that it is implemented in a secure medium with a digital signature and is only accessed by the mobile-ID;

[0015] However, most of these assumptions are difficult to guarantee, such as ensuring that the phone and website agree to trust an unknown intermediate server to use the protocol. In addition, the selected MIS can also be compromised and impersonated.

[0016] ④Kirda et al. proposed a solution in the form of a browser extension called Antiphish, which aims to protect users from phishing attacks.

[0017] Anti-phishing applications are integrated into web browsers to track sensitive information of users, if the application is not trusted, it checks the website address and then blocks the information from being passed to it. Hein et al. introduced the Secure Block Device (SDB), which implements cryptographic methods to ensure integrity and confidentiality, and the software used is based on a combination of Merkle-Tree and Authenticated Encryption (AE);

[0018] However, this approach requires users to protect the symmetric master key, and the system is vulnerable to denial of service attacks.

[0019] ⑤ Ichikawa et al. proposed a method to encrypt the communication between the server and the client using a key provided by the server by adding an additional security layer to the work done before using a virtual machine;

[0020] However, this method relies on only trusted and registered applications on the list to access sensitive files.

[0021] 2. Identity authentication based on dynamic password:

[0022] ①Ahmed et al. proposed a dynamic peer authentication protocol and developed a secure mutual authentication method that is not subject to Diffie-Hellman restrictions and is immune to basic or complex known attacks; and the protocol utilizes multi-factor authentication of username, password and one-time password (OTP).

[0023] However, this method relies on user passwords and OTPs, which has timeliness and network delay issues, affects authentication efficiency, and has high computational costs.

[0024] ②Hou et al. proposed WiASK, a device-free WiFi sensor-enabled authentication system that explores keystroke dynamics; more specifically, WiASK leverages existing WiFi infrastructure to capture users’ keystrokes as they enter a pre-defined, easy-to-remember string.

[0025] But rather than focusing on the string of characters itself, which is vulnerable to password attacks, WiASK interprets the way they are entered (i.e., keystroke dynamics) as user identity based on biometric correlations between them.

[0026] ③Paliwal et al. designed a probabilistic key exchange protocol suitable for vehicle ad hoc networking environments; the proposed protocol is lightweight and more efficient than bilinear pairings, modular exponentiation, and elliptic curve cryptography. At the same time, the authors can prove the security of the proposed system through automatic verification of Internet security protocols and applications and BAN logic.

[0027] However, the protocol relies heavily on randomness and computing resources, which may lead to performance issues; it is vulnerable to man-in-the-middle attacks during transmission, and has weak resistance to quantum computing. In addition, the complexity of the protocol increases the difficulty of implementation and management.

[0028] ④Liu et al. proposed a dynamic identity authentication method based on the risk UCON model. When authenticating users, their access rights are considered based on the risk level, except for password authentication. A risk assessment method is also proposed, and blockchain technology is used in the scheme to provide a reliable process for risk assessment and authorization. This scheme is an improvement on traditional identity authentication, with higher continuity and variability.

[0029] However, the accuracy of risk assessment depends on a large amount of real-time data and complex analysis algorithms, which may lead to high computational overhead, high complexity, and difficulty in management, affecting system performance. At the same time, over-reliance on automated decision-making may result in legitimate users being denied access or malicious users being authenticated.

[0030] With the development of other artificial intelligence technologies, biometric authentication provides a broader development prospect for identity authentication technology, such as:

[0031] ①Yang et al. proposed a new visual speaker verification scheme based on deep convolutional neural networks. The network consists of two parts, namely the basic feature extraction network (FFE-Net) and the typical lip feature extraction and classification network (RC-Net). FFE-Net focuses on dynamic lip movements to provide basic information for speaker verification. RC-Net extracts high-level lip features that distinguish human imposters while capturing the customer's speaking style. Finally, a multi-task learning scheme was designed and the proposed network was trained end-to-end.

[0032] ② In addition, Xu et al. proposed a dual-offload cross-domain biometric authentication scheme based on blockchain technology. First, the random key for biometric authentication is extracted using fuzzy extraction technology, thereby solving the problem of biometric leakage due to permanent unavailability. Secondly, an unalterable blockchain is used to store biometric public information, solving the threat that fuzzy extraction technology is vulnerable to active attacks. Finally, based on the distributed storage function and the joint blockchain architecture, cross-domain biometric authentication of users in local and remote environments is realized.

[0033] ③At the same time, Annadurai et al. proposed a new technology for data security transmission and intruder detection of biometric authentication system based on classification feature extraction. Intruders are detected by collecting biometric databases of smart buildings based on the Internet of Things. These biometric data are denoised, smoothed and normalized. The processed data features are extracted using kernel-based principal component analysis. Then, the processed features are classified using the convolutional VGG-16 network structure. Finally, the deterministic trust transfer protocol (DTTP) is used to protect the entire network.

[0034] ④Rahman et al. proposed a parametric fingerprint recognition method based on minutiae nodes to generate secure biometric templates from fingerprint images. This method extracts minutiae features from fingerprint images and uses their related parameters to hide the real minutiae information. A minutiae-based transformation algorithm is used to generate a secure vault by adding additional foil strips (pseudo-minutiae points) and reshaping the real minutiae points, thereby producing a cancelable fingerprint template.

[0035] The above-mentioned biometric authentication is an identity authentication technology that is becoming increasingly mature and has been applied in various fields. However, due to the particularity of the metaverse, the existing biometric authentication technology can no longer meet the development needs of the metaverse, posing a great threat to users' privacy data. At the same time, a large number of intelligent interactive devices in the metaverse have put forward higher requirements on computing resources and computing efficiency. How to achieve efficient authentication has also become the focus of future identity authentication development. Summary of the invention

[0036] In order to solve the above technical problems, the present invention provides a multi-factor security authentication system and method based on voiceprint recognition and privacy computing. By studying the architectural design of voiceprint recognition and privacy computing, the broad prospects of biometrics in the metaverse are explored. At the same time, a multi-factor security authentication scheme based on voiceprint recognition and privacy computing is used to improve the access authentication and data interaction capabilities of intelligent interactive devices and virtual digital humans in the metaverse.

[0037] A multi-factor security authentication system and method based on voiceprint recognition and privacy computing, wherein:

[0038] A multi-factor security authentication system based on voiceprint recognition and privacy computing, including: a voiceprint recognition module, a privacy computing module and a multi-factor authentication module;

[0039] The voiceprint recognition module: adopts a thin-ResNet-34 network architecture; uses the output before the last fully connected layer of the thin-ResNet-34 network architecture as the embedding output of the speech, and uses cosine similarity as an evaluation index. If the cosine similarity is greater than or equal to 0.8, the voiceprint recognition module is passed;

[0040] As an example, the thin-ResNet-34 network architecture greatly reduces the number of model parameters, making it easier to deploy in practical applications.

[0041] The privacy computing module: adopts the ASPE homomorphic encryption algorithm to first scramble the speech embedding, and then uses the ASPE to encrypt the scrambled embedding and store it in the database;

[0042] As an example, the ASPE homomorphic encryption algorithm can ensure that the encrypted voiceprint features can only be used for voiceprint recognition of this system. Even if the database is leaked, the attacker cannot obtain any useful information from the data. At the same time, the database administrator of this system cannot use the data beyond his or her authority.

[0043] As an example, ASPE refers to asymmetric calar-product-preserving encryption based on chaotic scrambling and semi-tensor product optimization.

[0044] The multi-factor authentication module: uses the cryptocode module to encrypt the user's password and stores the encrypted ciphertext in the database;

[0045] As an example, the cryptocode module can ensure that the encrypted ciphertext can only be used for ciphertext authentication of this system. Even if the database is leaked, the attacker cannot obtain the real user password plaintext. During verification, the user can only successfully log in to the system by passing both the voiceprint recognition module and the ciphertext authentication module.

[0046] The multi-factor security authentication method based on voiceprint recognition and privacy computing includes:

[0047] Step 1: Voiceprint recognition processing;

[0048] ①The first stage of the thin-ResNet-34 network architecture is to extract features from the input spectrogram;

[0049] As an example, the thin-ResNet-34 network architecture is as follows: based on ResNet-34, the number of channels in each residual block is reduced to obtain a 34-layer thin-ResNet model.

[0050] The second part of the thin-ResNet-34 network architecture uses GhostVLAD to aggregate frame-level descriptors into a single utterance-level vector.

[0051] As an example, GhostVLAD is proposed based on NetVLAD and is used to automatically reduce the weight of blurred images during face recognition clustering.

[0052] As an example, the present invention uses GhostVLAD to automatically calculate weights and automatically assigns noise data to a "Ghost" class, which will not contribute to the final feature representation, thereby reducing the weights of noise and bad parts when aggregating frame-level descriptors, ultimately achieving the purpose of removing speech data with large noise or severe damage.

[0053] ③ The voiceprint recognition module randomly cuts each voice into a spectrogram of 2.5s in length, and uses the overlapping segmentation method to frame the voice signal with a frame length of 25ms and a frame shift of 10ms. It finally generates a spectrogram of size 257×250 (frequency×time), and normalizes it by subtracting the mean and dividing it by the standard deviation of all frequency components in a single time step; the spectrogram is input into the thin-ResNet model to obtain a size of R 1×T / 32×512 In the feature aggregation stage, GhostVLAD layer is used to perform feature aggregation along the time axis, and the frame-level descriptors are aggregated into a fixed-length speech-level vector; the last fully connected layer is used to perform dimensionality reduction (the dimension is 512) and output the voiceprint embedding code;

[0054] As an example, the voiceprint recognition module uses AM-Softmax as the loss function; AM-Softmax improves the Softmax function by introducing an angular interval to make the embedding distribution of the two classes more compact. The formula is designed as follows:

[0055]

[0056] Among them, N is the total number of test samples, w0 and w1 are the weight vectors of the two types of data in the voiceprint recognition network model, and y i is the true category label of the i-th sample, which can be 0 or 1, x i is the feature vector of the i-th sample in the network model, α is a scaling factor, m is the cosine additive interval, and are the normalized representations of w0, w1 and x respectively;

[0057] Step 2: Privacy computing settings;

[0058] The ASPE method based on chaotic scrambling and semi-tensor product optimization is adopted; while ensuring the security of the user's voiceprint feature vector storage, it also meets the computability of the ciphertext feature vector and supports the cosine similarity calculation in the ciphertext state of the voiceprint feature vector;

[0059] The introduction of chaos increases the complexity of attacks and greatly improves the security of encryption;

[0060] Finally, the ASPE method optimized by semi-tensor product has the characteristics of small key size and wide adaptability, which effectively solves the difficulty of storing large-dimensional key matrices after massive user access;

[0061] As an example, the present invention mainly improves the security and storage efficiency of the ASPE method. When voiceprint collection and registration, the feature vector is first scrambled, and then an encryption matrix of appropriate size is selected to encrypt the feature using the semi-tensor product, and the encrypted vector is stored; when logging in and verifying, the inner product operation is directly performed on the two ciphertext vectors;

[0062] Step 3: Multi-factor authentication operation;

[0063] The multi-factor authentication module uses a double insurance method to ensure the accuracy and security of the user's identity. It mainly uses two methods: voiceprint recognition authentication and password comparison authentication. When the user authenticates, he must pass the double verification of these two authentication methods to obtain the verification;

[0064] Beneficial effects of the present invention:

[0065] The present invention uses voiceprint recognition technology to extract the biometric features of intelligent interactive devices and virtual digital humans to ensure the legitimacy of their identities when they access the metaverse world; at the same time, it uses ASPE encryption technology to match the identities and database entries of both parties in a ciphertext state, thereby resisting attacks from malicious users and avoiding the leakage of privacy information of intelligent interactive devices and virtual digital humans in the metaverse.

[0066] Based on the sensitivity characteristics of user privacy data, chaotic scrambling and semi-tensor product are introduced into the privacy computing module, and the voiceprint data is encrypted using the ASPE algorithm to accelerate the registration speed of virtual digital humans and intelligent interactive devices and the encryption speed of privacy information, thereby reducing the consumption of computing resources and ensuring the relative security of the registration information of virtual users and intelligent interactive devices in the digital space.

[0067] In order to address the access authentication and data interaction issues between intelligent interactive devices and virtual digital people in the Metaverse, a lightweight multi-factor security authentication solution is constructed based on the user-password login system, voiceprint recognition system and privacy computing technology. This ensures the authentication privacy, security and communication efficiency of both parties, while providing strong guarantees for the reliability, trustworthiness and availability of Metaverse data. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic diagram of the overall structure of the multi-factor security authentication system based on voiceprint recognition and privacy computing of the present invention.

[0069] Figure 2 It is a schematic diagram of the voiceprint recognition processing flow of the multi-factor security authentication method based on voiceprint recognition and privacy computing of the present invention.

[0070] Figure 3 This is a schematic diagram of the privacy computing setting process of the multi-factor security authentication method based on voiceprint recognition and privacy computing of the present invention.

[0071] Figure 4 The figure is a schematic diagram of the multi-factor authentication operation flow of the multi-factor security authentication method based on voiceprint recognition and privacy computing of the present invention. DETAILED DESCRIPTION

[0072] Below, reference Figures 1 to 4 As shown, a multi-factor security authentication system and method based on voiceprint recognition and privacy computing, wherein:

[0073] A multi-factor security authentication system based on voiceprint recognition and privacy computing, including: a voiceprint recognition module, a privacy computing module and a multi-factor authentication module;

[0074] The voiceprint recognition module: adopts a thin-ResNet-34 network architecture; uses the output before the last fully connected layer of the thin-ResNet-34 network architecture as the embedding output of the speech, and uses cosine similarity as an evaluation index. If the cosine similarity is greater than or equal to 0.8, the voiceprint recognition module is passed;

[0075] As an example, the thin-ResNet-34 network architecture greatly reduces the number of model parameters, making it easier to deploy in practical applications.

[0076] Furthermore, the thin-ResNet-34 network architecture includes: a feature extraction unit and an aggregation unit;

[0077] The feature extraction unit: uses a shared CNN to encode the spectrogram and extract frame-level features;

[0078] The aggregation unit: aggregates all local descriptors into a single compact representation of any length, thereby achieving the three characteristics that an ideal voiceprint recognition model should have;

[0079] As an example, the three characteristics are:

[0080] ① Receive speech of any length as input and generate a fixed-length speech-level descriptor;

[0081] ② The output descriptor should be compact (i.e., low-dimensional) and require little memory to facilitate efficient storage and retrieval;

[0082] ③ The output descriptors should be distinct so that the distance between descriptors of different speakers is greater than the distance between descriptors of the same speaker.

[0083] The privacy computing module: adopts the ASPE homomorphic encryption algorithm to first scramble the speech embedding, and then uses the ASPE to encrypt the scrambled embedding and store it in the database;

[0084] As an example, the ASPE homomorphic encryption algorithm can ensure that the encrypted voiceprint features can only be used for voiceprint recognition of this system. Even if the database is leaked, the attacker cannot obtain any useful information from the data. At the same time, the database administrator of this system cannot use the data beyond his or her authority.

[0085] As an example, ASPE refers to asymmetric calar-product-preserving encryption based on chaotic scrambling and semi-tensor product optimization.

[0086] The multi-factor authentication module: uses the cryptocode module to encrypt the user's password and stores the encrypted ciphertext in the database;

[0087] As an example, the cryptocode module can ensure that the encrypted ciphertext can only be used for ciphertext authentication of this system. Even if the database is leaked, the attacker cannot obtain the real user password plaintext. During verification, the user can only successfully log in to the system by passing both the voiceprint recognition module and the ciphertext authentication module.

[0088] The multi-factor security authentication method based on voiceprint recognition and privacy computing includes:

[0089] Step 1: Voiceprint recognition processing;

[0090] ①The first stage of the thin-ResNet-34 network architecture is to extract features from the input spectrogram;

[0091] As an example, the thin-ResNet-34 network architecture is as follows: based on ResNet-34, the number of channels in each residual block is reduced to obtain a 34-layer thin-ResNet model.

[0092] The second part of the thin-ResNet-34 network architecture uses GhostVLAD to aggregate frame-level descriptors into a single utterance-level vector.

[0093] As an example, GhostVLAD is proposed based on NetVLAD and is used to automatically reduce the weight of blurred images during face recognition clustering.

[0094] As an example, the present invention uses GhostVLAD to automatically calculate weights and automatically assigns noise data to a "Ghost" class, which will not contribute to the final feature representation, thereby reducing the weights of noise and bad parts when aggregating frame-level descriptors, ultimately achieving the purpose of removing speech data with large noise or severe damage.

[0095] ③ The voiceprint recognition module randomly cuts each voice into a spectrogram of 2.5s in length, and uses the overlapping segmentation method to frame the voice signal with a frame length of 25ms and a frame shift of 10ms. It finally generates a spectrogram of size 257×250 (frequency×time), and normalizes it by subtracting the mean and dividing it by the standard deviation of all frequency components in a single time step; the spectrogram is input into the thin-ResNet model to obtain a size of R 1×T / 32×512In the feature aggregation stage, GhostVLAD layer is used to perform feature aggregation along the time axis, and the frame-level descriptors are aggregated into a fixed-length speech-level vector; the last fully connected layer is used to perform dimensionality reduction (the dimension is 512) and output the voiceprint embedding code;

[0096] As an example, the voiceprint recognition module uses AM-Softmax as the loss function; AM-Softmax improves the Softmax function by introducing an angular interval to make the embedding distribution of the two classes more compact. The formula is designed as follows:

[0097]

[0098] Among them, N is the total number of test samples, w0 and w1 are the weight vectors of the two types of data in the voiceprint recognition network model, and y i is the true category label of the i-th sample, which can be 0 or 1, x i is the feature vector of the i-th sample in the network model, α is a scaling factor, m is the cosine additive interval, and are the standardized representations of w0, w1 and x respectively; the main steps of voiceprint recognition processing are as follows Figure 2 shown.

[0099] Step 2: Privacy computing settings;

[0100] The ASPE method based on chaotic scrambling and semi-tensor product optimization is adopted; while ensuring the security of the user's voiceprint feature vector storage, it also meets the computability of the ciphertext feature vector and supports the cosine similarity calculation in the ciphertext state of the voiceprint feature vector;

[0101] The introduction of chaos increases the complexity of attacks and greatly improves the security of encryption;

[0102] Finally, the ASPE method optimized by semi-tensor product has the characteristics of small key size and wide adaptability, which effectively solves the difficulty of storing large-dimensional key matrices after massive user access;

[0103] As an example, the present invention mainly improves the security and storage efficiency of the ASPE method. When voiceprint collection and registration, the feature vector is first scrambled, and then an encryption matrix of appropriate size is selected to encrypt the feature using the semi-tensor product, and the encrypted vector is stored; when logging in and verifying, the inner product operation is directly performed on the two ciphertext vectors; the process of privacy calculation setting is as follows: Figure 3 As shown;

[0104] Step 3: Multi-factor authentication operation;

[0105] The multi-factor authentication module uses a double insurance method to ensure the accuracy and security of the user's identity. It mainly uses voiceprint recognition authentication and password comparison authentication. When the user authenticates, he must pass the double verification of these two authentication methods to obtain the verification. The process of multi-factor authentication operation is as follows: Figure 4 shown.

[0106] In order to better illustrate the principle basis of the present invention, the principle is briefly described as follows through the architectural design ideas:

[0107] Voiceprint recognition authentication is a biometric recognition technology that uses a voiceprint recognition module to analyze and compare the user's voice characteristics to confirm the user's identity;

[0108] Password matching authentication is a quick authentication method. The user enters a preset password, which is then encrypted using an encryption algorithm and stored in the database.

[0109] When a user logs in, he / she enters the pre-set password, and then uses the decryption algorithm to decrypt the password stored in the database, and compares it with the entered password to confirm the user's identity. Although this authentication method is relatively common, in order to improve security, the password is hidden. The encrypted ciphertext is stored in the database to prevent attackers from breaking into the database and preventing the password from being cracked or stolen;

[0110] This means that when users log in to the Metaverse virtual platform, they not only need to provide the correct password, but also need to pass the comparison of voiceprint recognition technology; this dual authentication method can greatly improve the security of the solution and protect the user's account and information security; at the same time, this authentication method can also improve user credibility and satisfaction; by combining these two authentication methods, they can give full play to their respective advantages and improve security and user experience.

[0111] As an example, voiceprint recognition is a biometric recognition technology that identifies an individual based on their voice characteristics. It works by analyzing the sound waveform, frequency, duration and other features to extract individual voice feature information, and then compares it with the information in a pre-established voiceprint database to determine the identity.

[0112] As an example, the voiceprint recognition system mainly consists of model training, voiceprint comparison, and voiceprint recognition. Model training: feature extraction is performed by training the voice database, and the model library is finally trained. When training the model, the correct data input layer size needs to be set to adapt to the voice length used. The data input layer sets the shape according to the frequency and length of the sound data set. If other voice lengths are used, the input layer size needs to be adjusted accordingly. After each round of training, a model evaluation is required to observe the accuracy of the model and understand its situation. Voiceprint comparison: use speech recognition technology to identify and analyze two voices, compare their sound features, and use the set threshold to determine whether they are from the same person. If it is greater than the threshold, it is the same person, otherwise, it is not the same person. This program needs to encapsulate an audio feature extraction function, and output the classification results and audio feature values ​​in the model at the same time for voiceprint recognition. Voiceprint recognition: extract features from the voiceprint to be identified, and match and compare with the data in the voiceprint database to obtain the recognition result. The encapsulation function is used to load the voice data of the registered users in the voiceprint library into the system for voiceprint recognition and verification operations. Voiceprint recognition is a technology that determines the identity of the speaker by comparing the characteristics of the sound signal. When a user wants to log in, the system needs to obtain his / her voice and compare it with the registered voice. If the two voices are highly similar, the identity of the user can be verified and the information data provided by the user during registration can be recognized.

[0113] The ASPE algorithm is a new encryption scheme that is specifically used to perform dot product operations on encrypted data. It is a scheme based on asymmetric encryption technology that aims to protect data privacy while allowing dot product operations to be performed on encrypted data. This has important application value in various data analysis and machine learning tasks and can be widely used in cloud computing and multi-party computing scenarios.

[0114] As an example, the ASPE method includes three parts: key generation, data encryption and data decryption:

[0115] (1) Key generation: Select an appropriate public key algorithm to generate a pair of public key PK and private key SK. The public key is used to encrypt data, and the private key is used to decrypt data.

[0116] (2) Data encryption: Use the public key to encrypt the data vector. Assume that the original data vector is X = {x1, x2, ..., x n}, the encrypted data vector is C = {c1, c2, ..., c n}. The encryption process can be expressed as c i =E PK(x1). The encrypted data vector is obtained by encrypting the data vector directly, and then the dot product operation is directly performed on the encrypted data vector. The result of the homomorphic dot product operation is the encrypted dot product value. The operation process can be expressed as

[0117] (3) Data decryption: Use the private key SK to decrypt the dot product operation result to obtain the dot product value of the original data vector. The decryption process is expressed as dot = D SK (c dot ).

[0118] This encryption scheme involves operations such as vector addition and dot product during the encryption and decryption processes, which can ensure that data remains encrypted during transmission and processing. It is simple, efficient, and secure, and can effectively protect data privacy.

[0119] The present invention designs a multi-factor security authentication scheme based on voiceprint recognition and privacy computing in the virtual environment of the Metaverse, so as to provide access authentication and data interaction for virtual digital people and intelligent interactive devices, and ensure the security of private data. At the same time, the advantages of chaotic scrambling and semi-tensor product technology are utilized to reduce the consumption of computing resources and improve the efficiency of the entire security authentication scheme. It effectively alleviates the high latency problems currently faced by the Metaverse in access authentication, data privacy and device rendering, lays a certain technical foundation for the Metaverse to create a safe virtual reality environment, promotes the further development of the Metaverse, accelerates its pace of implementation, and makes full preparations for the digital intelligent society where opportunities and risks coexist.

[0120] The above are only preferred embodiments of the present invention. It should be understood that the description of the above embodiments is only used to help understand the method and core ideas of the present invention, and is not used to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, etc. made within the ideas and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-factor security authentication system based on voiceprint recognition and privacy computing, characterized by: include: Voiceprint recognition module, privacy computing module and multi-factor authentication module; The voiceprint recognition module: adopts a thin-ResNet-34 network architecture; uses the output before the last fully connected layer of the thin-ResNet-34 network architecture as the embedding output of the speech, and uses cosine similarity as an evaluation index. If the cosine similarity is greater than or equal to 0.8, the voiceprint recognition module is passed; The privacy computing module: adopts the ASPE homomorphic encryption algorithm to first perform chaotic scrambling on the speech embedding, and then uses the ASPE to encrypt the scrambled embedding and store it in the database; The multi-factor authentication module: uses a cryptocode module to encrypt the user's password and stores the encrypted ciphertext in a database.

2. The multi-factor security authentication system based on voiceprint recognition and privacy computing according to claim 1 is characterized in that: The thin-ResNet-34 network architecture includes: a feature extraction unit and an aggregation unit; The feature extraction unit: uses a shared CNN to encode the spectrogram and extract frame-level features; The aggregation unit aggregates all local descriptors into a single compact representation of any length, thereby achieving the three characteristics that an ideal voiceprint recognition model should have.

3. The multi-factor security authentication system based on voiceprint recognition and privacy computing according to claim 1 is characterized in that: The three characteristics are: ① Receive speech of any length as input and generate a fixed-length speech-level descriptor; ② The output descriptor should be compact and require little memory for efficient storage and retrieval; ③ The output descriptors should be distinct so that the distance between descriptors of different speakers is greater than the distance between descriptors of the same speaker.

4. The multi-factor security authentication system based on voiceprint recognition and privacy computing according to claim 1 is characterized in that: The ASPE homomorphic encryption algorithm can ensure that the encrypted voiceprint features can only be used for voiceprint recognition of this system. Even if the database is leaked, the attacker cannot obtain any useful information from these data. At the same time, the database administrator of this system cannot use the data beyond his or her authority. The ASPE refers to: asymmetric scalar-product-preserving encryption based on chaotic scrambling and semi-tensor product optimization.

5. The multi-factor security authentication system based on voiceprint recognition and privacy computing according to claim 1 is characterized in that: The cryptocode module can ensure that the encrypted ciphertext can only be used for ciphertext authentication of this system. Even if the database is leaked, the attacker cannot obtain the real user password plaintext. During verification, the user can only successfully log in to the system if he passes the voiceprint recognition module and the multi-factor authentication module at the same time.

6. A multi-factor security authentication method based on voiceprint recognition and privacy computing, characterized in that: include: Step 1: Voiceprint recognition processing; ①The first stage of the thin-ResNet-34 network architecture is to extract features from the input spectrogram; The second part of the thin-ResNet-34 network architecture uses GhostVLAD to aggregate frame-level descriptors into a single utterance-level vector. ③ The voiceprint recognition module randomly cuts each voice into a spectrogram of 2.5s in length, and uses the overlapping segmentation method to frame the voice signal with a frame length of 25ms and a frame shift of 10ms. It finally generates a spectrogram of size 257 frequencies × 250 times, and normalizes it by subtracting the mean and dividing it by the standard deviation of all frequency components in a single time step; the spectrogram is input into the thin-ResNet model to obtain a size of R 1×T / 32×512 In the feature aggregation stage, GhostVLAD layer is used to perform feature aggregation along the time axis, and the frame-level descriptors are aggregated into a fixed-length speech-level vector; the last fully connected layer is used to perform dimensionality reduction operation with a dimension of 512 and output the voiceprint embedding code; Step 2: Privacy computing settings; The ASPE method based on chaotic scrambling and semi-tensor product optimization is adopted; while ensuring the security of the user's voiceprint feature vector storage, it also meets the computability of the ciphertext feature vector and supports the cosine similarity calculation in the ciphertext state of the voiceprint feature vector; The introduction of chaos increases the complexity of attacks and greatly improves the security of encryption; Finally, the ASPE method optimized by semi-tensor product has the characteristics of small key size and wide adaptability, which effectively solves the difficulty of storing large-dimensional key matrices after massive user access; Step 3: Multi-factor authentication operation; The multi-factor authentication module adopts a double insurance method to ensure the accuracy and security of the user's identity; it mainly adopts voiceprint recognition authentication and password comparison authentication. When the user authenticates, he must pass the double verification of these two authentication methods to obtain the verification.

7. The multi-factor security authentication method based on voiceprint recognition and privacy computing according to claim 6 is characterized in that: The thin-ResNet-34 network architecture is as follows: based on ResNet-34, the number of channels in each residual block is reduced, thereby obtaining a 34-layer thin-ResNet model.

8. The multi-factor security authentication method based on voiceprint recognition and privacy computing according to claim 6 is characterized in that: The GhostVLAD is proposed based on NetVLAD and is used to automatically reduce the weight of blurred images during face recognition clustering.

9. The multi-factor security authentication method based on voiceprint recognition and privacy computing according to claim 8 is characterized in that: GhostVLAD is used to automatically calculate weights and automatically assign noisy data to a "Ghost" class, which will not contribute to the final feature representation, thereby reducing the weight of noise and bad parts when aggregating frame-level descriptors, ultimately achieving the purpose of removing noisy or severely damaged speech data.

10. The multi-factor security authentication method based on voiceprint recognition and privacy computing according to claim 6 is characterized in that: The voiceprint recognition module adopts AM-Softmax as the loss function; AM-Softmax improves the Softmax function by introducing an angular interval to make the embedding distribution of the two classes more compact. The formula is designed as follows: Among them, N is the total number of test samples, w0 and w1 are the weight vectors of the two types of data in the voiceprint recognition network model, and y i is the true category label of the i-th sample, which takes the value 0 or 1, x i is the feature vector of the i-th sample in the network model, α is a scaling factor, m is the cosine additive interval, and are the normalized representations of w0, w1 and x respectively.

Citation Information

Patent Citations

  • Internet-of-things-oriented service privacy protection voiceprint recognition method and system, and mobile terminal

    CN110166424A

  • Joint compression encryption method for multi-type spatial-temporal data in wireless sensor network

    CN112055325A

  • Control method of intelligent terminal, man-machine interaction equipment and intelligent terminal

    CN113489628A

  • Image encryption and decryption method based on two-dimensional discrete chaotic mapping and matrix semi-tensor product

    CN114401352A

  • Voiceprint verification method and device

    CN115083420A