Privacy protection voiceprint recognition reasoning method and system based on fully homomorphic encryption technology

By using fully homomorphic encryption technology to encrypt and infer voiceprint data in the voiceprint recognition system, the problem of privacy leakage risks and reduced inference speed in the voiceprint recognition system when processing and storing voiceprint data is solved, and efficient and accurate voiceprint recognition and privacy protection are achieved.

CN119851669AActive Publication Date: 2025-04-18XI'AN PETROLEUM UNIVERSITY
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510019879.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-01-07
Publication Date
2025-04-18
Estimated Expiration
2045-01-07

AI Technical Summary

Technical Problem

The existing voiceprint recognition system has the risk of privacy leakage when processing and storing voiceprint data. In a fully homomorphic encryption environment, the inference process of the deep learning model is complex and the calculation overhead is large, resulting in a reduced inference speed.

Method used

The privacy-protected voiceprint recognition inference method based on all homomorphic encryption technology is adopted. The original voiceprint data is collected from the preset voiceprint database, and the voiceprint data is preprocessed to form a voiceprint data set, and input it into the voiceprint recognition model to be trained for training, and the encryption model parameters and voiceprint data to be identified are generated for encryption processing. Finally, the voiceprint ciphertext data is calculated and reasoned based on the homomorphic encryption characteristics.

Benefits of technology

It improves the calculation accuracy in a fully homomorphic encryption environment, reduces resource consumption, improves inference efficiency, and effectively protects user privacy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119851669A_ABST
    Figure CN119851669A_ABST
Patent Text Reader

Abstract

The invention discloses a privacy protection voiceprint recognition reasoning method based on a fully homomorphic encryption technology, and the method comprises the steps: collecting a plurality of pieces of original voiceprint data from a preset voiceprint database, and carrying out the preprocessing of the original voiceprint data, and then forming a voiceprint data set; inputting the voiceprint data set into a voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model so as to generate encryption model parameters; inputting the encryption model parameters and voiceprint data to be recognized into an encryptor to obtain voiceprint ciphertext data; and performing computational reasoning on the voiceprint ciphertext data according to homomorphic encryption characteristics. In this way, the voiceprint data to be recognized are recognized through the voiceprint recognition model, so that the calculation accuracy in the fully homomorphic encryption environment is improved, resource consumption can be reduced, and the reasoning efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of information security, and specifically relates to a privacy-preserving voiceprint recognition reasoning method and system based on fully homomorphic encryption technology. Background Art

[0002] Against the backdrop of the rapid development of information technology, voiceprint recognition technology, as a biometric recognition technology, has gradually attracted widespread attention. Voiceprint recognition can achieve identity authentication, user identification, and security authentication by analyzing the characteristics of individual sound waves. According to market research, voiceprint recognition technology is becoming increasingly popular in many fields such as finance, security, and medical care. However, with the increasing awareness of data privacy protection, how to ensure the privacy and security of users while performing voiceprint recognition has become an important challenge for the development of this technology.

[0003] Currently, voiceprint recognition technology mainly relies on deep learning models, such as convolutional neural networks (CNN) and recurrent neural networks (RNN), to process and analyze voiceprint data. These models have made significant progress in recognition accuracy and robustness. However, when processing raw voiceprint data, traditional voiceprint recognition systems usually face the risk of privacy leakage when users' voiceprint data is vulnerable to attacks during storage and processing, and there is a risk of theft or abuse. Existing encryption technologies are mostly static encryption and cannot support direct calculation of encrypted data, resulting in frequent encryption and decryption operations, which increases the complexity of the system and delays the lack of data security.

[0004] As an emerging encryption method, fully homomorphic encryption technology allows direct calculations on encrypted data, avoiding the risk of privacy leakage caused by data decryption. Therefore, it is a powerful tool for voiceprint recognition systems to protect privacy. There are many existing neural network architectures based on homomorphic encryption schemes such as BFV, BGV, TFHE, and CKKS, which are mainly used for encrypted reasoning on convolutional neural networks to achieve privacy protection for deep learning models. Convolution operations are crucial to convolutional neural networks (CNNs), but performing convolution operations on ciphertexts in homomorphic encryption requires a lot of computing resources. Each kernel shift requires element-wise multiplication and addition operations, which is a resource-intensive operation on ciphertext. Therefore, in a homomorphic encryption environment, the reasoning process of deep learning models is usually more complicated, with high computational overhead, resulting in a significant reduction in reasoning speed. At the same time, there is also the problem that the characteristics of homomorphic encryption are often not considered in the design of existing deep learning models, resulting in poor accuracy and effect when reasoning on encrypted data. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a privacy-preserving voiceprint recognition reasoning method and system based on fully homomorphic encryption technology. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] A privacy-preserving voiceprint recognition inference method based on fully homomorphic encryption technology, comprising:

[0007] Collecting a number of original voiceprint data from a preset voiceprint database, and preprocessing the original voiceprint data to form a voiceprint data set;

[0008] Inputting the voiceprint data set into the voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encryption model parameters;

[0009] Inputting the encryption model parameters and the voiceprint data to be identified into the encryptor to obtain the voiceprint ciphertext data;

[0010] The voiceprint ciphertext data is calculated and inferred according to the homomorphic encryption characteristics.

[0011] In a specific implementation, the original voiceprint data is preprocessed to form a voiceprint data set, including:

[0012] Removing the speech silence portion in the original voiceprint data to form complete sample data;

[0013] The complete sample data is subjected to denoising, feature extraction and standardization processing in sequence;

[0014] The standardized voiceprint data is converted into a voiceprint dataset after data format and data length conversion to adapt to the homomorphic encryption characteristics.

[0015] In a specific implementation, training the voiceprint recognition model to be trained includes:

[0016] The stochastic gradient descent optimizer is used to train the voiceprint recognition model to be trained.

[0017] In a specific embodiment, the encryptor adopts the RNS-CKKS fully homomorphic encryption algorithm.

[0018] In a specific implementation, before calculating and reasoning the voiceprint ciphertext data according to the homomorphic encryption characteristics, the method further includes:

[0019] Receiving input of multiple voiceprint ciphertext sub-data;

[0020] Calculate the output vector size of multiple voiceprint ciphertext sub-data according to the structure of the voiceprint recognition model;

[0021] Setting the number of slots for storing voiceprint ciphertext sub-data according to the size of the voiceprint ciphertext sub-data;

[0022] Traverse the encrypted data in each slot in turn, and right-rotate each encrypted data according to the preset rotation bit number so that the position of each data in the ciphertext does not overlap;

[0023] When the total size of the ciphertexts is determined to be less than the number of slots after the ciphertexts are merged, the voiceprint ciphertext data is obtained.

[0024] In a specific implementation, the preset number of rotation bits is: (i-1)×N(i-1)×N, where i represents the order of encrypted data and N represents the number of slots.

[0025] The present invention also provides a privacy-preserving voiceprint recognition inference system based on fully homomorphic encryption technology, comprising:

[0026] A test set generation module is used to collect a number of original voiceprint data from a preset voiceprint database, and form a voiceprint data set after preprocessing the original voiceprint data;

[0027] A model training module, used for inputting the voiceprint data set into the voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encrypted model parameters;

[0028] An encryption module, used for inputting the encryption model parameters and the voiceprint data to be identified into an encryptor to obtain voiceprint ciphertext data;

[0029] The reasoning module is used to perform computational reasoning on the voiceprint ciphertext data according to the homomorphic encryption characteristics.

[0030] In a specific implementation, the original voiceprint data is preprocessed to form a voiceprint data set, including:

[0031] Removing the speech silence portion in the original voiceprint data to form complete sample data;

[0032] The complete sample data is subjected to denoising, feature extraction and standardization processing in sequence;

[0033] The standardized voiceprint data is converted into a voiceprint dataset after data format and data length conversion to adapt to the homomorphic encryption characteristics.

[0034] In a specific implementation, training the voiceprint recognition model to be trained includes:

[0035] The stochastic gradient descent optimizer is used to train the voiceprint recognition model to be trained.

[0036] In a specific embodiment, it also includes a batch ciphertext processing module, which is used to receive the input of multiple voiceprint ciphertext sub-data; calculate the output vector size of multiple voiceprint ciphertext sub-data according to the structure of the voiceprint recognition model; set the number of slots for storing voiceprint ciphertext sub-data according to the size of the voiceprint ciphertext sub-data; traverse the encrypted data in each slot in turn, and right-rotate each encrypted data according to a preset number of rotation bits so that the position of each data in the ciphertext does not overlap; after merging the ciphertexts, when it is determined that the total size of the ciphertexts does not exceed the number of slots, the voiceprint ciphertext data is obtained; wherein the preset number of rotation bits is: (i-1)×N(i-1)×N, wherein i represents the order of the encrypted data, and N represents the number of slots.

[0037] Beneficial effects of the present invention:

[0038] The privacy-preserving voiceprint recognition and reasoning method based on fully homomorphic encryption technology of the present invention comprises: collecting a number of original voiceprint data from a preset voiceprint database, preprocessing the original voiceprint data to form a voiceprint data set; inputting the voiceprint data set into a voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encryption model parameters; inputting the encryption model parameters and the voiceprint data to be recognized into an encryptor to obtain voiceprint ciphertext data; and calculating and reasoning the voiceprint ciphertext data according to homomorphic encryption characteristics. In this way, the voiceprint data to be recognized is recognized by the voiceprint recognition model, thereby improving the calculation accuracy in a fully homomorphic encryption environment, and reducing resource consumption and improving reasoning efficiency.

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of a privacy-preserving voiceprint recognition reasoning method based on fully homomorphic encryption technology provided by an embodiment of the present invention;

[0041] Figure 2 It is a module block diagram of a privacy-preserving voiceprint recognition and inference system based on fully homomorphic encryption technology provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0043] Embodiment 1

[0044] See also Figure 1 , Figure 1 The following is a flowchart of a privacy-preserving voiceprint recognition inference method based on fully homomorphic encryption technology provided by an embodiment of the present invention, including:

[0045] S1. Collecting a number of original voiceprint data from a preset voiceprint database, and preprocessing the original voiceprint data to form a voiceprint data set; specifically, the voiceprint data set includes a training set and a test set.

[0046] In this step, the original voiceprint data is preprocessed to form a voiceprint data set, including:

[0047] Removing the speech silence portion in the original voiceprint data to form complete sample data;

[0048] The complete sample data is subjected to denoising, feature extraction and standardization processing in sequence;

[0049] The standardized voiceprint data is converted into a voiceprint dataset after data format and data length conversion to adapt to the homomorphic encryption characteristics.

[0050] This process is to denoise, extract features, and standardize the voiceprint data to improve the accuracy of subsequent recognition. At the same time, the test data is preprocessed to ensure that its format and features meet the requirements of the homomorphic encryption algorithm. Homomorphic encryption algorithms usually have strict requirements on the type and range of input data. For example, some encryption algorithms only support integer operations, so it is necessary to ensure that the data input to the model can be converted to an appropriate format.

[0051] S2. Inputting the voiceprint data set into the voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encryption model parameters;

[0052] It should be noted that the model training process is carried out in plain text using the voiceprint dataset, in order to ensure that the model can effectively identify and classify voiceprint signals after optimization, thereby obtaining a model for subsequent reasoning. The voiceprint recognition model of this embodiment adopts the MobioleNetV2 model, for example, uses the stochastic gradient descent (SGD) optimizer to perform deep training on MobilNetV2, and uses the cross entropy loss function or the AAM loss function to calculate the training cost.

[0053] S3, inputting the encryption model parameters and the voiceprint data to be identified into the encryptor to obtain the voiceprint ciphertext data;

[0054] Since operations such as convolution in a homomorphic environment are more complex to calculate and require a longer reasoning time, in order to reduce the ciphertext reasoning time, this embodiment uses a combined ciphertext batch processing technology, which combines multiple encrypted input data into a single ciphertext for batch processing, greatly reducing the overall calculation time and having significant advantages in large-scale data processing.

[0055] The specific processing steps also include:

[0056] Receiving input of multiple voiceprint ciphertext sub-data;

[0057] Calculate the output vector size of multiple voiceprint ciphertext sub-data according to the structure of the voiceprint recognition model;

[0058] Setting the number of slots for storing voiceprint ciphertext sub-data according to the size of the voiceprint ciphertext sub-data;

[0059] Traverse the encrypted data in each slot in turn, and right-rotate each encrypted data according to the preset rotation bit number so that the position of each data in the ciphertext does not overlap;

[0060] When the total size of the ciphertexts is determined to be less than the number of slots after the ciphertexts are merged, the voiceprint ciphertext data is obtained. Preferably, the preset number of rotation bits is: (i-1)×N(i-1)×N, where i represents the order of the encrypted data and N represents the number of slots.

[0061] S4. Perform computational reasoning on the voiceprint ciphertext data according to the homomorphic encryption characteristics. Specifically, the encrypted voiceprint ciphertext data is passed to the forward method of the homomorphic encryption architecture. The architecture will perform convolution, activation, pooling and other operations based on the layer name or identifier of the pre-trained model using the characteristics of homomorphic encryption, perform reasoning calculations on the encrypted data, thereby achieving voiceprint recognition and returning the encrypted result. Afterwards, the reasoning result is decrypted, the encrypted output is converted into plaintext, and the final voiceprint recognition result is output, which is then analyzed and compared with the voiceprint recognition result in plaintext. Therefore, by establishing such an encrypted reasoning framework, it is possible to accurately identify voiceprints while protecting the privacy of voiceprint data.

[0062] Preferably, this embodiment uses the RNS-CKKS fully homomorphic encryption algorithm for data encryption and decryption and ciphertext operations, so as to facilitate the encryption of real or complex vectors. Specifically, the calculation function of the homomorphic encryption algorithm is implemented by the SEAL-Python encryption calculation library. And the encryption scheme parameters are determined according to the training model architecture to ensure that all security parameters meet the 128-bit security level. And by adjusting the MobioleNetV2 model to be trained based on the homomorphic encryption algorithm and the encryption calculation library, the encryption model architecture suitable for the voiceprint recognition task is adapted.

[0063] In a specific example, first, the system uses the TIMIT dataset to collect and preprocess voiceprint data. In the preprocessing stage, the voiceprint data will be preprocessed, the silent part of the voice will be removed, and it will be divided into complete samples. The model will be directly output to extract special audio features, and then the voiceprint recognition model will be trained. Next, the optimized MobileNetV2 model is selected for the voiceprint recognition task, the model is trained with the preprocessed data, and the model performance is improved by adjusting the hyperparameters. At the same time, the accuracy and robustness of the model are evaluated by cross-validation and other methods. After the model training is completed, the system will set the RNS-CKKS encryption context and generate encryption parameters, including key generation, encoder and encryptor. Then, the trained model parameters and voiceprint data are input into the encryptor to generate encrypted ciphertext data. Next, the encrypted voiceprint features are used as input, reasoned through the MobileNetV2 model, and the homomorphic encryption feature is used to directly calculate on the ciphertext to ensure data privacy. Finally, the system decrypts the output of the model, restores the result of voiceprint recognition, and outputs the decrypted result to the user. Through this series of steps, the voiceprint recognition privacy protection system based on homomorphic encryption effectively protects user privacy while realizing efficient voiceprint recognition function.

[0064] For the multi-ciphertext batch processing method, first receive multiple encrypted voiceprint data and prepare for batch processing. Then, according to the structure of the MobileNetV2 model, calculate the size of the intermediate or final output vector to ensure that there is no overlap when merging. Next is to initialize the slots and set the number of slots used to store encrypted data, for example, 800. Traverse each encrypted data and process each encrypted data one by one. Rotate the encrypted data to right rotate the ii-th encrypted data, and the number of bits rotated is (i-1)×800(i-1)×800 to ensure that the position of each data in the ciphertext does not overlap. Next is to insert the rotated encrypted data into the ciphertext. Then check that the total size of the merged ciphertext does not exceed the number of initialized slots. Output the merged ciphertext, return the merged ciphertext, and complete the batch processing.

[0065] See also Figure 2 , Figure 2 This is a module block diagram of a privacy-preserving voiceprint recognition and reasoning system based on fully homomorphic encryption technology provided by an embodiment of the present invention, including:

[0066] A test set generation module is used to collect a number of original voiceprint data from a preset voiceprint database, and form a voiceprint data set after preprocessing the original voiceprint data;

[0067] A model training module, used for inputting the voiceprint data set into the voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encrypted model parameters;

[0068] An encryption module, used for inputting the encryption model parameters and the voiceprint data to be identified into an encryptor to obtain voiceprint ciphertext data;

[0069] The reasoning module is used to perform computational reasoning on the voiceprint ciphertext data according to the homomorphic encryption characteristics.

[0070] In a specific implementation, the original voiceprint data is preprocessed to form a voiceprint data set, including:

[0071] Removing the speech silence portion in the original voiceprint data to form complete sample data;

[0072] The complete sample data is subjected to denoising, feature extraction and standardization processing in sequence;

[0073] The standardized voiceprint data is converted into a voiceprint dataset after data format and data length conversion to adapt to the homomorphic encryption characteristics.

[0074] In a specific implementation, training the voiceprint recognition model to be trained includes:

[0075] The stochastic gradient descent optimizer is used to train the voiceprint recognition model to be trained.

[0076] In a specific embodiment, it also includes a batch ciphertext processing module, which is used to receive the input of multiple voiceprint ciphertext sub-data; calculate the output vector size of multiple voiceprint ciphertext sub-data according to the structure of the voiceprint recognition model; set the number of slots for storing voiceprint ciphertext sub-data according to the size of the voiceprint ciphertext sub-data; traverse the encrypted data in each slot in turn, and right-rotate each encrypted data according to a preset number of rotation bits so that the position of each data in the ciphertext does not overlap; after merging the ciphertexts, when it is determined that the total size of the ciphertexts does not exceed the number of slots, the voiceprint ciphertext data is obtained; wherein the preset number of rotation bits is: (i-1)×N(i-1)×N, wherein i represents the order of the encrypted data, and N represents the number of slots.

[0077] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.

[0078] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0079] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware, which are collectively referred to as "modules" or "systems" herein. Moreover, the present application may adopt the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The computer program is stored / distributed in a suitable medium, provided together with other hardware or as a part of hardware, or may adopt other distribution forms, such as by Internet or other wired or wireless telecommunication systems.

[0080] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses (devices) and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A privacy-preserving voiceprint recognition inference method based on fully homomorphic encryption technology, characterized in that: include: Collecting a number of original voiceprint data from a preset voiceprint database, and preprocessing the original voiceprint data to form a voiceprint data set; Inputting the voiceprint data set into the voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encryption model parameters; Inputting the encryption model parameters and the voiceprint data to be identified into the encryptor to obtain the voiceprint ciphertext data; The voiceprint ciphertext data is calculated and inferred according to the homomorphic encryption characteristics.

2. According to claim 1, the privacy-preserving voiceprint recognition inference method based on fully homomorphic encryption technology is characterized in that: The original voiceprint data is preprocessed to form a voiceprint data set, including: Removing the speech silence portion in the original voiceprint data to form complete sample data; The complete sample data is subjected to denoising, feature extraction and standardization processing in sequence; The standardized voiceprint data is converted into a voiceprint dataset after data format and data length conversion to adapt to the homomorphic encryption characteristics.

3. According to claim 1, the privacy-preserving voiceprint recognition reasoning method based on fully homomorphic encryption technology is characterized in that: Training the voiceprint recognition model to be trained includes: The stochastic gradient descent optimizer is used to train the voiceprint recognition model to be trained.

4. The privacy-preserving voiceprint recognition inference method based on fully homomorphic encryption technology according to claim 3 is characterized in that: The encryptor adopts the RNS-CKKS fully homomorphic encryption algorithm.

5. According to claim 1, the privacy-preserving voiceprint recognition reasoning method based on fully homomorphic encryption technology is characterized in that: Before calculating and reasoning the voiceprint ciphertext data according to the homomorphic encryption characteristics, the method further includes: Receiving input of multiple voiceprint ciphertext sub-data; Calculate the output vector size of multiple voiceprint ciphertext sub-data according to the structure of the voiceprint recognition model; Setting the number of slots for storing voiceprint ciphertext sub-data according to the size of the voiceprint ciphertext sub-data; Traverse the encrypted data in each slot in turn, and right-rotate each encrypted data according to the preset rotation bit number so that the position of each data in the ciphertext does not overlap; When the total size of the ciphertexts is determined to be less than the number of slots after the ciphertexts are merged, the voiceprint ciphertext data is obtained.

6. The privacy-preserving voiceprint recognition reasoning method based on fully homomorphic encryption technology according to claim 5 is characterized in that: The preset number of rotation bits is: (i-1)×N(i-1)×N, where i represents the order of encrypted data and N represents the number of slots.

7. A privacy-preserving voiceprint recognition inference system based on fully homomorphic encryption technology, characterized in that: include: A test set generation module is used to collect a number of original voiceprint data from a preset voiceprint database, and form a voiceprint data set after preprocessing the original voiceprint data; A model training module, used for inputting the voiceprint data set into the voiceprint recognition model to be trained, and training the voiceprint recognition model to be trained to obtain a trained voiceprint recognition model to generate encrypted model parameters; An encryption module, used for inputting the encryption model parameters and the voiceprint data to be identified into an encryptor to obtain voiceprint ciphertext data; The reasoning module is used to perform computational reasoning on the voiceprint ciphertext data according to the homomorphic encryption characteristics.

8. The privacy-preserving voiceprint recognition and reasoning system based on fully homomorphic encryption technology according to claim 7 is characterized in that: The original voiceprint data is preprocessed to form a voiceprint data set, including: Removing the speech silence portion in the original voiceprint data to form complete sample data; The complete sample data is subjected to denoising, feature extraction and standardization processing in sequence; The standardized voiceprint data is converted into a voiceprint dataset after data format and data length conversion to adapt to the homomorphic encryption characteristics.

9. The privacy-preserving voiceprint recognition and reasoning system based on fully homomorphic encryption technology according to claim 7 is characterized in that: Training the voiceprint recognition model to be trained includes: The stochastic gradient descent optimizer is used to train the voiceprint recognition model to be trained.

10. The privacy-preserving voiceprint recognition and reasoning system based on fully homomorphic encryption technology according to claim 7 is characterized in that: It also includes a batch ciphertext processing module, which is used to receive the input of multiple voiceprint ciphertext sub-data; calculate the output vector size of multiple voiceprint ciphertext sub-data according to the structure of the voiceprint recognition model; set the number of slots for storing voiceprint ciphertext sub-data according to the size of the voiceprint ciphertext sub-data; traverse the encrypted data in each slot in turn, and right-rotate each encrypted data according to a preset number of rotation bits so that the position of each data in the ciphertext does not overlap; after merging the ciphertexts, when it is determined that the total size of the ciphertexts does not exceed the number of slots, the voiceprint ciphertext data is obtained; wherein the preset number of rotation bits is: (i-1)×N(i-1)×N, wherein i represents the order of the encrypted data, and N represents the number of slots.

Citation Information

Patent Citations

  • Privacy protection equipment identification model design and use method based on homomorphic encryption

    CN111984960A

  • Voiceprint recognition method and control equipment

    CN116189690A

  • Speech feature authentication method and system based on fully homomorphic encryption algorithm

    CN116318605A

  • Privacy decision tree reasoning method based on fully homomorphic encryption

    CN116484415A

  • Method and system for identifying individual using homomorphically encrypted voice

    WO2023128342A1