Speaker anonymization method based on voice content desensitization and voiceprint anonymization

Abstract

A speaker anonymization method based on voice content desensitization and voiceprint anonymization belongs to the field of intelligent voice, and comprises the following steps: performing voice recognition on original voice; preprocessing the text; carrying out keyword extraction by using a TextRank algorithm; performing security word replacement to obtain a text after voice content desensitization; performing voiceprint feature extraction on the original voice by using x-vector; performing voiceprint anonymization on the voiceprint features by using the WGAN to obtain anonymized voiceprints; and speech synthesis is carried out to obtain safe speech. According to the method, the sensitive words of the voice content can be replaced with the safe words under the condition of not influencing semantics; meanwhile, anonymous voiceprint features are generated by using a WGAN algorithm, and stable and high-quality generation samples are provided; the method has high-performance speech feature extraction capability, has relatively high robustness to environmental noise and speech change, and is suitable for a large-scale speech database.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent voice technology, and in particular relates to a speaker anonymization method based on voice content desensitization and voiceprint anonymization. Background Art

[0002] With the continuous development of information technology, intelligent voice technology has become the most convenient and effective way for people to obtain information and communicate. Various voice-based smart devices and applications have facilitated people's daily lives. In almost all voice input applications, the user's voice recordings are sent to cloud servers for voice recognition. These servers usually store the recordings in a database and may use them for research or commercial interests. Therefore, while intelligent voice technology brings convenience to users, it also causes voice data privacy leakage issues.

[0003] On the one hand, attackers can extract personal information from speech and detect sensitive content by using technologies such as speech recognition and natural language processing. Speakers can be easily identified through chain attacks. Investigations and studies have found that in terms of privacy protection of speech data content, the contributions of researchers using algorithms such as TF-IDF and KeyBert cannot be ignored. However, these algorithms have some challenges in processing long texts. In particular, the TF-IDF algorithm used in most papers combines word frequency and inverse document frequency to calculate the weight of keywords, but ignores the semantic relationship between words and cannot fully capture the contextual information of words. Therefore, a method is needed that can not only consider the contextual relevance between words, but also has good robustness, and can effectively extract text content regardless of whether it is a short or long text, and whether there is noise or errors in the text.

[0004] On the other hand, attackers extract people's voiceprints through technologies such as voice cloning and speech synthesis, generate synthetic voices that sound similar to people's real voices, or deceive voiceprint recognition systems through replay attacks to obtain unauthorized access. In order to meet this challenge, many researchers focus on methods to protect voiceprint data. At present, commonly used voiceprint privacy protection methods include differential privacy, homomorphic encryption, and secure multi-party computing. However, these methods usually require a lot of communication and computing overhead, have high requirements on system resources, and may lead to reduced data availability. Due to their low efficiency, they cannot meet the needs of efficient processing in practical applications. Therefore, a method is needed to remove the speaker's identity information from the voice signal as much as possible, and to improve the computational efficiency and the accuracy of voiceprint recognition. It is of great significance to study the problems of voice content desensitization and speaker anonymization in voice data.

[0005] In terms of dual privacy protection of voiceprint and content, Qian et al. implemented a speech purifier, which protects the privacy of voice content based on the DTW speech content keyword modification method, and proposed to use the voice conversion technology based on the composite warping function to protect the privacy of user voiceprint (Qian J, Du H, Hou J, et al. Speech sanitizer: Speech content desensitization and voice anonymization [J]. IEEE Transactions on Dependable and Secure Computing, 2019, 18 (6): 2631-2642.). Based on the work of Qian et al., Zhang et al. defined the correlation between speech content and the speaker's voice to reduce the privacy leakage caused by this irrelevance (Zhang G, Ni S, Zhao P. Enhancing privacy preservation in speech data publishing [J]. IEEE Internet of Things Journal, 2020, 7 (8): 7357-7367.). Han et al. extended differential privacy and designed a voiceprint privacy measurement mechanism called voice indistinguishability to achieve privacy protection before voice data is released (Han Y, Cao Y, Li S, et al. Voice-Indistinguishability--Protecting voiceprint with differential privacy under an untrusted server[C]. Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 2020: 2125-2127.). Ahmed et al. proposed a privacy-preserving end-to-end speech transcription system that uses differential privacy technology to add noise to the frequency of keywords in the voice content, thereby protecting the privacy of the user's voice content (Ahmed S, Chowdhury AR, Fawaz K, et al. Preech: A system for privacy-preserving speech transcription[C]. 29th USENIX Security Symposium (USENIX Security 20), 2020: 2703-2720.).Shamsabadi et al. adopted the architecture of an autoencoder and automatic speech recognizer with a Laplace noise layer, and designed a pitch and bottleneck (Bottleneck, BN) feature extractor that meets differential privacy, removing the speaker information from the pitch and bottleneck features, and protecting the speaker's acoustic characteristics (Shamsabadi AS, Srivastava B ML, Bellet A, et al. Differentially private speaker anonymization [EB / OL]. arXiv preprint arXiv: 2202.11823, 2022.).

[0006] Existing papers also use TF-IDF-based keyword recognition methods and speech conversion technology. The TF-IDF algorithm does not consider the semantic information of words and has difficulty processing longer texts (Zhang G, Ni S, Zhao P. Enhancing privacy preservation in speech data publishing [J]. IEEE Internet of Things Journal, 2020, 7 (8): 7357-7367.) and (Ahmed S, Chowdhury A R, Fawaz K, et al. Preech: A system for privacy-preserving speech transcription [C]. 29th USENIX Security Symposium (USENIX Security 20), 2020: 2703-2720.). Therefore, the TF-IDF algorithm cannot fully capture the keywords in the user's voice content. Hu et al. proposed a speech data privacy protection mechanism that uses named entity recognition and Keybert keyword extraction algorithm to protect speech content, and designed a voiceprint anonymization method based on differential privacy (HuY, Li R, Wang S, et al. SpeechHide: A hybrid privacy-preserving mechanism for speech content and voiceprint in speech data sharing[C]. 20227th IEEE International Conference on Data Science in Cyberspace(DSC), IEEE, 2022: 345-352.).

[0007] In terms of voice data privacy protection, most existing methods focus on research on voice content desensitization and speaker anonymization. In voice content desensitization methods, most adopt the TF-IDF keyword recognition algorithm that does not consider the semantic information of words; in speaker anonymization methods, most adopt differential privacy algorithms, and speakers are still at risk of deanonymization. In terms of voiceprint privacy protection, existing voiceprint privacy protection research focuses on protecting the privacy of data, which has problems such as high computational overhead, limited accuracy, and high computational complexity. Summary of the invention

[0008] The purpose of the present invention is to provide a speaker anonymization method based on voice content desensitization and voiceprint anonymization to solve the problem of voice data privacy leakage and the problem that the speaker's voiceprint is vulnerable to attack. The present invention studies voice content desensitization and voiceprint anonymization, and designs a new voice data protection method according to the characteristics of the problem itself. On the one hand, it can protect the content of voice data and provide new ideas for corresponding research. On the other hand, it can also ensure the security and reliability of not being de-anonymized by the speaker.

[0009] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0010] The present invention provides a speaker anonymization method based on speech content desensitization and voiceprint anonymization, comprising the following steps:

[0011] Step S1: Perform speech recognition on the original speech to obtain a text document;

[0012] Step S2: preprocessing the obtained text document;

[0013] Step S3: extract keywords using the TextRank algorithm;

[0014] Step S4: Replace the safe word to obtain a text with voice content desensitized;

[0015] Step S5: Use x-vector to extract voiceprint features from the original speech to obtain voiceprint features;

[0016] Step S6: Use WGAN to anonymize the obtained voiceprint features to obtain an anonymous voiceprint;

[0017] Step S7: Perform speech synthesis on the desensitized text of the speech content and the anonymized voiceprint to obtain a secure speech.

[0018] Furthermore, in step S1, the Vosk offline speech recognition system is used to perform speech recognition on the original speech into a text document.

[0019] Furthermore, in step S2, the text document is segmented into sentences, words and parts of speech to obtain sentence feature items; then stop words and punctuation marks are removed from the sentence feature items to retain words with specific parts of speech.

[0020] Furthermore, the specific implementation process of step S3 is as follows:

[0021] Represent the text as a graph. For a given word sequence, construct an undirected graph with each word corresponding to a node. If two words are adjacent in the text, connect them with an edge and iterate to calculate the weight of each node or word. Extract text keywords in descending order according to the node weight values.

[0022] Furthermore, suppose the text is composed of n elements V i (1≤i≤n) and represent it as V={1≤i≤n}, with V i The nodes are nodes. Whether there is a similar relationship between each node determines the edge. The value of similarity is the weight value of the edge, forming a directed TextRank text network graph G = (V, E, W), where V represents the n elements V i The text set composed of (1≤i≤n);

[0023] In the TextRank text network graph G, the finite set of edges between nodes is represented by It is non-empty, and the set is symbolized as E = {(V i ,V j |V i ∈V∧V j ∈V∧w ij ∈W∧w ij ≠0)}, in this expression, the symbol ∧ represents logical AND, W represents the weight set of the network graph edge, which is symbolically represented as W = {w ij |1≤i≤n∧1≤j≤n},w ij Represents node V i and node V j The weight value of the edge between them is the similarity;

[0024] According to the TextRank text network graph G = (V, E, W), we get an n×n similarity matrix SM about the nodes n×n :

[0025]

[0026] According to the TextRank text network graph G and n×n similarity matrix SM n×n , the weight of each node is iteratively calculated continuously, and the specific calculation formula is as follows:

[0027]

[0028] Among them, WS(V i ) represents node V i The weight value; d represents the damping coefficient, which ranges from (0,1) and represents the probability of a node jumping to any other node in the TextRank text network graph G; ln(V i ) indicates pointing to node V i The set of all nodes of Out(V j ) represents node V j The set of all nodes pointed to; V k Indicates that it belongs to the set Out(V j ) node; |Out(V j )| represents the set Out(V j ) The number of elements in w jk Represents node V j and node V k The sum on the right side of the formula represents the contribution of each adjacent node to the current node; WS(V j ) indicates the node V after the last iteration j The weight value of .

[0029] Furthermore, in step S4, a set of safe words is given and divided into several categories, and a safe word is randomly selected from the extracted sensitive words according to the category to which they belong to replace them, so as to obtain a text after desensitization of the speech content.

[0030] Furthermore, in step S5, the speech signal is modeled by a deep neural network, and the speech signal is encoded into a vector representation of a fixed dimension using time series modeling and statistical pooling techniques.

[0031] Furthermore, in step S6, the structures of the WGAN generator network and the WGAN discriminator network are defined. The WGAN generator network is responsible for generating synthetic data samples, and the WGAN discriminator network is used to distinguish between real data and generated data. WGAN uses Wasserstein distance as a loss function to measure the difference between the real data distribution and the generated data distribution. The loss function is implemented by calculating the distance between the generator and the discriminator. The network parameters are optimized by alternately training the generator and the discriminator.

[0032] Furthermore, in each alternating training step, the generator is first fixed and the parameters of the discriminator are updated, then the discriminator is fixed and the parameters of the generator are updated; in order to meet the constraints of the Wasserstein distance, the parameters of the discriminator are usually gradient clipped to limit their gradient range; finally, the samples generated by the generator are used to evaluate its performance.

[0033] Furthermore, in step S7, the desensitized text and anonymized voiceprint are processed for speech synthesis based on the VITS speech synthesis model to obtain the final secure speech.

[0034] The beneficial effects of the present invention are:

[0035] 1. In terms of voice content data, existing methods have difficulty considering the semantic relationship between words, cannot fully capture the contextual information of words, and voice audio cannot be efficiently and accurately converted into text information. In this regard, the present invention designs a voice content desensitization model based on keyword extraction, which replaces sensitive words in voice content with safe words while minimizing the impact on semantics; at the same time, the TextRank algorithm is used to extract keywords from the text, considering the correlation between words, and is used to extract sensitive words from voice text; in addition, in the face of large-scale text data, the voice content desensitization model based on keyword extraction designed by the present invention can combine contextual information and better iterate to obtain words.

[0036] 2. In terms of speaker voiceprint privacy, traditional voice feature extraction algorithms are mainly based on signal processing and spectrum analysis technology, which have high data requirements, poor robustness, and difficulty in handling changes between speakers. Voiceprint hiding algorithms based on differential privacy are difficult to resist attacks from de-anonymization systems. In response to this, the present invention designs a new speaker anonymization algorithm to create sounds that sound as natural and non-existent as possible, with high-performance voice feature extraction capabilities, and strong robustness to environmental noise and voice changes, suitable for large-scale voice databases; the WGAN algorithm is used to generate anonymous voiceprint features, which effectively solves problems such as gradient vanishing and mode collapse, and provides stable and high-quality generated samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flow chart of a speaker anonymization method based on speech content desensitization and voiceprint anonymization provided by the present invention.

[0038] Figure 2 This is the flow chart of the speech content desensitization model.

[0039] Figure 3 This is the flow chart of voiceprint feature extraction based on x-vector.

[0040] Figure 4The figure is the flow chart of voiceprint anonymization based on WGAN. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0042] The present invention provides a speaker anonymization method based on voice content desensitization and voiceprint anonymization, which studies voice content desensitization, voiceprint anonymization and speech synthesis related contents with the help of knowledge such as keyword extraction and generative adversarial network. The voice content is recognized through Vosk API, sensitive words are extracted from the text using the TextRank algorithm, and sensitive words are replaced with safe words to achieve voice content desensitization; at the same time, the voiceprint feature vector is extracted using the x-vector algorithm, and a WGAN (Wasserstein GAN) algorithm is designed to anonymize the voiceprint, and finally speech synthesis is completed based on the VITS speech synthesis model.

[0043] See also Figure 1 To illustrate, the present invention provides a speaker anonymization method based on voice content desensitization and voiceprint anonymization, and its specific implementation process is as follows:

[0044] Step 1: Establish a voice content desensitization model based on keyword extraction;

[0045] Voice content contains rich semantic information. When an attacker obtains and analyzes the voice data of a specific user, he can infer the user's identity information, health status, etc., which leads to the leakage of voice content privacy. In order to solve the problem of voice data privacy leakage, this paper proposes a voice content desensitization model based on keyword extraction. Under the condition of not affecting the semantics as much as possible, the sensitive words in the voice content are replaced with safe words. The focus is on the keyword extraction part. The flow chart of the voice content desensitization model is shown in the figure below. Figure 2 As shown, the specific implementation process is as follows:

[0046] Step S1.1: speech recognition;

[0047] First, the Vosk offline speech recognition system is used to perform speech recognition on the original speech into a text document.

[0048] Step S1.2: text preprocessing, including text segmentation and removal of stop words;

[0049] Specifically, before keyword extraction, the above text document needs to be preprocessed. The preprocessing mainly includes sentence segmentation, word segmentation and part-of-speech tagging, so as to obtain sentence feature items; then the sentence feature items are processed by removing stop words, punctuation marks, etc., and retaining terms with specific parts of speech.

[0050] Step S1.3: keyword extraction;

[0051] The TextRank algorithm is used for keyword extraction. TextRank is a graph-based unsupervised algorithm that is improved from the PageRank algorithm for ranking web page importance. In information retrieval, the TextRank algorithm is usually used to calculate the similarity matrix between words. The present invention uses the TextRank algorithm to detect keywords in the text that may leak personal information. Specifically:

[0052] Step S1.3.1: Represent the text as a graph. For a given word sequence, construct an undirected graph in which each word corresponds to a node. If two words are adjacent in the text, an edge is connected between them. The weight of each node, i.e., word, is iteratively calculated to identify the most important words.

[0053] Specifically, suppose the text is composed of n elements V i (1≤i≤n) and represent it as V={1≤i≤n}, with V i The nodes are nodes. Whether there is a similar relationship between each node determines the edge. The value of similarity is the weight value of the edge, forming a directed TextRank text network graph G = (V, E, W), where V represents the n elements V i The text set composed of (1≤i≤n).

[0054] In the TextRank text network graph G, the finite set of edges between nodes is represented by It is non-empty, and the set is symbolized as E = {(V i ,V j |V i ∈V∧V j ∈V∧w ij ∈W∧w ij ≠0)}, in this expression, the symbol ∧ represents logical AND, W represents the weight set of the network graph edge, which is symbolically represented as W = {w ij |1≤i≤n∧1≤j≤n},w ij Represents node V i and node V j The weight value of the edge, that is, the similarity, can be calculated by various similarity calculation functions.

[0055] Step S1.3.2: According to the TextRank text network graph G = (V, E, W), we can get an n×n similarity matrix SM about the nodes n×n :

[0056]

[0057] Step S1.3.3: Because the contribution of two interacting nodes to each other in the TextRank text network graph G is the same, that is, node V i For node V j Contribution and node V j For node V i The contribution made is the same. According to the TextRank text network graph G and the n×n similarity matrix SM n×n , the weight of each node is iteratively calculated continuously, and the specific calculation formula is as follows:

[0058]

[0059] Among them, WS(V i ) represents the node V i The weight value; d represents the damping coefficient, which ranges from (0,1). It represents the probability of a node jumping to any other node in the TextRank text network graph G. Generally, d is set to 0.85; ln(V i ) indicates pointing to node V i The set of all nodes of Out(V j ) represents the node V j The set of all nodes pointed to; V k Indicates that it belongs to the set Out(V j ) node; |Out(V j )| represents the set Out(V j ) The number of elements in w jk Represents node V j and node V k The sum on the right side of the formula represents the contribution of each adjacent node to the current node; WS(V j ) indicates the node V after the last iteration j The weight value of .

[0060] Step S1.3.4: Extract text keywords in descending order according to the node weight values.

[0061] Step S1.4: safe word replacement;

[0062] Given a set of safe words and dividing them into several categories, in order to make words in the same category more similar than words in other categories, for example, they can be divided into categories such as nouns, verbs, adjectives and adverbs, the extracted sensitive words can be randomly replaced with a safe word according to their category to obtain the text after voice content desensitization.

[0063] At present, great progress has been made in voice data protection, but there are few studies on voice content desensitization, and it is impossible to effectively prevent the attacker from training deceptive attack models by collecting speech. Therefore, the present invention proposes a voice content desensitization model based on keyword extraction, which can desensitize the voice content without affecting the semantics as much as possible. The present invention breaks through the traditional method of constructing data set training, trains in an unsupervised manner, uses a sorting algorithm based on a directed graph, and iteratively calculates the weight value of the keyword node; in addition, when processing longer texts, the voice content desensitization model based on keyword extraction proposed by the present invention can take into account contextual information and can better obtain text keywords.

[0064] Step 2: Establish a voiceprint anonymization algorithm based on generative adversarial network;

[0065] In order to solve the problem that the speaker's voiceprint is vulnerable to attack, the present invention proposes a voiceprint anonymization method based on a generative adversarial network. The focus of the present invention is on feature vector extraction and generation of new anonymous vectors. The specific implementation process is as follows: Figure 3 and Figure 4 shown.

[0066] Step S2.1: voiceprint feature extraction based on x-vector;

[0067] First, the present invention extracts voiceprint features from the original speech. Different from the traditional discrete cosine transform method, the present invention adopts an x-vector voiceprint feature extraction method based on deep learning. Figure 3 As shown in the figure, the method consists of a frame processing layer, a statistical pooling layer, a segment processing layer and an output layer. The input speech features x1, x2, ..., x T After being processed by the frame processing layer, the output frame level is output, and then processed by the statistical pooling layer and input to the segment processing layer for processing. In the segment processing layer, the output embedding a and embedding b of the first and second layers can be used as a low-dimensional vector representation of a segment of corpus, and the segment level is output to encode the speech signal into a fixed-dimensional vector representation P (spkr i |x1,x2,…,x T ), and finally output through the output layer. The speech signal is modeled through a deep neural network, and the speech signal is encoded into a vector representation of fixed dimension using techniques such as time series modeling and statistical pooling.

[0068] Step S2.2: Voiceprint anonymization based on WGAN;

[0069] The WGAN (Wasserstein GAN) algorithm, which is improved on the basis of GAN, is used to anonymize the extracted voiceprint features. The Wasserstein distance is defined as follows:

[0070]

[0071] Where W(P,Q) represents the set of all possible joint distributions of distribution P and distribution Q. The marginal distribution of each distribution in the set W(P,Q) is distribution P and distribution Q. Y represents a possible joint distribution. Samples x and y of different distributions can be sampled from the joint distribution Y, so that the expected value E of the distance between sample x and sample y can be calculated. (x,y)-γ [Px-yP], γ represents the distribution of sample pairs; E (x,y)～γ [||xy||] represents the expected value of the distance between sample x and sample y under the γ distribution.

[0072] The WGAN objective function is as follows:

[0073]

[0074] Among them, f w represents the discriminator; x represents the sample; f w (x) represents the function of the discriminator; Indicates that sample x follows the true data distribution P data When the discriminator function f w The expected value of (x), Indicates that sample x obeys the data distribution P generated by the generator G When the discriminator function f w The expected value of (x); P data represents the real data distribution; P G Denotes that the generator generates data distribution.

[0075] The Lipschitz function is defined as follows:

[0076] ||f(x1)-f(x2)‖≤K||x1-x2||

[0077] Among them, f(x1) and f(x2) represent the function values ​​of function f at point x1 and point x2 respectively, x1 and x2 represent two points defined in the input space, K represents the Lipschitz constant, ||f(x1)-f(x2)|| represents the norm of the output difference of function f at point x1 and point x2, and ||x1-x2|| represents the Euclidean distance between point x1 and point x2.

[0078] The objective functions of the WGAN generator network and the discriminator network are:

[0079]

[0080] in, represents the objective function of the WGAN generator network; represents the average score of the samples generated by the generator in the discriminator; G(z) represents the samples generated by the generator; represents the objective function of the WGAN discriminator network; Represents the average score of the real data in the discriminator; λ1 represents the gradient penalty factor; Represents the gradient The 2-norm of Denotes the discriminator f w For input x t The gradient of w (x t ) represents the discriminator f w For input x t Output.

[0081] The structures of the WGAN generator network and the WGAN discriminator network are defined, where the WGAN generator network is responsible for generating synthetic data samples, while the WGAN discriminator network is used to distinguish between real data and generated data. WGAN uses a loss function called Wasserstein distance to measure the difference between the real data distribution and the generated data distribution. This loss function is implemented by calculating the distance between the generator and the discriminator. The network parameters are optimized by alternately training the generator and the discriminator. In each training step, the generator is first fixed and the parameters of the discriminator are updated. Then the discriminator is fixed and the parameters of the generator are updated. In order to meet the restrictions of the Wasserstein distance, the parameters of the discriminator are usually gradient clipped to limit the range of its gradient. This helps to improve the stability and convergence of the training. Finally, the samples generated by the generator can be used to evaluate its performance. The quality and diversity of the generated results can be checked by visualizing the generated images or using other evaluation indicators. In the implementation process, hyperparameters will be adjusted according to actual conditions to further improve the performance and generation quality of WGAN.

[0082] like Figure 4 As shown in the figure, the input noise z is a random variable sampled from the standard normal distribution N(0, 1) or the uniform distribution U(-1, 1), which is used as the input of the generator, and x represents the real voiceprint data; the random input noise z is converted into the generated voiceprint sample g through the generator. θ (z); The discriminator is used to calculate the generated sample g θ (z) and the discriminant value of the real sample x; the generator and discriminator are optimized through the objective function of WGAN; the formula for the discriminator output is in, represents the gradient of the discriminator parameter w, f w (x) represents the discriminator function, g θ(z) represents the output of the generator, m and n represent the number of real samples and generated samples respectively; finally, the loss function of WGAN is used to optimize the objectives of the generator and the discriminator; the generator minimizes the Wasserstein distance to make the generated samples closer to the real sample distribution. The core of voiceprint anonymization is to use the generator to map the input random noise into samples similar to the real voiceprint distribution, thereby achieving anonymity. Figure 4 middle, It means that the generator optimizes its parameters θ by gradient descent to maximize the discriminator f w Scoring of generated samples to make them closer to the real data distribution; The symbol for finding the gradient of the generator parameter θ represents the partial derivative of the generator objective function with respect to the parameter θ, and Critic represents the discriminator network f w , its function is to score the input samples, f represents the output result of Critic, and cost represents the target loss function used for optimization in WGAN.

[0083] Traditional anonymization methods follow two directions, signal processing technology and methods based on machine learning to modify speaker embeddings. The latter is usually superior to signal processing technology in terms of privacy and utility indicators. Therefore, the present invention proposes a voiceprint anonymization algorithm based on a generative adversarial network, and uses the x-vector algorithm to extract the original voiceprint vector. This method can be modeled according to the context based on a deep neural network and has good robustness. GAN is a powerful framework that can train a generative model in an unsupervised manner. The present invention proposes to use the WGAN (Wasserstein GAN) algorithm improved on the basis of GAN to anonymize the extracted voiceprint features, which provides higher optimization characteristics and can generate better anonymous voiceprints.

[0084] Step 3: speech synthesis;

[0085] Based on the VITS speech synthesis model, the desensitized text of the speech content obtained in step 1 and the anonymized voiceprint obtained in step 2 are processed for speech synthesis to obtain the final secure speech.

[0086] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A speaker anonymization method based on speech content desensitization and voiceprint anonymization, characterized in that: The following steps are involved: Step S1: Perform speech recognition on the original speech to obtain a text document; Step S2: preprocessing the obtained text document; Step S3: extract keywords using the TextRank algorithm; Step S4: Replace the safe word to obtain a text with voice content desensitized; Step S5: Use x-vector to extract voiceprint features from the original speech to obtain voiceprint features; Step S6: Use WGAN to anonymize the obtained voiceprint features to obtain an anonymous voiceprint; Step S7: Perform speech synthesis on the desensitized text of the speech content and the anonymized voiceprint to obtain a secure speech.

2. According to claim 1, a speaker anonymization method based on voice content desensitization and voiceprint anonymization is characterized in that: In step S1, the Vosk offline speech recognition system is used to perform speech recognition on the original speech into a text document.

3. According to claim 1, a speaker anonymization method based on voice content desensitization and voiceprint anonymization is characterized in that: In step S2, the text document is segmented into sentences, words and parts of speech to obtain sentence feature items; then stop words and punctuation marks are removed from the sentence feature items to retain words with specific parts of speech.

4. The speaker anonymization method based on speech content desensitization and voiceprint anonymization according to claim 1, characterized in that: The specific implementation process of step S3 is as follows: Represent the text as a graph. For a given word sequence, construct an undirected graph with each word corresponding to a node. If two words are adjacent in the text, connect them with an edge and iterate to calculate the weight of each node or word. Extract text keywords in descending order according to the node weight values.

5. The speaker anonymization method based on speech content desensitization and voiceprint anonymization according to claim 4, characterized in that: Assume that the text is composed of n elements V i (1≤i≤n) and represent it as V={1≤i≤n}, with V i The nodes are nodes. Whether there is a similar relationship between each node determines the edge. The value of similarity is the weight value of the edge, forming a directed TextRank text network graph G = (V, E, W), where V represents the n elements V i The text set composed of (1≤i≤n); In the TextRank text network graph G, the finite set of edges between nodes is represented by It is non-empty, and the set is symbolized as E = {(V i ,V j |V i ∈VnV j ∈V∧w ij ∈W∧w ij ≠0)}, in this expression, the symbol ∧ represents logical AND, W represents the weight set of the network graph edge, which is symbolically represented as W = {w ij |1≤i≤n∧1≤j≤n},w ij Represents the weight value of the edge between node Vi and node Vj, that is, the similarity; According to the TextRank text network graph G = (V, E, W), we get an n×n similarity matrix SM about the nodes n×n : According to the TextRank text network graph G and n×n similarity matrix SM n×n , the weight of each node is iteratively calculated continuously, and the specific calculation formula is as follows: Among them, WS(V i ) represents the node V i The weight value; d represents the damping coefficient, which ranges from (0,1) and represents the probability of a node jumping to any other node in the TextRank text network graph G; ln(V i ) indicates pointing to node V i The set of all nodes of Out(V j ) represents the node V j The set of all nodes pointed to; V k Indicates that it belongs to the set Out(V j ) node; |Out(V j )| represents the set Out(V j ) The number of elements in w jk Represents node V j and node V k The sum on the right side of the formula represents the contribution of each adjacent node to the current node; WS(V j ) indicates the node V after the last iteration j The weight value of .

6. The speaker anonymization method based on speech content desensitization and voiceprint anonymization according to claim 1, characterized in that: In step S4, a set of safe words is given and divided into several categories, and a safe word is randomly selected from the extracted sensitive words according to the category to which they belong to replace them, so as to obtain a text after desensitization of the speech content.

7. The speaker anonymization method based on voice content desensitization and voiceprint anonymization according to claim 1, characterized in that: In step S5, the speech signal is modeled by a deep neural network, and the speech signal is encoded into a vector representation of a fixed dimension using time series modeling and statistical pooling techniques.

8. The speaker anonymization method based on speech content desensitization and voiceprint anonymization according to claim 1, characterized in that: In step S6, the structures of the WGAN generator network and the WGAN discriminator network are defined. The WGAN generator network is responsible for generating synthetic data samples, and the WGAN discriminator network is used to distinguish between real data and generated data. WGAN uses Wasserstein distance as a loss function to measure the difference between the real data distribution and the generated data distribution. The loss function is implemented by calculating the distance between the generator and the discriminator. The network parameters are optimized by alternately training the generator and the discriminator.

9. The speaker anonymization method based on speech content desensitization and voiceprint anonymization according to claim 8, characterized in that: In each alternating training step, the generator is first fixed and the parameters of the discriminator are updated. Then, the discriminator is fixed and the parameters of the generator are updated. To meet the constraints of the Wasserstein distance, the parameters of the discriminator are usually gradient clipped to limit their gradient range. Finally, the samples generated by the generator are used to evaluate its performance.

10. The speaker anonymization method based on speech content desensitization and voiceprint anonymization according to claim 1, characterized in that: In step S7, the desensitized text and anonymized voiceprint are processed for speech synthesis based on the VITS speech synthesis model to obtain the final secure speech.

Images