Voice anonymization and apparatus

Abstract

The invention provides a voice anonymization method, which comprises the following steps of: segmenting voice data into voice data fragments according to an activation detection result, and transferring the voice data fragments into a text sequence; comparing a sensitive word list with the text sequence to obtain a sensitive word, and replacing the sensitive word with a sensitive word replacement character to generate a desensitized text fragment; generating a text embedded characterization code according to the desensitized text fragment, and splicing the text embedded characterization code with a random voiceprint embedded characterization code to generate a spliced characterization code; and after the splicing representation code is decoded into a time-frequency spectrum through a backward decoding network, a vocoder converts the time-frequency spectrum into an audio waveform. The method has the beneficial effects that the voice data are desensitized and anonymized under the condition that the attributes of the voice data are not changed by modifying the voice voiceprint and proposing the sensitive words.

Description

Technical Field

[0001] The present invention relates to a speech processing technology, and in particular to a speech anonymization method. The present application also relates to a speech anonymization device. Background Art

[0002] Science and technology are developing at a rapid pace. With the continuous advancement of science and technology, the illegal collection and detection of personal information data generated by citizens in their lives and work is becoming more and more serious. Among them, personal voice information is particularly susceptible to illegal detection and is an important part of personal sensitive information.

[0003] In today's society, when citizens use technological means to interact with others in society, such as voice communication and surfing the Internet, it is very easy for them to send out their voice messages containing personal information. This personal information usually includes personal voiceprint information and sensitive content. In order to protect this personal information, many sound processing methods have been developed.

[0004] In the prior art, methods for processing voice to protect personal information are generally divided into three situations:

[0005] The first is voice transcription, which is based on voice data and converts voice data into text data, thereby removing personal voiceprint information. At the same time, the text data is filtered out by sensitive words to achieve anonymization of sensitive content. Although this method removes personal voiceprint information and sensitive content, it changes the original attributes of the voice data and turns it into text data, making the subsequent application and processing of the voice data difficult.

[0006] The second method is voice clipping. This method clips the voice data into shorter segments and achieves the effect of voice anonymization by changing the order of the voice or directly discarding part of it. However, this method converts the original voice data into voice data without practical meaning, affecting the availability of the data. At the same time, due to the advancement of voice detection technology, it is possible to locate the actual corresponding person of the voice data through the clipped voice segments, so this method cannot effectively perform voice anonymization processing.

[0007] The third method is voice changing and pitch changing. This method changes the voice and pitch of the voice by adjusting the voice spectrum characteristics, and anonymizes the voiceprint sensitive information of the voice data to a certain extent. Then, if the attacker obtains the voice changing and pitch changing software module, or obtains the voice changing and pitch changing adjustment parameters, de-anonymization can be achieved. In addition, this method cannot desensitize or anonymize the sensitive content of the voice data. Summary of the invention

[0008] In order to solve the above technical problems, the present application provides a voice anonymization method, which can realize effective voice desensitization and anonymization operations under the data attributes of voice data. At the same time, the present application also provides a face anonymization device.

[0009] The present application provides a method for anonymizing speech, comprising:

[0010] Segmenting the voice data into voice data segments according to the activation detection results, and transcribing the voice data segments into text sequences;

[0011] Comparing the sensitive word list with the text sequence to obtain sensitive words, replacing the sensitive words with sensitive word replacement symbols, and generating a desensitized text segment;

[0012] Generate a text embedding representation code according to the desensitized text segment, and concatenate it with the random voiceprint embedding representation code to generate a concatenated representation code;

[0013] After the concatenated representation code is decoded into a time-frequency spectrum by a backward decoding network, the vocoder converts the time-frequency spectrum into an audio waveform.

[0014] Optionally, the activation detection result refers to distinguishing silence and real sound in the voice data.

[0015] Optionally, the comparison between the sensitive word list and the text sequence refers to comparing the characters or words of each transcription result in the sensitive word list in turn by a fuzzy filtering method.

[0016] Optionally, the comparison between the sensitive word list and the text sequence refers to judging by pinyin approximation.

[0017] Optionally, the sensitive word replacement symbol includes: a specific symbol, a space or a phrase.

[0018] Optionally, the text sequence includes: a text result sequence and a pinyin result sequence.

[0019] Optionally, generating the text embedding representation code comprises the following steps:

[0020] Encoding the text sequence into a one-hot vector code;

[0021] The one-hot vector code is converted into a text embedding representation code through a deep neural network.

[0022] Optionally, before the vocoder converts the time-frequency spectrum into an audio waveform, the vocoder further performs the following steps:

[0023] The input concatenated mask and the output time-frequency spectrum are aligned.

[0024] Optionally, the audio waveforms are spliced ​​in sequence into anonymized speech.

[0025] The present application also provides a voice anonymization device, comprising:

[0026] A transcription module, used for dividing the voice data into voice data segments according to the activation detection results, and transcribing the voice data segments into text sequences;

[0027] A matching module is used to compare the sensitive word list with the text sequence to obtain sensitive words, replace the sensitive words with sensitive word replacement symbols, and generate a desensitized text segment;

[0028] An encoding module, used for generating a text embedding representation code according to the desensitized text segment, and splicing it with the random voiceprint embedding representation code to generate a spliced ​​representation code;

[0029] The synthesis module is used to decode the concatenated representation code into a time-frequency spectrum through a backward decoding network, and then the vocoder converts the time-frequency spectrum into an audio waveform.

[0030] Optionally, the activation detection result refers to distinguishing silence and real sound in the voice data.

[0031] Optionally, the comparison between the sensitive word list and the text sequence refers to comparing the characters or words of each transcription result in the sensitive word list in turn by a fuzzy filtering method.

[0032] Optionally, the comparison between the sensitive word list and the text sequence refers to judging by pinyin approximation.

[0033] Optionally, the sensitive word replacement symbol includes: a specific symbol, a space or a phrase.

[0034] Optionally, the text sequence includes: a text result sequence and a pinyin result sequence.

[0035] Optionally, the encoding module further includes:

[0036] An encoding unit, configured to encode the text sequence into a one-hot vector code;

[0037] A neural network unit is used to convert the one-hot vector code into a text embedding representation code through a deep neural network.

[0038] Optionally, also include:

[0039] An alignment module is used to align the input concatenated symbol code and the output time-frequency spectrum.

[0040] Optionally, also include:

[0041] A splicing module is used to splice the audio waveforms into anonymized speech in sequence.

[0042] The advantages of this application over the prior art are as follows:

[0043] The present application provides a method for anonymizing speech, which divides speech data into speech data segments according to activation detection results, transcribes the speech data segments into text sequences; compares the sensitive word list with the text sequence to obtain sensitive words, replaces the sensitive words with sensitive word replacement symbols, and generates desensitized text segments; generates text embedding representation codes according to the desensitized text segments, and splices them with random voiceprint embedding representation codes to generate spliced ​​representation codes; after decoding the spliced ​​representation codes into time-frequency spectra through a backward decoding network, the vocoder converts the time-frequency spectra into audio waveforms. By modifying the speech voiceprint and proposing sensitive words, the speech data can be desensitized and anonymized without changing the attributes of the speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the speech anonymization processing in this application.

[0045] Figure 2 It is a flow chart of voice data anonymization in this application.

[0046] Figure 3 It is a schematic diagram of the speech anonymization processing device in this application. DETAILED DESCRIPTION

[0047] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0048] This application mainly provides a voice anonymization method, including: dividing the voice data into voice data segments according to the activation detection results, transcribing the voice data segments into text sequences; comparing the sensitive word list with the text sequence to obtain sensitive words, replacing the sensitive words with sensitive word replacement symbols, and generating desensitized text segments; generating text embedding representation codes according to the desensitized text segments, splicing with random voiceprint embedding representation codes, and generating splicing representation codes; after decoding the splicing representation codes into time-frequency spectra through a backward decoding network, the vocoder converts the time-frequency spectra into audio waveforms. Through the processing of the above steps, the attributes of the voice data are not changed, and the voiceprint information of the voice and the sensitive words in the voice are removed.

[0049] Figure 1 is a schematic diagram of voice anonymization processing in this application, Figure 2 It is a flow chart of voice data anonymization in this application.

[0050] Please refer to Figure 1 as well as Figure 2As shown, the speech anonymization process in this application mainly includes:

[0051] Step S101 divides the voice data into voice data segments according to the activation detection results, and transcribes the voice data segments into text sequences;

[0052] Voice activation detection (VDA) is a detection method for distinguishing between voice signals and noise signals for voice input signals. In the present application, the voice activation detection is mainly used to distinguish between the silent and actual sound parts in a voice sequence, and the activation detection result is the result of distinguishing between the silent and actual sound parts.

[0053] After the activation detection result is obtained, the voice data is segmented into voice data segments according to the result, and then each segment of the voice data is transcribed into a text sequence. The text sequence mainly includes two forms, one is a text result sequence, and the other is a pinyin result sequence.

[0054] S102 compares the sensitive word list with the text sequence to obtain sensitive words, replaces the sensitive words with sensitive word replacement symbols, and generates a desensitized text segment;

[0055] Transcribing a voice data segment into a text result sequence refers to transcribing the voice data segment into a text segment, so that the sensitivity of characters and words can be matched according to a predetermined sensitive word list, and the sensitive word content in the voice data can be identified.

[0056] The sensitive word matching is to match the words in the text result sequence in sequence by a fuzzy filtering method, and find out the sensitive words in the text result sequence that are the same or similar to those in the sensitive word list.

[0057] The above is a method for matching sensitive words after transcribing into a text result sequence. In addition, this application also provides a method for judging pinyin myopia.

[0058] The transcription of the voice data segment into a pinyin result sequence refers to the transcription of the voice data segment into a pinyin segment, and the pinyin can be the pinyin of the text result sequence or the pronunciation of the voice data.

[0059] Pinyin similarity judgment means that one pinyin can represent the pronunciation of many characters, so judging by pinyin can achieve redundancy of phonetic transcription errors.

[0060] Through the above steps, after finding the same or similar characters or words in the sensitive word list, the sensitive words are replaced with sensitive word replacement symbols to achieve anonymization of sensitive content. The sensitive word replacement symbol in this application refers to: a specific symbol, space or phrase, such as pinyin a.

[0061] After the sensitive words in the text segment are replaced, the text segment becomes a desensitized text segment without sensitive words.

[0062] S103 generates a text embedding token according to the desensitized text segment, and concatenates it with the random voiceprint embedding token to generate a concatenated token;

[0063] The above-mentioned desensitized text fragment is a text fragment with sensitive content removed. The next step is to remove the personal voiceprint in the desensitized text fragment, and the method used is the voiceprint interference method.

[0064] The desensitized text fragment is encoded into a one-hot vector code by one-hot encoding, and then a data processing model based on a convolutional neural network, a residual network, a feedforward neural network or a bidirectional recurrent structure neural network is used to generate a text embedding characterization code with fixed latitude, stronger representation ability and a length corresponding to the length of the desensitized text fragment.

[0065] After obtaining the text embedding mask, a random voiceprint embedding mask is constructed, and the text embedding mask and the random voiceprint embedding mask are concatenated to form a concatenated mask. By accessing the random voiceprint embedding mask, the personal voiceprint information is changed, so the voiceprint information is anonymized.

[0066] S104 After the concatenated representation code is decoded into a time-frequency spectrum through a backward decoding network, the vocoder converts the time-frequency spectrum into an audio waveform.

[0067] The concatenated mask has anonymized the voiceprint information, so the concatenated mask needs to be reversely restored and finally synthesized into voice data.

[0068] First, the concatenated representation code needs to be decoded and decoded into multi-step Mel filter bank coefficients through a recurrent neural network. Then, a time-frequency spectrum is generated through a backward decoding network. The backward decoding network is composed of the results of a convolutional neural network, a residual network, a feedforward neural network, and a double recurrent neural network.

[0069] In the process of generating the time spectrum, the input concatenated representation code and the output time spectrum are aligned through the attention mechanism of the recurrent neural network. Then the time spectrum is converted into an audio waveform through a vocoder to realize speech synthesis.

[0070] After performing the above operations on each voice data segment, the audio waveforms are spliced ​​in sequence to form anonymized voice data.

[0071] The above is the voice anonymization method in the present application. On this basis, the present application also provides a voice anonymization device.

[0072] Figure 3It is a schematic diagram of the speech anonymization processing device in this application.

[0073] Please refer to Figure 3 As shown, a speech anonymization device includes: a transcription module 101, a matching module 102, an encoding module 103, and a synthesis module 104.

[0074] A transcription module 101, configured to segment the speech data into speech data segments according to the activation detection results, and transcribe the speech data segments into text sequences;

[0075] In the present application, the speech activation detection is mainly used to distinguish between silence and actual sound parts in a speech sequence, and the activation detection result is the result of distinguishing between silence and actual sound parts.

[0076] After the activation detection result is obtained, the voice data is segmented into voice data segments according to the result, and then each segment of the voice data is transcribed into a text sequence. The text sequence mainly includes two forms, one is a text result sequence, and the other is a pinyin result sequence.

[0077] Matching module 102. Used to compare the sensitive word list with the text sequence to obtain sensitive words, replace the sensitive words with sensitive word replacement symbols, and generate desensitized text segments;

[0078] Transcribing a voice data segment into a text result sequence refers to transcribing the voice data segment into a text segment, so that the sensitivity of characters and words can be matched according to a predetermined sensitive word list, and the sensitive word content in the voice data can be identified.

[0079] The sensitive word matching is to match the words in the text result sequence in sequence by a fuzzy filtering method, and find out the sensitive words in the text result sequence that are the same or similar to those in the sensitive word list.

[0080] The above is a method for matching sensitive words after transcribing into a text result sequence. In addition, this application also provides a method for judging pinyin myopia.

[0081] The transcription of the voice data segment into a pinyin result sequence refers to the transcription of the voice data segment into a pinyin segment, and the pinyin can be the pinyin of the text result sequence or the pronunciation of the voice data.

[0082] Pinyin similarity judgment means that one pinyin can represent the pronunciation of many characters, so judging by pinyin can achieve redundancy of phonetic transcription errors.

[0083] After finding the same or similar characters or words in the sensitive word list, the sensitive words are replaced with sensitive word replacement symbols to achieve anonymization of sensitive content. The sensitive word replacement symbol in this application refers to: a specific symbol, space or phrase, such as pinyin a.

[0084] After the sensitive words in the text segment are replaced, the text segment becomes a desensitized text segment without sensitive words.

[0085] The encoding module 103 is used to generate a text embedding representation code according to the desensitized text segment, and concatenate it with the random voiceprint embedding representation code to generate a concatenated representation code;

[0086] The above-mentioned desensitized text fragment is a text fragment with sensitive content removed. The next step is to remove the personal voiceprint in the desensitized text fragment, and the method used is the voiceprint interference method.

[0087] The encoding module also includes: an encoding unit, used to encode the text sequence into a one-hot vector code; and a neural network unit, used to convert the one-hot vector code into a text embedding representation code through a deep neural network.

[0088] The desensitized text fragment is encoded into a one-hot vector code by one-hot encoding, and then a data processing model based on a convolutional neural network, a residual network, a feedforward neural network or a bidirectional recurrent structure neural network is used to generate a text embedding characterization code with fixed latitude, stronger representation ability and a length corresponding to the length of the desensitized text fragment.

[0089] After obtaining the text embedding mask, a random voiceprint embedding mask is constructed, and the text embedding mask and the random voiceprint embedding mask are concatenated to form a concatenated mask. By accessing the random voiceprint embedding mask, the personal voiceprint information is changed, so the voiceprint information is anonymized.

[0090] The synthesis module 104 is configured to decode the concatenated representation code into a time-frequency spectrum through a backward decoding network, and then convert the time-frequency spectrum into an audio waveform through a vocoder.

[0091] The concatenated mask has anonymized the voiceprint information, so the concatenated mask needs to be reversely restored and finally synthesized into voice data.

[0092] First, the concatenated representation code needs to be decoded and decoded into multi-step Mel filter bank coefficients through a recurrent neural network. Then, a time-frequency spectrum is generated through a backward decoding network. The backward decoding network is composed of the results of a convolutional neural network, a residual network, a feedforward neural network, and a double recurrent neural network.

[0093] In this application, the voice anonymization device also includes:

[0094] The alignment module is used to align the input concatenated representation code and the output time spectrum. In the process of generating the time spectrum, the input concatenated representation code and the output time spectrum are aligned through the attention mechanism of the recurrent neural network. Then, the time spectrum is converted into an audio waveform through a vocoder to realize speech synthesis.

[0095] The splicing module is used to splice the audio waveforms into anonymized speech in sequence. After performing the above operation on each speech data segment, the audio waveforms are spliced ​​in sequence to form anonymized speech data.

Claims

1. A method for anonymizing speech, characterized in that: include: Segmenting the voice data into voice data segments according to the activation detection results, and transcribing the voice data segments into text sequences; Comparing the sensitive word list with the text sequence to obtain sensitive words, replacing the sensitive words with sensitive word replacement symbols, and generating a desensitized text segment; Generate a text embedding representation code according to the desensitized text segment, and concatenate it with the random voiceprint embedding representation code to generate a concatenated representation code; After the concatenated representation code is decoded into a time-frequency spectrum by a backward decoding network, the vocoder converts the time-frequency spectrum into an audio waveform.

2. The speech anonymization method according to claim 1, characterized in that: The activation detection result refers to distinguishing silence and real sound in the voice data.

3. The speech anonymization method according to claim 1, characterized in that: The comparison between the sensitive word list and the text sequence refers to comparing the characters or words of each transcription result in the sensitive word list in sequence by a fuzzy filtering method.

4. The speech anonymization method according to claim 1, characterized in that: The comparison between the sensitive word list and the text sequence refers to judging by using pinyin approximation.

5. The method for voice anonymization according to claim 1, characterized in that: The sensitive word replacement symbols include: specific symbols, spaces or phrases.

6. The method for anonymizing speech according to claim 1, characterized in that: The text sequence includes: a text result sequence and a pinyin result sequence.

7. The method for anonymizing speech according to claim 1, characterized in that: The generating of the text embedding code comprises the following steps: Encoding the desensitized text segment into a one-hot vector code; The one-hot vector code is converted into a text embedding representation code through a deep neural network.

8. The method for voice anonymization according to claim 1, characterized in that: Before the vocoder converts the time-frequency spectrum into an audio waveform, the vocoder further performs the following steps: The input concatenated mask and the output time-frequency spectrum are aligned.

9. The method for voice anonymization according to claim 1, characterized in that: The audio waveforms are concatenated in sequence into anonymized speech.

10. A speech anonymization device, characterized in that: include: A transcription module, used for dividing the voice data into voice data segments according to the activation detection results, and transcribing the voice data segments into text sequences; Matching module. Used to compare the sensitive word list with the text sequence to obtain sensitive words, replace the sensitive words with sensitive word replacement symbols, and generate desensitized text fragments; An encoding module, used for generating a text embedding representation code according to the desensitized text segment, and splicing it with the random voiceprint embedding representation code to generate a spliced ​​representation code; The synthesis module is used to decode the concatenated representation code into a time-frequency spectrum through a backward decoding network, and then the vocoder converts the time-frequency spectrum into an audio waveform.

11. The speech anonymization device according to claim 10, characterized in that: The activation detection result refers to distinguishing silence and real sound in the voice data.

12. The speech anonymization device according to claim 10, characterized in that: The comparison between the sensitive word list and the text sequence refers to comparing the characters or words of each transcription result in the sensitive word list in sequence by a fuzzy filtering method.

13. The speech anonymization device according to claim 10, characterized in that: The comparison between the sensitive word list and the text sequence refers to judging by using pinyin approximation.

14. The speech anonymization device according to claim 10, characterized in that: The sensitive word replacement symbols include: specific symbols, spaces or phrases.

15. The speech anonymization device according to claim 10, characterized in that: The text sequence includes: a text result sequence and a pinyin result sequence.

16. The speech anonymization device according to claim 10, characterized in that: The encoding module also includes: An encoding unit, configured to encode the text sequence into a one-hot vector code; A neural network unit is used to convert the one-hot vector code into a text embedding representation code through a deep neural network.

17. The speech anonymization device according to claim 10, characterized in that: Also includes: An alignment module is used to align the input concatenated symbol code and the output time-frequency spectrum.

18. The speech anonymization device according to claim 10, characterized in that: Also includes: A splicing module is used to splice the audio waveforms into anonymized speech in sequence.

Images