A Strong Robust Speech Steganography Method, System, Medium and Device Based on a Frequency Domain Super-Resolution Network

Abstract

The present invention discloses a highly robust speech steganography method, system, medium and device based on a frequency-domain super-resolution network in the technical fields of speech steganography and signal processing, aiming to solve the problem that the existing technology is difficult to meet the actual needs. The method includes: preprocessing the obtained carrier speech and secret speech to obtain the amplitude spectrum of the carrier speech, the phase spectrum of the carrier speech, and the Mel spectrum of the secret speech, and obtaining the latent representation of the Mel spectrum of the secret speech by using a pre-trained frequency-domain super-resolution network; adding the latent representation of the Mel spectrum of the secret speech in the form of a residual to the amplitude spectrum of the carrier speech, and obtaining the stego-carrier speech through inverse short-time Fourier transform; preprocessing the stego-carrier speech to obtain the amplitude spectrum of the stego-carrier speech, and inputting it into a pre-trained decoder network to output the Mel spectrum for reconstructing the secret speech; inputting the Mel spectrum for reconstructing the secret speech into a pre-trained HiFi-GAN vocoder to output the time-domain waveform of the reconstructed secret speech.

Description

Technical Field

[0001] The present invention relates to the technical fields of voice steganography and signal processing, and particularly relates to a strong robustness voice steganography method, system, medium and device based on a frequency domain super-resolution network. Background Art

[0002] Voice steganography is a technology that hides secret information in ordinary, non-secret, and playable audio files. It is mainly used in the fields of secret communication of military intelligence and information encryption.

[0003] Traditional voice steganography methods represented by the least significant bit (LSB) and quantization index modulation (QIM) have the advantages of simple algorithms, easy implementation, and low computational complexity. However, they use a manual selection of redundancy to embed secret information, resulting in serious traces of modification and poor overall security. And they do not fully consider the characteristics of voice as secret information, resulting in insufficient reconstruction of the speaker identity characteristics of secret voice and insufficient steganography robustness.

[0004] With the application of deep learning technology, various voice steganography methods have emerged and achieved good results. However, these methods of hiding voice in voice often directly borrow image steganography methods, which require obtaining the representations of the voice carrier and the stego carrier through a deep neural network successively, making the model complex and the imperceptibility poor. Recently, various methods based on the residual structure have been proposed to simplify the embedding structure and improve the steganography effect. However, the encoders in these residual structures are designed for images and are not applicable to the voice spectrum, so the potential representation of the secret voice cannot be added to the carrier in the form of a residual. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a strong robustness voice steganography method, system, medium and device based on a frequency domain super-resolution network, which solves the technical problems of poor imperceptibility and complex model in existing voice steganography.

[0006] To solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] On the one hand, the present invention provides a strong robustness voice steganography method based on a frequency domain super-resolution network, including:

[0008] Preprocess the obtained carrier voice and secret voice respectively to obtain the amplitude spectrum of the carrier voice, the phase spectrum of the carrier voice, and the Mel spectrum of the secret voice;

[0009] According to the secret speech Melp, a pre-trained frequency domain super-resolution network is used to process and output a potential representation of the secret speech Melp;

[0010] Adding the latent representation of the secret speech Melp to the amplitude spectrum of the carrier speech in the form of a residual to obtain the amplitude spectrum of the secret carrier;

[0011] According to the amplitude spectrum of the encrypted carrier and the phase spectrum of the carrier speech, the encrypted carrier speech is obtained by inverse short-time Fourier transform;

[0012] Preprocessing the encrypted carrier speech to obtain an amplitude spectrum of the encrypted carrier speech;

[0013] Inputting the amplitude spectrum of the secret carrier speech into a pre-trained decoder network, and outputting the Mel spectrum of the reconstructed secret speech;

[0014] The Mel spectrum of the reconstructed secret speech is input into the pre-trained HiFi-GAN vocoder, and the time domain waveform of the reconstructed secret speech is output.

[0015] Optionally, the preprocessing of the acquired carrier voice and secret voice respectively includes:

[0016] The acquired carrier voice signal c = ( c [ 0 ], c [ 1 ], ⋯ , c [ N − 1 ]) Perform a short-time Fourier transform to obtain a complex matrix ,in The columns represent the Fourier transform of a given time frame, the rows represent the frame index, and the complex matrix The imaginary part is the phase spectrum of the carrier speech, let , get the amplitude spectrum of the carrier speech;

[0017] The secret voice signal obtained m = ( m [ 0 ], m [ 1 ], ⋯ , m [ N − 1 ]) , pre-emphasis, framing, Fourier transform, Mel filter bank, logarithmic operation and discrete cosine transform are performed in sequence to obtain the Mel signal of the secret speech;

[0018] The short-time Fourier transform is implemented by a differentiable Fourier transform module implemented by a convolutional layer.

[0019] Optionally, the frequency domain super-resolution network includes three modules, wherein the first module includes one causal convolutional network and three high-speed networks, the second module includes one transposed convolutional network and two high-speed networks, and the third module includes one causal convolutional network, two high-speed networks, one causal convolutional network, one high-speed network, one causal convolutional network, and one LeakyReLU activation function in sequence;

[0020] The processing using the pre-trained frequency-domain super-resolution network includes:

[0021] Input the Mel spectrogram of the secret speech into the first module of the frequency-domain super-resolution network, and after being processed by one causal convolutional network and three high-speed networks, obtain the first feature of the Mel spectrogram of the secret speech;

[0022] Input the first feature of the Mel spectrogram of the secret speech into the second module of the frequency-domain super-resolution network, and after being processed by one transposed convolutional network and two high-speed networks, obtain the second feature of the Mel spectrogram of the secret speech;

[0023] Input the second feature of the Mel spectrogram of the secret speech into the third module of the frequency-domain super-resolution network, and successively process it through one causal convolutional network, two high-speed networks, one causal convolutional network, one high-speed network, one causal convolutional network and one LeakyReLU activation function to obtain the latent feature of the Mel spectrogram of the secret speech.

[0024] Optionally, the preprocessing of the encrypted carrier speech includes:

[0025] Perform short-time Fourier transform on the encrypted carrier speech to obtain a complex matrix , where The columns of represent the Fourier transform of a given time frame, and the rows represent the frame index. Let , and obtain the amplitude spectrogram of the encrypted carrier speech;

[0026] The short-time Fourier transform is implemented by one differentiable Fourier transform module implemented by a convolutional layer.

[0027] Optionally, the decoder network includes two modules. The first module includes one long short-term memory network, three convolutional neural networks, batch normalization, two long short-term memory networks and one linear layer, and the second module includes a post-network;

[0028] The post-network includes a five-layer convolutional network, batch normalization and one Tanh activation function;

[0029] The input of the amplitude spectrogram of the encrypted carrier speech into the decoder network and the output of the reconstructed Mel spectrogram of the secret speech include:

[0030] Input the amplitude spectrogram of the encrypted carrier speech into the first module of the decoder network, and successively process it through one long short-term memory network, three convolutional neural networks, batch normalization, two long short-term memory networks and one linear layer to output the feature of the reconstructed Mel spectrogram of the secret speech;

[0031] Input the features of the reconstructed secret speech Mel spectrogram into the second module of the decoder network, and successively process them through a 5-layer convolutional network, batch normalization, and a Tanh activation function to output the features of the secret speech Mel spectrogram;

[0032] Add the features of the secret speech Mel spectrogram in the form of residuals to the features of the reconstructed secret speech Mel spectrogram to obtain the Mel spectrogram of the reconstructed secret speech.

[0033] Optionally, the frequency-domain super-resolution network and the decoder network are trained by the following loss function:

[0034]

[0035] wherein, represents the magnitude spectrum of the stego speech, represents the magnitude spectrum of the cover speech, represents the Mel spectrogram of the reconstructed secret speech, represents the Mel spectrogram of the secret speech.

[0036] In a second aspect, the present invention provides a strongly robust speech steganography system based on a frequency-domain super-resolution network, including:

[0037] A preprocessing module for: respectively preprocessing the obtained cover speech and secret speech to obtain the magnitude spectrum of the cover speech, the phase spectrum of the cover speech, and the Mel spectrogram of the secret speech; preprocessing the stego speech to obtain the magnitude spectrum of the stego speech;

[0038] A frequency-domain super-resolution network module for: processing the Mel spectrogram of the secret speech using a pre-trained frequency-domain super-resolution network to output a latent representation of the Mel spectrogram of the secret speech;

[0039] A secret speech embedding module for: adding the latent representation of the Mel spectrogram of the secret speech in the form of residuals to the magnitude spectrum of the cover speech to obtain the magnitude spectrum of the stego carrier; obtaining the stego speech through inverse short-time Fourier transform according to the magnitude spectrum of the stego carrier and the phase spectrum of the cover speech;

[0040] A decoder network module for: inputting the magnitude spectrum of the stego speech into a pre-trained decoder network to output the Mel spectrogram of the reconstructed secret speech;

[0041] A secret speech reconstruction module for: inputting the Mel spectrogram of the reconstructed secret speech into a pre-trained HiFi-GAN vocoder to output the time-domain waveform of the reconstructed secret speech.

[0042] In a third aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program / instructions, which when executed by a processor, implement the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network according to any one of the first aspects.

[0043] In a fourth aspect, the present invention provides a computer device / apparatus / system, comprising:

[0044] a memory for storing computer programs / instructions;

[0045] a processor for executing the computer programs / instructions to implement the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network according to any one of the first aspects.

[0046] In a fifth aspect, the present invention provides a computer program product comprising computer programs / instructions, characterized in that the computer programs / instructions, when executed by a processor, implement the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network according to any one of the first aspects.

[0047] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0048] 1. For the strong robust speech steganography method based on a frequency-domain super-resolution network provided by the present invention, by converting the secret speech into a mel-spectrum speech steganography method, it can better retain the speaker characteristics of the secret speech, thereby improving the detection rate of speaker consistency; for the first time, cross-domain redundant speech steganography is explored, and the method of embedding the mel-spectrum of the secret speech in the amplitude spectrum of the carrier speech is adopted, which has high speech quality and inaudibility; reconstructing the secret speech through a vocoder can well retain the content and speaker identity of the secret speech, and can well judge the accuracy of the secret speech from both the speaker and the content of the secret speech in covert communication;

[0049] 2. For the strong robust speech steganography system based on a frequency-domain super-resolution network provided by the present invention, by setting a preprocessing module, a frequency-domain super-resolution network module, a secret speech embedding module, a decoder network module, and a secret speech reconstruction module, it jointly realizes strong robust speech steganography based on a frequency-domain super-resolution network, which has practical significance and good application prospects;

[0050] 3. The computer-readable storage medium, computer device / apparatus / system, and computer program product provided by the present invention can execute the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network provided by the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 FIG. is a working flow chart of the strong robust speech steganography method based on a frequency-domain super-resolution network according to an embodiment of the present invention;

[0052] Figure 2 It is the network structure diagram of the frequency-domain super-resolution network provided according to the embodiments of the present invention;

[0053] Figure 3 It is the network structure diagram of the decoder network provided according to the embodiments of the present invention;

[0054] Figure 4 It is the schematic diagram of the signal-to-noise ratio SNR results of the stego carrier voice and the secret voice provided according to the embodiments of the present invention. Detailed implementation manners

[0055] The technical solution of the present invention will be described in detail below through the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present application and the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0056] It should be noted that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0057] Embodiment 1:

[0058] The embodiments of the present invention disclose a strong-robustness speech steganography method based on a frequency-domain super-resolution network. Referring to Figure 1 as shown, the specific steps are as follows:

[0059] S1. Respectively preprocess the obtained carrier voice and secret voice to obtain the amplitude spectrum of the carrier voice, the phase spectrum of the carrier voice, and the Mel cepstrum of the secret voice;

[0060] S2. Process according to the Mel cepstrum of the secret voice by using a pre-trained frequency-domain super-resolution network, and output the latent representation of the Mel cepstrum of the secret voice;

[0061] S3. Add the latent representation of the Mel cepstrum of the secret voice to the amplitude spectrum of the carrier voice in the form of a residual to obtain the amplitude spectrum of the stego carrier;

[0062] S4. According to the amplitude spectrum of the stego carrier and the phase spectrum of the carrier voice, obtain the stego carrier voice through inverse short-time Fourier transform;

[0063] S5. Preprocess the stego carrier voice to obtain the amplitude spectrum of the stego carrier voice;

[0064] S6. Input the amplitude spectrum of the encrypted carrier voice into a pre-trained decoder network to output the Mel spectrum of the reconstructed secret voice.

[0065] S7. Input the Mel spectrum of the reconstructed secret voice into a pre-trained HiFi-GAN vocoder to output the time-domain waveform of the reconstructed secret voice.

[0066] Specifically,

[0067] In step S1, the preprocessing of the obtained carrier voice and secret voice respectively includes:

[0068] Perform short-time Fourier transform on the obtained carrier voice signal composed of N samples c = ( c [ 0 ], c [ 1 ], ⋯ , c [ N − 1 ]) to obtain a complex matrix , where the columns of represent the Fourier transform of a given time frame, the rows represent the frame index, and the imaginary part of the complex matrix is the phase spectrum of the carrier voice. Let , and obtain the amplitude spectrum of the carrier voice;

[0069] Perform pre-emphasis, framing, Fourier transform, Mel filter bank, logarithmic operation, and discrete cosine transform on the obtained secret voice signal composed of N samples m = ( m [ 0 ], m [ 1 ], ⋯ , m [ N − 1 ]) , in sequence, to obtain the Mel spectrum of the secret voice;

[0070] The short-time Fourier transform is implemented by a differentiable Fourier transform module implemented by a convolutional layer.

[0071] In step S2, referring to Figure 2 shown in the network structure diagram of the frequency-domain super-resolution network provided in this embodiment, the frequency-domain super-resolution network includes 3 modules. The first module includes 1 causal convolutional network and 3 high-speed networks. The second module includes 1 transposed convolutional network and 2 high-speed networks. The third module sequentially includes 1 causal convolutional network, 2 high-speed networks, 1 causal convolutional network, 1 high-speed network, 1 causal convolutional network, and 1 LeakyReLU activation function.

[0072] The processing using the pre-trained frequency-domain super-resolution network includes:

[0073] Input the Mel spectrum of the secret voice into the first module of the frequency-domain super-resolution network, and after being processed by 1 causal convolutional network and 3 high-speed networks, obtain the first feature of the Mel spectrum of the secret voice.

[0074] Input the first feature of the secret speech Mel spectrogram into the second module of the frequency-domain super-resolution network. After being processed by 1 transposed convolutional network and 2 high-speed networks, obtain the second feature of the secret speech Mel spectrogram;

[0075] Input the second feature of the secret speech Mel spectrogram into the third module of the frequency-domain super-resolution network. After being successively processed by 1 causal convolutional network, 2 high-speed networks, 1 causal convolutional network, 1 high-speed network, 1 causal convolutional network, and 1 LeakyReLU activation function, obtain the latent feature of the secret speech Mel spectrogram.

[0076] In steps S3 and S4, the encrypted carrier speech is obtained through the following formula:

[0077]

[0078] Where, represents the encrypted carrier speech, represents the inverse short-time Fourier transform, represents the magnitude spectrum of the encrypted carrier, represents the magnitude spectrum of the carrier speech, represents the latent representation of the secret speech Mel spectrogram, is a complex matrix The imaginary part of, represents the phase spectrum of the carrier speech.

[0079] In step S5, the preprocessing of the encrypted carrier speech includes:

[0080] Perform short-time Fourier transform on the encrypted carrier speech to obtain a complex matrix , where The columns of represent the Fourier transform of a given time frame, and the rows represent the frame index. Let , to obtain the magnitude spectrum of the encrypted carrier speech; specifically, it is implemented through the following formula:

[0081]

[0082] Where, represents the magnitude spectrum of the encrypted carrier speech, represents the phase spectrum of the encrypted carrier speech, represents the short-time Fourier transform, represents the encrypted carrier speech.

[0083] The short-time Fourier transform is implemented through 1 differentiable Fourier transform module implemented by a convolutional layer.

[0084] In step S6, refer to Figure 3The figure shows the network structure diagram of the decoder network provided according to this embodiment. The decoder network includes 2 modules. The first module includes 1 long short-term memory network (LSTM), 3 convolutional neural networks, batch normalization, 2 long short-term memory networks, and a linear layer. The second module includes a post network.

[0085] The steps of inputting the magnitude spectrum of the encrypted carrier voice into the decoder network and outputting the mel spectrum of the reconstructed secret voice include:

[0086] Input the magnitude spectrum of the encrypted carrier voice into the first module of the decoder network, and successively process it through 1 long short-term memory network, 3 convolutional neural networks, batch normalization, 2 long short-term memory networks, and a linear layer to output the features of the mel spectrum of the reconstructed secret voice.

[0087] Input the features of the mel spectrum of the reconstructed secret voice into the second module of the decoder network, and successively process it through 5 convolutional layers, batch normalization, and a Tanh activation function to output the features of the mel spectrum of the secret voice.

[0088] Add the features of the mel spectrum of the secret voice to the features of the mel spectrum of the reconstructed secret voice in the form of residuals to obtain the mel spectrum of the reconstructed secret voice.

[0089] The decoder network uses LSTM to obtain the sequential dependence features of the input sequence over a long distance; passes through a stack of 3 convolutional layers and 2 unidirectional LSTM layers, and maps to the mel spectrum of the target reconstructed secret voice through the linear transformation of the linear layer; at the same time, passes the result of the linear transformation through the post network layer to enhance the effect of the decoder network in reconstructing the mel spectrum of the secret voice; adds the output result of the post network layer to the structure of the linear transformation in the form of residuals to obtain the mel spectrum of the reconstructed secret voice.

[0090] In step S7, the parameters of the pre-trained HiFi-GAN vocoder can be fine-tuned in combination with the dataset used.

[0091] In this embodiment, the frequency-domain super-resolution network and the decoder network are respectively trained using the losses of the carrier voice and the encrypted carrier voice, and the losses of the original secret voice mel spectrum and the reconstructed voice mel spectrum. Then, the trained networks are used for the embedding and reconstruction of the secret voice. The frequency-domain super-resolution network and the decoder network are trained through the following loss functions:

[0092]

[0093] Among them, represents the magnitude spectrum of the encrypted carrier voice, represents the magnitude spectrum of the carrier voice, represents the mel spectrum of the reconstructed secret voice, Mel spectrogram representing the secret speech.

[0094] To evaluate the effectiveness of the strongly robust speech steganography method based on the frequency-domain super-resolution network provided in this embodiment, a comparative experiment was conducted by comparing with the prior art:

[0095] This embodiment was compared among 4 speech steganography methods, namely Hide&speak, BNSNGAN, HyperSteg, and the speech steganography method 4S (Proposed) proposed in this embodiment.

[0096] All experiments were conducted on the LJSpeech and VCTK datasets. The LJSpeech dataset is a public-domain speech dataset, often used in the task of vocoder training. Each audio is a single-channel 16-bit PCM WAV with a sampling rate set to 22.05 kHz. The VCTK dataset is mainly used for voice conversion and automatic speaker verification. The dataset consists of 110 speakers with different English accents. Each speaker speaks approximately 400 sentences. All utterances were recorded at a sampling rate of 48 kHz using the same device and then uniformly downsampled to 22.05 kHz.

[0097] Speech steganography uses the signal-to-noise ratio SNR to measure inaudibility and intelligibility, and the calculation formula is as follows:

[0098] SNR = 10 log 10 ∑ i = 1 L Y 2 ( i ) ∑ i = 1 L [ Y ( i ) − Y ′ ( i )] 2

[0099] Among them, when calculating SNR to judge the inaudibility of the stego speech, the cover speech is the signal, the secret speech is used as the noise, and the stego cover speech is the processed signal;

[0100] By calculating SNR to judge the intelligibility of the secret speech, the original secret speech is the signal, the reconstructed secret speech is the processed signal, and the operations introduced by steganography are used as the noise.

[0101] To judge the content accuracy of the reconstructed secret speech and the speaker identity consistency, in the speech recognition task, the word error rate and character error rate are used to measure the speech recognition effect. The word error rate refers to the proportion of the number of incorrect words in the recognized text to the total number of words, and the character error rate refers to the proportion of the number of incorrect characters. The word error rate is marked as WER, and the character error rate is marked as CER. The calculation formulas are as follows:

[0102]

[0103]

[0104] Among them, represents the number of words that are inconsistent between the reconstructed secret speech and the original secret speech after speech recognition, represents the total number of words in the original secret speech, represents the number of characters that are inconsistent between the reconstructed secret speech and the original secret speech after speech recognition, represents the total number of characters in the original secret speech.

[0105] The speaker consistency between the reconstructed secret speech and the original secret speech is evaluated through accuracy, and the calculation formula is as follows:

[0106]

[0107] Among them, represents the total number of pairs between the reconstructed secret speech and the original secret speech, represents the number of pairs that are judged to have the same speaker by the speaker verification system, and Acc represents the accuracy.

[0108] Result analysis:

[0109] In the experiment, the carrier speech and the secret speech are embedded in a 1:1 ratio, with a duration of 1 s, n_fft = 1024 and hop_size = 256 for STFT, and num_mels = 80 when converted to Mel-spectrum. During training, the LJSpeech dataset is divided into 10,000 training sets, 3,000 validation sets, and 300 test sets; the parameters in the network are learned through the Adam optimization algorithm, with a learning rate of 0.0001, and the weights of the loss function are 0.45 and 0.55 respectively; finally, the Mel-spectrum of the secret speech is converted into speech through the Hifi-GAN vocoder, and the Hifi-GAN vocoder adopts the V1 structure and directly uses the provided pre-trained model.

[0110] To better reflect the superiority of the method proposed in this embodiment, a deep comparison is made with the current advanced speech steganography methods.

[0111] As shown in Table 1, it can be clearly seen that the SNR of the method proposed in this embodiment on the two datasets is much greater than this standard, and is 9.14 dB higher than the sub-optimal method; this indicates that the stego-voice fully meets the human auditory requirements and has good inaudibility. At the same time, the Loss is also at a relatively low level, indicating that the distortion of the stego-voice is very small. Although the loss of the pixinwav method on the cover is also very low, there is a large amount of "noise" in the stego-voice, making the stego-voice difficult to understand. The residual structure of the present invention based on SSRN can well embed the secret speech Mel spectrogram into the cover, and can generate stego-voice with low distortion and good inaudibility. In addition, the decoding network of this method can well reconstruct the Mel spectrogram of the secret speech from the amplitude signal, and the secret speech reconstructed by the Hifi-GAN vocoder also has a relatively high SNR, making it have better clarity.

[0112]

[0113] Explore the influence of key components in the model and the type of secret speech on the model performance. "-S": indicates that the secret speech is converted into a spectrogram through STFT; "-H": indicates removing the Highway activation function in the encoder structure; "-P": indicates removing the Postnet layer in the decoder structure and no longer connecting to the output in the form of residuals; "4S (Proposed)": the model proposed in this embodiment, and the experimental results of SNR are shown in Table 2:

[0114]

[0115] Without changing the model components and parameter settings, only set the carrier signal types to Mel spectrogram (Mel), magnitude spectrogram (Magnitude) in the frequency domain, phase (Phase), and the combination of magnitude and phase (Magnitude+Phase) respectively, and conduct embedding experiments on the dataset. The SNR results of the stego-carrier and the secret speech are as Figure 4 shown.

[0116] Through the automatic speech recognition framework, compare the original secret speech and the reconstructed speech content respectively recognized. At the same time, in order to better verify the accuracy of the content, the experiment is based on the open-source framework NVIDIA NeMo, and uses advanced Quartznet model, jasper model and HybridASR-TTS model for speech recognition, and calculates WER and CER. The results are shown in Table 3:

[0117]

[0118] Compared with the Hide&speak method, the secret speech content reconstructed by this method is more accurate, with smaller errors in WER and CER in the speech content recognized by three advanced ASR models. Hifi-GAN vocoder reconstructs the secret speech. At the same time, when the recipient of this method is inconvenient to receive the secret speech by voice, there is no need to reconstruct the secret speech through the vocoder, and the content can be accurately received directly in the form of text by using ASR. It is undeniable that the CER and WER of the three models have not reached the lowest, which may be because the selected speech is short during the test and the model is not fine-tuned on this dataset.

[0119] Use the Speaker Verification (SV) model to evaluate the consistency of the reconstructed secret speech and the original secret speech speaker identity. Adopt the open-source framework NVIDIA NeMo, and evaluate the accuracy of the reconstructed secret speech and the original secret speech speaker consistency through advanced SR models such as TitaNet, SpeakerNet, and ECAPA_TDNN. The results are shown in Table 4:

[0120]

[0121] Add different noise coefficients Additive white Gaussian noise (Gaussion) and speckle noise (Speckle). The SNR of the encrypted carrier speech and the reconstructed secret speech is used to reflect the imperceptibility after adding noise, and the reconstruction of the secret content is reflected by the accuracy of the speech content and the speaker of the reconstructed secret speech and the original secret speech, so as to measure the robustness of the model through the comprehensive situation.

[0122]

[0123] It can be found from Result Table 5 that the method proposed in this embodiment has good robustness for the model training with added noise. The SNR of the encrypted carrier speech and the reconstructed secret speech has not changed significantly, and the speaker of the reconstructed secret speech and the original secret speech always maintains good consistency.

[0124] In summary, for the strong robustness speech steganography method based on the frequency-domain super-resolution network provided in this embodiment, first, the carrier speech and the secret speech are respectively preprocessed to obtain the corresponding amplitude spectra. The spectrum of the secret speech is then passed through a Mel filter bank and normalized to obtain the Mel spectrum. Secondly, the latent representation of the secret speech Mel spectrum is obtained through an encoder composed of a frequency-domain super-resolution network, and combined with the amplitude spectrum of the carrier speech spectrum in a residual manner to form the stego-carrier speech. In the decoding stage, the reconstructed secret speech Mel spectrum is obtained from the stego-speech amplitude spectrum through a decoding network, and the reconstructed secret speech waveform is obtained by using a pre-trained HiFi-Gan vocoder. The loss of the carrier speech amplitude and the stego-carrier amplitude, the original secret speech Mel spectrum and the reconstructed secret speech Mel spectrum is calculated to optimize the parameters of the encoder and the decoding network. This method follows the idea of cross-domain redundancy exploration in speech steganography, and embeds the secret speech Mel spectrum into the carrier speech amplitude spectrum to achieve the embedding of the secret speech. High-quality stego-speech can be obtained, and the content and speaker identity of the secret speech can be well preserved by reconstructing the secret speech through the vocoder. In covert communication, the accuracy of the secret speech can be well judged from both the speaker and the content of the secret speech.

[0125] Embodiment 2:

[0126] Based on the same inventive concept as Embodiment 1, the present invention embodiment discloses a strong robustness speech steganography system based on a frequency-domain super-resolution network, specifically including:

[0127] A preprocessing module, configured to: respectively preprocess the obtained carrier speech and secret speech to obtain the amplitude spectrum of the carrier speech, the phase spectrum of the carrier speech, and the Mel spectrum of the secret speech; preprocess the stego-carrier speech to obtain the amplitude spectrum of the stego-carrier speech;

[0128] A frequency-domain super-resolution network module, configured to: process the Mel spectrum of the secret speech by using a pre-trained frequency-domain super-resolution network, and output the latent representation of the Mel spectrum of the secret speech;

[0129] A secret speech embedding module, configured to: add the latent representation of the Mel spectrum of the secret speech to the amplitude spectrum of the carrier speech in a residual form to obtain the amplitude spectrum of the stego-carrier; and obtain the stego-carrier speech through inverse short-time Fourier transform according to the amplitude spectrum of the stego-carrier and the phase spectrum of the carrier speech;

[0130] A decoder network module, configured to: input the amplitude spectrum of the stego-carrier speech into a pre-trained decoder network, and output the Mel spectrum of the reconstructed secret speech;

[0131] A secret speech reconstruction module, configured to: input the Mel spectrum of the reconstructed secret speech into a pre-trained HiFi-GAN vocoder, and output the reconstructed secret speech time-domain waveform.

[0132] For the specific function implementation of each of the above modules, refer to the relevant content in the method of Embodiment 1, which will not be elaborated here.

[0133] Embodiment 3:

[0134] This embodiment provides a computer-readable storage medium, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the steps of the strong robust voice steganography method based on a frequency-domain super-resolution network as described in any one of Embodiment 1 are implemented.

[0135] Embodiment 4:

[0136] This embodiment provides a computer device / equipment / system, including:

[0137] A memory for storing computer programs / instructions;

[0138] A processor for executing the computer programs / instructions to implement the steps of the strong robust voice steganography method based on a frequency-domain super-resolution network as described in any one of Embodiment 1.

[0139] Embodiment 5:

[0140] This embodiment provides a computer program product, including computer programs / instructions, characterized in that when the computer programs / instructions are executed by a processor, the steps of the strong robust voice steganography method based on a frequency-domain super-resolution network as described in any one of Embodiment 1 are implemented.

[0141] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0142] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing devices generate for implementing in the process Figure 1 each process or multiple processes and / or blocks Figure 1A device for the functions specified in one or more boxes.

[0143] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the process Figure 1 One process or multiple processes and / or boxes Figure 1 The functions specified in one box or multiple boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 One process or multiple processes and / or boxes Figure 1 The functions specified in one box or multiple boxes.

[0145] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the claims of the present invention. These all fall within the protection scope of the present invention.

Claims

1. A strong robustness speech steganography method based on a frequency domain super-resolution network, characterized in that Including: Preprocess the obtained carrier speech and secret speech respectively to obtain the amplitude spectrum of the carrier speech, the phase spectrum of the carrier speech, and the Mel spectrum of the secret speech; Process the Mel spectrum of the secret speech using a pre-trained frequency-domain super-resolution network to output the latent representation of the Mel spectrum of the secret speech; Add the latent representation of the Mel spectrum of the secret speech to the amplitude spectrum of the carrier speech in the form of a residual to obtain the amplitude spectrum of the encrypted carrier; According to the amplitude spectrum of the encrypted carrier and the phase spectrum of the carrier speech, obtain the encrypted carrier speech through inverse short-time Fourier transform; Preprocess the encrypted carrier speech to obtain the amplitude spectrum of the encrypted carrier speech; Input the amplitude spectrum of the encrypted carrier speech into a pre-trained decoder network to output the Mel spectrum of the reconstructed secret speech; Input the Mel spectrum of the reconstructed secret speech into a pre-trained HiFi-GAN vocoder to output the time-domain waveform of the reconstructed secret speech.

2. The strong robustness speech steganography method based on the frequency domain super-resolution network according to claim 1, wherein The preprocessing of the obtained carrier speech and secret speech respectively includes: The acquired carrier voice signal Perform a short-time Fourier transform to obtain a complex matrix ,in The columns represent the Fourier transform of a given time frame, the rows represent the frame index, and the complex matrix The imaginary part is the phase spectrum of the carrier speech, let , get the amplitude spectrum of the carrier speech; The obtained secret voice signal is successively subjected to pre-emphasis, framing, Fourier transform, Mel filter bank, logarithmic operation, and discrete cosine transform processing to obtain the Mel cepstrum of the secret voice; The short-time Fourier transform is implemented by a differentiable Fourier transform module implemented by a convolutional layer.

3. The strong robustness speech steganography method based on the frequency domain super-resolution network according to claim 1, characterized in that, The frequency-domain super-resolution network includes 3 modules. The first module includes 1 causal convolutional network and 3 high-speed networks. The second module includes 1 transposed convolutional network and 2 high-speed networks. The third module sequentially includes 1 causal convolutional network, 2 high-speed networks, 1 causal convolutional network, 1 high-speed network, 1 causal convolutional network, and 1 LeakyReLU activation function; The processing using the pre-trained frequency-domain super-resolution network includes: Input the Mel spectrum of the secret speech into the first module of the frequency-domain super-resolution network, and after being processed by 1 causal convolutional network and 3 high-speed networks, obtain the first feature of the Mel spectrum of the secret speech; Input the first feature of the Mel spectrum of the secret speech into the second module of the frequency-domain super-resolution network, and after being processed by 1 transposed convolutional network and 2 high-speed networks, obtain the second feature of the Mel spectrum of the secret speech; Input the second feature of the Mel spectrum of the secret speech into the third module of the frequency-domain super-resolution network, and sequentially after being processed by 1 causal convolutional network, 2 high-speed networks, 1 causal convolutional network, 1 high-speed network, 1 causal convolutional network, and 1 LeakyReLU activation function, obtain the latent feature of the Mel spectrum of the secret speech.

4. The strong robustness speech steganography method based on the frequency domain super-resolution network according to claim 1, characterized in that The preprocessing of the encrypted carrier speech includes: Perform a short-time Fourier transform on the encrypted carrier voice to obtain a complex matrix , where the columns represent the Fourier transform of a given time frame, and the rows represent the frame index. Let , to obtain the amplitude spectrum of the encrypted carrier voice; The short-time Fourier transform is implemented by a differentiable Fourier transform module implemented by a convolutional layer.

5. The strong robustness speech steganography method based on the frequency domain super-resolution network according to claim 1, characterized in that The decoder network includes 2 modules. The first module includes 1 long short-term memory network, 3 convolutional neural networks, batch normalization, 2 long short-term memory networks, and a linear layer. The second module includes a post-network; The post-network includes 5 layers of convolutional networks, batch normalization, and a Tanh activation function; Inputting the amplitude spectrum of the encrypted carrier speech into the decoder network and outputting the Mel spectrum of the reconstructed secret speech includes: Input the amplitude spectrum of the encrypted carrier voice into the first module of the decoder network, and successively process it through one long short-term memory network, three convolutional neural networks, batch normalization, two long short-term memory networks, and a linear layer to output the features of the reconstructed secret voice Mel spectrum; Input the features of the reconstructed secret voice Mel spectrum into the second module of the decoder network, and successively process it through five convolutional networks, batch normalization, and a Tanh activation function to output the features of the secret voice Mel spectrum; Add the features of the secret voice Mel spectrum in the form of residuals to the features of the reconstructed secret voice Mel spectrum to obtain the Mel spectrum of the reconstructed secret voice.

6. The strong robustness speech steganography method based on a frequency domain super-resolution network according to claim 1, wherein The frequency-domain super-resolution network and the decoder network are trained by the following loss function: ， Among them, represents the amplitude spectrum of the encrypted carrier voice, represents the amplitude spectrum of the carrier voice, represents the Mel spectrum of the reconstructed secret voice, represents the Mel spectrum of the secret voice.

7. A strongly robust speech steganography system based on a frequency-domain super-resolution network, characterized in that, Including: A preprocessing module for: respectively preprocessing the obtained carrier voice and secret voice to obtain the amplitude spectrum of the carrier voice, the phase spectrum of the carrier voice, and the Mel spectrum of the secret voice; preprocessing the obtained encrypted carrier voice to obtain the amplitude spectrum of the encrypted carrier voice; A frequency-domain super-resolution network module for: processing according to the Mel spectrum of the secret voice by using a pre-trained frequency-domain super-resolution network to output the latent representation of the Mel spectrum of the secret voice; A secret voice embedding module for: adding the latent representation of the Mel spectrum of the secret voice to the amplitude spectrum of the carrier voice in the form of residuals to obtain the amplitude spectrum of the encrypted carrier; and obtaining the encrypted carrier voice through inverse short-time Fourier transform according to the amplitude spectrum of the encrypted carrier and the phase spectrum of the carrier voice; A decoder network module for: inputting the amplitude spectrum of the encrypted carrier voice into a pre-trained decoder network to output the Mel spectrum of the reconstructed secret voice; A secret voice reconstruction module for: inputting the Mel spectrum of the reconstructed secret voice into a pre-trained HiFi-GAN vocoder to output the time-domain waveform of the reconstructed secret voice.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by a processor, the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network according to any one of claims 1-6 are implemented.

9. A computer device / apparatus / system, characterized in that, Including: A memory for storing computer programs / instructions; A processor for executing the computer programs / instructions to implement the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network according to any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the strong robust speech steganography method based on a frequency-domain super-resolution network according to any one of claims 1-6 are implemented.

Images