Voice hiding method and system based on Flow-Flow architecture, medium and equipment

Abstract

The invention discloses a Flow-Flow architecture-based voice hiding method and system, a medium and equipment in the technical field of information hiding, and aims to solve the optimization problem of the existing information hiding technology. The method comprises the following steps: acquiring a Mel spectrogram of a to-be-hidden carrier text; mapping the secret voice to a hidden variable by using a FlowMap mapper; according to the hidden variable and the Mel spectrogram of the to-be-hidden carrier text, voice hiding is carried out through a FlowVocoder voice generator, and secret-carrying voice is obtained; performing text format conversion on the encrypted voice to obtain a Mel spectrogram of the encrypted voice in the text format; restoring the Mel spectrogram of the encrypted voice and the Mel spectrogram of the encrypted voice in the text format into hidden variables through a FlowVocoder voice inverter; and mapping the restored hidden variables into restored secret voice by using a FlowMap inverse mapper. According to the invention, the problem of secret voice extraction quality reduction caused by secret-carrying voice distortion is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of information hiding, and particularly relates to a voice hiding method, system, medium and device based on a Flow-Flow architecture. Background Art

[0002] Different from the voice steganography technology that re-hides the bitstream in the voice, voice hiding is a technology that embeds the secret voice into the carrier voice for covert communication, and requires the hiding and recovery of the secret voice to be realized under the premise of imperceptibility. The existing voice hiding methods based on deep learning usually hide the secret voice in natural voice, which will inevitably leave embedding traces, resulting in weak security. At the same time, there will be problems such as the imbalance between the quality of the encrypted voice and the quality of the recovered secret voice and difficulty in resisting lossy compression. The current generative voice steganography methods mainly embed the bitstream during the voice generation process, and the neural vocoder models used have large parameters and require a large amount of GPU resources for training, making it difficult to be deployed on mobile devices; in addition, the quality of the generated encrypted voice is not high. Finally, the current voice hiding scheme does not consider the problem that it is difficult to recover the secret voice due to lossy compression during the transmission of the encrypted voice, and lacks a robust design against lossy compression. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art, and provide a voice hiding method, system, medium and device based on a Flow-Flow architecture, which integrates the secret voice into the voice synthesis process in an imperceptible manner, so as to generate the encrypted voice, ensuring that the secret voice can be recovered with high quality even when the carrier voice is distorted.

[0004] To solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0005] The present invention provides a robust voice hiding method based on a Flow-Flow architecture, including:

[0006] Obtain the Mel spectrogram of the carrier text to be hidden;

[0007] Input the secret voice and the Mel spectrogram of the carrier text to be hidden into the hiding network of the Flow-Flow architecture: use the FlowMap mapper to map the secret voice to the latent variable; according to the latent variable and the Mel spectrogram of the carrier text to be hidden, perform voice hiding through the FlowVocoder voice generator to obtain the encrypted voice;

[0008] Perform text format conversion on the encrypted voice to obtain the Mel spectrogram of the encrypted voice in text format;

[0009] Input the encrypted voice and its Mel spectrogram into the recovery network of the Flow-Flow architecture: Restore the encrypted voice and the Mel spectrogram of the encrypted voice in text format to latent variables through the FlowVocoder voice inverter; Use the FlowMap inverse mapper to map the restored latent variables to the recovered secret voice.

[0010] Optionally, the Mel spectrogram of the text of the carrier to be hidden and the Mel spectrogram of the encrypted voice in text format are obtained through the spectral generation network SGN, and the text format conversion is implemented through automatic speech recognition ASR.

[0011] Optionally, the FlowMap adopts a reversible speech mapper based on Flow, including 8 flow modules;

[0012] The flow module includes a shared density estimator, a flow indicator embedding module, a parameter prediction network, and a mixture cumulative distribution transformation;

[0013] The flow indicator embedding module adopts manifold embedding to map the index of each flow layer to a high-dimensional vector through a learnable lookup table;

[0014] The shared density estimator adopts a two-dimensional dilated convolution structure with a convolution kernel size of 3×3 and the dilation factor increasing as a power of 2; A gated activation mechanism is introduced after each layer of convolution, and the gradient vanishing is alleviated through residual connection; The gated activation mechanism includes tanh activation and sigmoid gating;

[0015] The parameter prediction network includes a fully connected network for generating transformation parameters;

[0016] The mixture cumulative distribution transformation includes a weighted sum of 32 distributions for outputting latent variables through probability density estimation.

[0017] Optionally, the data processing flow of the FlowMap mapper includes:

[0018] Perform reverse processing on the secret voice to obtain the reversed secret voice;

[0019] Perform time dimension inversion and width dimension displacement on the reversed secret voice to obtain the preprocessed secret voice;

[0020] According to the preprocessed secret voice, obtain the latent features of the secret voice through the shared density estimator and the flow indicator embedding module;

[0021] According to the latent features of the secret voice, generate transformation parameters through the parameter prediction network, and the transformation parameters include a scaling factor and a translation factor , Mixed weight , Mean and Standard deviation ;

[0022] According to the reverse secret speech, mixed weight , Mean and Standard deviation , calculate the mixed logistic CDF and mixed logistic PDF through the mixed cumulative distribution transformation;

[0023] According to the mixed logistic CDF, obtain the secret speech affine transformation parameters and log determinant calculation parameters through probability integral and Logit transformation;

[0024] According to the secret speech affine transformation parameters, scaling factor and translation factor perform affine transformation to obtain the latent variable;

[0025] According to the log determinant calculation parameters, mixed logistic PDF and scaling factor , calculate the log determinant.

[0026] Optionally, the potential feature of the secret speech has the following calculation formula:

[0027]

[0028] where, represents the shared density estimator, represents the flow indicator embedding module, represents the number of flow layers;

[0029] The calculation formula of the mixed logistic CDF is:

[0030]

[0031] where, represents the cumulative distribution function;

[0032] The calculation formula of the mixed logistic PDF is:

[0033]

[0034] where, represents the probability density function;

[0035] The calculation formulas of the secret speech affine transformation parameters and log determinant calculation parameters are:

[0036]

[0037] where, Represents the secret speech affine transformation parameters, Represents the logarithmic determinant calculation parameters;

[0038] The calculation formula for the hidden variable is:

[0039]

[0040] Among them, Represents the hidden variable;

[0041] The calculation formula for the logarithmic determinant is:

[0042]

[0043] Among them, Represents the logarithmic determinant.

[0044] Optionally, the FlowVocoder speech generator and speech reverser adopt a lightweight model with an autoregressive flow structure;

[0045] The formula for generating encrypted speech by the FlowVocoder speech generator is:

[0046]

[0047] Among them, Represents the encrypted speech, Represents the encryption operation, Represents the hidden variable, Represents the Mel spectrogram of the carrier text to be hidden;

[0048] The formula for restoring encrypted speech by the FlowVocoder speech reverser is:

[0049]

[0050] Among them, Represents the restored hidden variable, Represents the restoration operation, Represents the encrypted speech to be restored, Represents the Mel spectrogram of the encrypted speech in text format.

[0051] Optionally, the Flow-Flow architecture adopts an asynchronous training strategy for model training, including:

[0052] During the hiding stage, the FlowMap mapper and the FlowVocoder speech generator are respectively trained using the maximum likelihood function as the loss function. The latent variables of the FlowMap mapper and the FlowVocoder speech generator are set to the same dimension, and the trained FlowMap mapper and FlowVocoder speech generator are obtained;

[0053] During the recovery stage, the FlowMap inverse mapper and the FlowVocoder speech inverter are initialized based on the pre-trained weights in the hiding stage; the FlowMap inverse mapper and the FlowVocoder speech inverter are fine-tuned using the L1 norm loss function to obtain the trained FlowMap inverse mapper and FlowVocoder speech inverter; the L1 norm loss function is:

[0054]

[0055] where, represents the loss of the FlowMap inverse mapper, represents the loss of the FlowVocoder speech inverter, represents the restored latent variable, represents the latent variable, represents the mel spectrogram conversion function, represents the restored secret speech, represents the secret speech.

[0056] In a second aspect, the present invention provides a robust speech hiding device based on a Flow-Flow architecture, including:

[0057] A mel spectrogram conversion module for: obtaining the mel spectrogram of the carrier text to be hidden; obtaining the mel spectrogram of the text-format encrypted speech;

[0058] A speech hiding module for: using the FlowMap mapper to map the secret speech to a latent variable; performing speech hiding on the basis of the latent variable and the mel spectrogram of the carrier text to be hidden through the FlowVocoder speech generator to obtain encrypted speech;

[0059] A text format conversion module for: performing text format conversion on the encrypted speech;

[0060] A speech recovery module for: restoring the encrypted speech and the mel spectrogram of the text-format encrypted speech to a latent variable through the FlowVocoder speech inverter; using the FlowMap inverse mapper to map the restored latent variable to the restored secret speech.

[0061] In a third aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of the robust voice hiding method based on the Flow-Flow architecture described in any one of the first aspects are implemented.

[0062] In a fourth aspect, the present invention provides a computer device, including:

[0063] a memory for storing computer instructions;

[0064] a processor for executing the computer instructions to implement the steps of the robust voice hiding method based on the Flow-Flow architecture described in any one of the first aspects.

[0065] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0066] 1. By introducing a flow model into voice hiding and embedding secret information during the voice synthesis process, the modification traces of natural speech in traditional methods are avoided.

[0067] 2. The latent variables generated by FlowMap follow a standard Gaussian distribution, making the distribution of the voice carrying the secret almost indistinguishable from that of normal speech, enhancing the security of the secret voice.

[0068] 3. In order to cope with the possible distortion of the voice carrying the secret during transmission, such as lossy compression and noise interference, an asynchronous training strategy is adopted to optimize the Flow-Flow architecture. The hiding network and the recovery network are trained respectively for different distortion situations, solving the problem of the degradation of the quality of the extracted secret voice due to the distortion of the voice carrying the secret.

[0069] 4. Both the FlowMap model and the flow-based vocoder FlowVocoder adopt a lightweight design, significantly reducing the number of model parameters and being suitable for deployment on resource-constrained devices. Description of the Drawings

[0070] Figure 1 It is a flowchart of the robust voice hiding method based on the Flow-Flow architecture provided according to an embodiment of the present invention;

[0071] Figure 2 It is a Flow-Flow architecture diagram provided according to an embodiment of the present invention;

[0072] Figure 3 It is a structural diagram of a flow module provided according to an embodiment of the present invention;

[0073] Figure 4 It is a structural diagram of a shared density estimator provided according to an embodiment of the present invention;

[0074] Figure 5Data processing flowchart of the FlowMap mapper provided according to an embodiment of the present invention. Detailed implementation manners

[0075] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present patent are detailed descriptions of the technical solution of the present patent, rather than limitations on the technical solution of the present patent. Without conflict, the technical features in the embodiments of the present patent and the embodiments can be combined with each other.

[0076] It should be noted that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the front and back associated objects.

[0077] Embodiment 1:

[0078] An embodiment of the present invention discloses a robust voice hiding method based on a Flow-Flow architecture. Referring to Figure 1 as shown, the specific steps are as follows:

[0079] S1. Obtain the Mel spectrogram of the carrier text to be hidden;

[0080] S2. Input the secret voice and the Mel spectrogram of the carrier text to be hidden into the hiding network of the Flow-Flow architecture: map the secret voice to a latent variable using the FlowMap mapper; perform voice hiding on the basis of the latent variable and the Mel spectrogram of the carrier text to be hidden through the FlowVocoder voice generator to obtain the encrypted voice;

[0081] S3. Perform text format conversion on the encrypted voice to obtain the Mel spectrogram of the encrypted voice in text format;

[0082] S4. Input the encrypted voice and the Mel spectrogram of the encrypted voice in text format into the recovery network of the Flow-Flow architecture: restore the encrypted voice and the Mel spectrogram of the encrypted voice in text format to a latent variable through the FlowVocoder voice reverser; map the restored latent variable to the recovered secret voice using the FlowMap inverse mapper.

[0083] Specifically, in step S1, the carrier text to be hidden is converted into a Mel spectrogram through the spectral generation network SGN ; in this embodiment, two text-to-speech TTS models, FastSpeech and Tacotron2, are used to convert the carrier text Convert to Mel spectrogram , which have publicly available pre-trained models and dictionaries that can be directly used to generate Mel spectrograms; in this embodiment, two publicly available speech datasets, LJSpeech (single-speaker English, sampling rate of 22.05 kHz) and Aishell-3 (multi-speaker Mandarin, sampling rate of 44.1 kHz), are selected. These datasets are used to generate Mel spectrograms as inputs for the hiding stage and the recovery stage.

[0084] In step S3, the Mel spectrogram of the text-format encrypted speech is obtained through the spectrogram generation network SGN, and the text format conversion is implemented through automatic speech recognition ASR.

[0085] In steps S2 and S4, the Flow-Flow architecture cascades the FlowMap mapper with the forward model of Flowvocoder (i.e., the speech generator) to hide the secret speech; cascades the FlowMap inverse mapper with the inverse model of Flowvocoder (i.e., the speech reverser) to recover the secret speech. Refer to Figure 2 As shown, the main workflow of the Flow-Flow architecture includes:

[0086] (I) Latent variable mapping of the secret speech: A model named FlowMap is designed. This model uses a shared density estimator and flow indication embeddings to map the secret speech to latent codes that follow a standard Gaussian distribution; this mapping is achieved through a series of invertible transformations to ensure that the secret speech can be efficiently encoded and decoded.

[0087] (II) Generation of encrypted speech: The latent variables generated by FlowMap and the Mel spectrogram generated by the carrier text are Figure 1 input into the Flow-based lightweight vocoder FlowVocoder together to generate the encrypted speech (stegospeech); this vocoder uses normal Gaussian distribution sampling to generate waveform signals to ensure that the encrypted speech is statistically indistinguishable from normal speech.

[0088] (III) Secret speech recovery: The goal of this stage is to extract the secret speech from the damaged encrypted speech that may have undergone lossy compression or been interfered with by noise during transmission; the carrier Mel spectrogram is regenerated from the distorted encrypted speech through automatic speech recognition (ASR) and the spectrogram generation network (SGN), then the distorted encrypted speech is remapped to latent variables through the FlowVocoder speech reverser, and finally inversely mapped back to the secret speech through the FlowMap network.

[0089] (4) Asynchronous training strategy: To address the possible distortion that encrypted voice may suffer during transmission (such as compression and noise, including MP3, AAC, or Opus encoding), an asynchronous training strategy is adopted to optimize the Flow-Flow architecture; the hidden network and the recovery network are trained separately for different distortion situations, thus solving the problem of the degradation of the quality of secret voice extraction caused by the distortion of encrypted voice.

[0090] The FlowMap mapper realizes the mapping from the secret voice waveform to the latent variable through a series of invertible flow layers defined by the mixture cumulative distribution function (CDF); the latent variable follows a normal Gaussian distribution with a mean of 0 and a variance of 1, ensuring that the embedded information is statistically indistinguishable from random noise and enhancing security; in FlowMap, there are two mapping relationships:

[0091] (1) Forward mapping (Mapper completed): Used to model the distribution of the waveform samples of the secret voice and map it to the latent variable ;

[0092] (2) Inverse mapping (Inverse mapper completed): Generate the original waveform samples from the latent variable ;

[0093] The FlowMap adopts a flow-based invertible voice mapper, which includes 8 flow modules; as shown in Figure 3 , each flow module includes a shared density estimator, a flow indicator embedding module, a parameter prediction network, and a mixture cumulative distribution transformation; the flow indicator embedding module adopts manifold embedding, which is used to create a learnable lookup table to map the index of each flow layer to a high-dimensional vector, and in the figure is the mapping parameter of the flow indicator embedding module, and the initial mapping parameter is obtained by random initialization; as shown in Figure 4 , the shared density estimator adopts a two-dimensional dilated convolution structure, the convolution kernel size is 3×3, and the dilation factor increases in powers of 2; a gated activation mechanism is introduced after each layer of convolution, and the gradient vanishing is alleviated through residual connection; the gated activation mechanism includes tanh activation and sigmoid gating, and the combination of tanh activation and sigmoid gating enhances the ability to model complex distributions. The gradient vanishing is alleviated through residual connection, and the skip connection fuses local details and global features to improve the naturalness of the generated voice; the parameter prediction network includes a fully connected network, which is used to generate transformation parameters, and the scaling factor , the translation factor , and the transformation of x are output through the parameter prediction network, ; The mixed cumulative distribution transformation includes a weighted sum of 32 distributions, which is used to output latent variables through probability density estimation.

[0094] refer to Figure 5 As shown, the data processing flow of the FlowMap mapper includes:

[0095] performing reverse processing on the secret voice to obtain a reversed secret voice;

[0096] Performing time dimension reversal and width dimension shift on the reversed secret speech to obtain a preprocessed secret speech;

[0097] According to the pre-processed secret speech, the potential features of the secret speech are obtained by the shared density estimator and the flow indication embedding module;

[0098] According to the potential characteristics of the secret speech, a transformation parameter is generated by a parameter prediction network, and the transformation parameter includes a scaling factor , translation factor , mixed weight , mean and standard deviation ;

[0099] According to the reverse secret voice, mixed weight , mean and standard deviation , calculate the hybrid logic CDF and hybrid logic PDF by hybrid cumulative distribution transformation;

[0100] According to the hybrid logical CDF, through probability integration and Logit transformation, the secret speech affine transformation parameters and logarithmic determinant calculation parameters are obtained;

[0101] According to the secret speech affine transformation parameters, scaling factors and translation factors Perform affine transformation to obtain latent variables;

[0102] Calculate parameters, hybrid logic PDFs, and scaling factors based on the logarithmic determinant , calculate the logarithmic determinant.

[0103] Potential features of the secret speech The calculation formula is:

[0104]

[0105] in, represents the shared density estimator, Indicates that the stream indicates the embedded module, Indicates the number of flow layers.

[0106] The calculation formula of the mixed logic CDF is as follows:

[0107]

[0108] Wherein, represents the cumulative distribution function.

[0109] The calculation formula of the mixed logic PDF is as follows:

[0110]

[0111] Wherein, represents the probability density function.

[0112] The calculation formulas of the secret speech affine transformation parameters and the log determinant calculation parameters are as follows:

[0113]

[0114] Wherein, represents the secret speech affine transformation parameter, represents the log determinant calculation parameter.

[0115] The calculation formula of the hidden variable is as follows:

[0116]

[0117] Wherein, represents the hidden variable.

[0118] The calculation formula of the log determinant is as follows:

[0119]

[0120] Wherein, represents the log determinant.

[0121] FlowVocoder is a lightweight vocoder based on normalizing flow, used for generating encrypted voice and voice inversion. The network contains 4 reversible 1×1 convolutional layers and gated tanh non-linear units, with the input channel number set to 512 and the residual connection channel number to 256. FlowVocoder generates waveform signals by sampling from a Gaussian distribution, ensuring that the generated encrypted voice is statistically indistinguishable from normal voice. Using the mel spectrogram of the original carrier voice as a condition, the secret voice is embedded into the encrypted voice through conditional generation. FlowVocoder adopts a lightweight design, with only 4.5 million parameters and a model size of 6.6MB, suitable for deployment on resource-constrained devices. FlowVocoder uses a Gaussian distribution as the target distribution, trains by maximizing the log-likelihood of the data, and optimizes the model parameters using the backpropagation algorithm to ensure that the generated encrypted voice has high fidelity.

[0122] The formula for generating encrypted voice of the FlowVocoder voice generator is as follows:

[0123]

[0124] Where, represents the encrypted voice, represents the encryption operation, represents the latent variable, represents the mel spectrogram of the carrier text to be hidden;

[0125] The formula for restoring the encrypted voice of the FlowVocoder voice inverter is as follows:

[0126]

[0127] Where, represents the restored latent variable, represents the restoration operation, represents the encrypted voice to be restored, represents the mel spectrogram of the encrypted voice in text format.

[0128] To ensure the robustness and efficiency of the Flow-Flow architecture, the Flow-Flow architecture adopts an asynchronous training strategy for model training. The hidden network and the recovery network are trained separately for different distortion situations to solve the problem of the degradation of the secret voice extraction quality caused by the distortion of the encrypted voice, including:

[0129] During the hiding stage, the FlowMap mapper and the FlowVocoder speech generator are trained separately using the maximum likelihood function as the loss function. The latent variables of the FlowMap mapper and the FlowVocoder speech generator are set to the same dimension, and the trained FlowMap mapper and FlowVocoder speech generator are obtained.

[0130] During the recovery stage, the FlowMap inverse mapper and the FlowVocoder speech reverser are initialized based on the pre-trained weights in the hiding stage; the FlowMap inverse mapper and the FlowVocoder speech reverser are fine-tuned using the L1 norm loss function to obtain the trained FlowMap inverse mapper and FlowVocoder speech reverser; the L1 norm loss function is:

[0131]

[0132] where represents the restored latent variable, represents the latent variable, represents the Mel spectrogram conversion function, represents the restored secret speech, represents the secret speech.

[0133] To verify the effectiveness of the proposed robust speech hiding method based on the Flow-Flow architecture in this embodiment, the following experiments and evaluations are carried out, specifically as follows:

[0134] 1. Dataset selection

[0135] The LJSpeech dataset (single female voice, 22.05 kHz) and the Aishell-3 dataset (multiple speakers, 44.1 kHz) are used for experiments; the dataset is divided into a training set (90%) and a test set (10%).

[0136] 2. Evaluation metrics

[0137] The following metrics are used to evaluate the inaudibility of the stego speech and the recovery quality of the secret speech:

[0138] Mean Opinion Score (MOS): Evaluate the subjective quality of the stego speech.

[0139] Mel-cepstral distortion (MCD): Evaluate the objective quality of the stego speech. [[ID=4l]]

[0140] Perceptual Evaluation of Speech Quality (PESQ): Evaluate the recovery quality of the secret speech.

[0141] Short-Term Objective Intelligibility (STOI): Evaluate the intelligibility of the covert speech.

[0142] 3. Experimental Results

[0143] (1) Quality of the encrypted speech

[0144] On the LJSpeech and Aishell-3 datasets, the quality of the encrypted speech of this method is comparable to that of the existing state-of-the-art vocoders, and the recovery quality of the covert speech is significantly better than that of the existing methods, as shown in Table 1; at different compression rates (such as MP3@64 kbps), this method can still recover the covert speech with high fidelity, and the STOI reaches 0.98.

[0145] Table 1: Comparison of the experimental results between this method and the existing methods.

[0146]

[0147] (2) Security verification

[0148] To verify the security of this method, statistical analysis and steganalysis attacks were carried out.

[0149] The Kolmogorov-Smirnov (KS) test was used to verify the statistical similarity between the latent variables generated by FlowMap and random Gaussian samples; the experimental results show that there is no significant difference between the generated latent variables and random Gaussian samples, and the p-value is 0.47.

[0150] Advanced neural steganalysis methods (such as Lin-Net, Chen-Net, and Yang-Net) were used for attack experiments; the experimental results show that the present invention can effectively resist steganalysis attacks and ensure the security of the covert speech.

[0151] (3) Real-time performance and efficiency

[0152] FlowMap contains 2.1 million parameters, FlowVocoder contains 4.5 million parameters, and the total model size is 6.6 MB; compared with the existing methods (such as WaveGlow), the model of this method is significantly reduced.

[0153] When processing 22.05 kHz audio on an NVIDIA 4090 GPU, this method can run at 1.2 times the real-time speed (processing 1 second of audio in 0.83 seconds); this shows that this method is suitable for deployment on resource-constrained devices.

[0154] Through the above detailed steps, this method realizes a speech hiding method with high security, strong robustness and high efficiency, providing an efficient and reliable solution for the fields of covert communication and privacy protection.

[0155] In summary, this embodiment proposes a robust speech hiding method based on the Flow-Flow architecture, which realizes the hiding and extraction of secret speech through the Flow-Flow architecture, including the following steps: designing a FlowMap network to map the secret speech into a latent variable that follows a Gaussian distribution; combining the mel spectrogram generated by the text, and using a lightweight streaming vocoder FlowVocoder to generate the stego speech; adopting an asynchronous training strategy to optimize the hiding and extraction networks to solve the problem of stego speech distortion during transmission; the stego speech of this method reaches the level of normal generated speech in terms of inaudibility and speech quality; it can still maintain a high intelligibility of the extracted secret speech in lossy compression scenarios such as AAC, MP3, and Opus; in addition, the hiding and extraction models have a small model complexity and are suitable for resource-constrained devices.

[0156] Embodiment 2:

[0157] Based on the same inventive concept as Embodiment 1, this embodiment of the present invention discloses a robust speech hiding device based on the Flow-Flow architecture, including:

[0158] A mel spectrogram conversion module for: obtaining the mel spectrogram of the carrier text to be hidden; obtaining the mel spectrogram of the stego speech in text format;

[0159] A speech hiding module for: using a FlowMap mapper to map the secret speech to a latent variable; performing speech hiding through a FlowVocoder speech generator according to the latent variable and the mel spectrogram of the carrier text to be hidden to obtain stego speech;

[0160] A text format conversion module for: performing text format conversion on the stego speech;

[0161] A speech recovery module for: restoring the stego speech and the mel spectrogram of the stego speech in text format to a latent variable through a FlowVocoder speech reverser; using a FlowMap inverse mapper to map the restored latent variable to the recovered secret speech.

[0162] For the specific function implementation of the above modules, refer to the relevant content in the method of Embodiment 1, which will not be elaborated here.

[0163] Embodiment 3:

[0164] This embodiment provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of the robust voice hiding method based on the Flow-Flow architecture described in any one of the first embodiments are implemented.

[0165] Embodiment 4:

[0166] This embodiment provides a computer device, including:

[0167] A memory for storing computer instructions;

[0168] A processor for executing the computer instructions to implement the steps of the robust voice hiding method based on the Flow-Flow architecture described in any one of the first embodiments.

[0169] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0170] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0171] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, thereby providing instructions for implementing the steps specified in one process or a plurality of processes and / or blocks Figure 1 in one block or a plurality of blocks Figure 1 of the functions specified in the flowchart.

[0173] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these are within the protection scope of the present invention.

Claims

1. A robust speech hiding method based on the Flow-Flow architecture, characterized in that, Including: Obtain the Mel spectrogram of the carrier text to be hidden; Input the secret speech and the Mel spectrogram of the carrier text to be hidden into the hiding network of the Flow-Flow architecture: use the FlowMap mapper to map the secret speech to a latent variable; according to the latent variable and the Mel spectrogram of the carrier text to be hidden, perform speech hiding through the FlowVocoder speech generator to obtain the encrypted speech; Perform text format conversion on the encrypted speech to obtain the Mel spectrogram of the text format encrypted speech; Input the encrypted speech and the Mel spectrogram of the encrypted speech in text format into the recovery network of the Flow-Flow architecture: restore the encrypted speech and the Mel spectrogram of the encrypted speech in text format to a latent variable through the FlowVocoder speech reverser; use the FlowMap inverse mapper to map the restored latent variable to the recovered secret speech.

2. The robust speech hiding method based on the Flow-Flow architecture according to claim 1, wherein The Mel spectrogram of the carrier text to be hidden and the Mel spectrogram of the encrypted speech in text format are obtained through the spectral generation network SGN, and the text format conversion is realized through automatic speech recognition ASR.

3. The robust speech hiding method based on the Flow-Flow architecture according to claim 1, characterized in that, The FlowMap adopts a reversible speech mapper based on Flow, including 8 flow modules; The flow module includes a shared density estimator, a flow indicator embedding module, a parameter prediction network, and a mixture cumulative distribution transformation; The flow indicator embedding module adopts manifold embedding, which is used to map the index of each flow layer to a high-dimensional vector through a learnable lookup table; The shared density estimator adopts a two-dimensional dilated convolution structure with a convolution kernel size of 3×3 and the dilation factor increasing as a power of 2; a gated activation mechanism is introduced after each layer of convolution, and gradient vanishing is alleviated through residual connection; the gated activation mechanism includes tanh activation and sigmoid gating; The parameter prediction network includes a fully connected network for generating transformation parameters; The mixture cumulative distribution transformation includes a weighted sum of 32 distributions for outputting a latent variable through probability density estimation.

4. The robust speech hiding method based on the Flow-Flow architecture according to claim 3, characterized in that The data processing flow of the FlowMap mapper includes: Perform reverse order processing on the secret speech to obtain the reverse order secret speech; Perform time dimension inversion and width dimension displacement on the reverse order secret speech to obtain the preprocessed secret speech; According to the preprocessed secret speech, obtain the latent features of the secret speech through the shared density estimator and the flow indicator embedding module; According to the potential features of the secret speech, transformation parameters are generated by a parameter prediction network, and the transformation parameters include a scaling factor , a translation factor , a mixing weight , a mean , and a standard deviation ; According to the reverse secret speech and the mixing weight , the mean value and the standard deviation , calculate the mixed logic CDF and the mixed logic PDF through the mixed cumulative distribution transformation; According to the mixture logic CDF, obtain the affine transformation parameters of the secret speech and the logarithmic determinant calculation parameters through probability integration and Logit transformation; According to the secret voice affine transformation parameters, scaling factor and translation factor perform an affine transformation to obtain a latent variable; According to the logarithmic determinant calculation parameter, the mixed logic PDF, and the scaling factor , the logarithmic determinant is calculated.

5. The robust speech hiding method based on the Flow-Flow architecture according to claim 4, characterized in that The potential features of the secret voice The calculation formula is as follows: Among them, represents a shared density estimator, represents a flow indication embedding module, represents the number of flow layers; The calculation formula of the mixture logic CDF is: Among them, represents the cumulative distribution function; The calculation formula of the mixture logic PDF is: Among them, represents the probability density function; The calculation formulas of the affine transformation parameters of the secret speech and the logarithmic determinant calculation parameters are: Among them, represents the secret voice affine transformation parameter, represents the logarithmic determinant calculation parameter; The calculation formula of the latent variable is: Among them, represents a latent variable; The calculation formula of the logarithmic determinant is: Among them, represents the log determinant.

6. The robust speech hiding method based on the Flow-Flow architecture according to claim 1, characterized in that The FlowVocoder speech generator and speech reverser adopt a lightweight model with an autoregressive flow structure; The formula for generating the encrypted speech by the FlowVocoder speech generator is: Among them, represents encrypted speech, represents an encryption operation, represents a latent variable, represents the Mel spectrogram of the carrier text to be hidden; The formula for restoring the encrypted speech of the FlowVocoder speech reverser is as follows: Among them, represents the restored latent variable, represents the restoration operation, represents the encrypted voice to be restored, represents the Mel spectrogram of the encrypted voice in text format.

7. The robust speech hiding method based on the Flow-Flow architecture according to claim 1, characterized in that The Flow-Flow architecture adopts an asynchronous training strategy for model training, including: In the hidden stage, the FlowMap mapper and the FlowVocoder speech generator are trained separately using the maximum likelihood function as the loss function. The latent variables of the FlowMap mapper and the FlowVocoder speech generator are set to the same dimension, and the trained FlowMap mapper and FlowVocoder speech generator are obtained; In the recovery stage, the FlowMap inverse mapper and the FlowVocoder speech reverser are initialized based on the pre-trained weights in the hidden stage; the FlowMap inverse mapper and the FlowVocoder speech reverser are fine-tuned using the L1 norm loss function to obtain the trained FlowMap inverse mapper and FlowVocoder speech reverser; the L1 norm loss function is: Among them, represents the loss of the FlowMap inverse mapper, represents the loss of the FlowVocoder voice inverter, represents the restored latent variable, represents the latent variable, represents the Mel spectrogram conversion function, represents the restored secret voice, represents the secret voice.

8. A robust speech hiding system based on the Flow-Flow architecture, characterized in that, Applicable to the robust speech hiding method based on the Flow-Flow architecture described in any one of claims 1-7, including: A Mel spectrogram conversion module for: obtaining the Mel spectrogram of the carrier text to be hidden; obtaining the Mel spectrogram of the encrypted speech in text format; A speech hiding module for: mapping the secret speech to a latent variable using the FlowMap mapper; performing speech hiding through the FlowVocoder speech generator according to the latent variable and the Mel spectrogram of the carrier text to be hidden to obtain encrypted speech; A text format conversion module for: converting the text format of the encrypted speech; A speech recovery module for: restoring the Mel spectrograms of the encrypted speech and the encrypted speech in text format to latent variables through the FlowVocoder speech reverser; mapping the restored latent variables to the recovered secret speech using the FlowMap inverse mapper.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the computer instruction is executed by a processor, the steps of the robust speech hiding method based on the Flow-Flow architecture described in any one of claims 1-7 are implemented.

10. A computer device, characterized in that, Including: A memory for storing computer instructions; A processor for executing the computer instructions to implement the steps of the robust speech hiding method based on the Flow-Flow architecture described in any one of claims 1-7.

Images