An extensible bone conduction speech signal transmission method based on diffusion probability model

Abstract

The application discloses an extensible bone conduction speech signal transmission method based on a diffusion probability model, comprising the following steps: acquiring a bone conduction speech signal and a corresponding air conduction speech signal, establishing a training set and a test set based on the bone conduction speech signal and the air conduction speech signal, and constructing an extensible bone conduction speech codec network based on a diffusion probability model. The method can realize speech enhancement while coding, and reduces the complexity of use. The method avoids the problem that a traditional speech codec cannot code speech at a low bit rate. In addition, the extensible idea is introduced, and data packets can be dynamically adjusted according to the change of network load in the transmission process, so as to meet the transmission demand of the network. Even if part of the data packets are lost, high-quality speech signals can be decoded, so that the extensible bone conduction speech signal transmission method is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bone conduction speech signal transmission, and in particular to an extensible bone conduction speech signal transmission method based on a diffusion probability model. BACKGROUND

[0002] Air conduction speech is recorded by converting sound waves propagating in the air into an electrical signal, and bone conduction speech signal is recorded by converting vibrations around the speaker's skull into an electrical signal, which means that in a noisy environment such as public transportation or a noisy workplace, the use of bone conduction devices can deliver clearer and more accurate speech information. However, due to the limitations of its propagation mechanism, bone conduction speech has the following shortcomings: because the human body bone is equivalent to a low-pass filter, the high-frequency components in the bone conduction speech signal are lost, and its energy distribution is usually limited below 2 kHz. This greatly reduces the intelligibility of the speech, making the bone conduction speech sound dull. In the process of speech communication, the sending end needs to encode and compress the bone conduction speech signal into a binary code stream and send it to the receiving end. In order to fully utilize the advantages of bone conduction speech and overcome its limitations, some research works attempt to develop high-frequency compensation techniques for bone conduction speech signals and use existing speech codecs to encode the enhanced bone conduction speech. This makes bone conduction speech more widely applicable.

[0003] Traditional speech codecs are used to encode the enhanced bone conduction speech obtained by the above method for transmission on the communication channel, and then decoded and played by the receiving end. But this process includes two steps: enhancement of bone conduction speech and encoding of enhanced speech, and the two independent steps result in high computational complexity. Traditional speech codecs cannot perform speech coding at a low bit rate. SUMMARY

[0004] In view of the problems existing in the prior art, the present application discloses an extensible bone conduction speech signal transmission method based on a diffusion probability model, which specifically comprises the following steps:

[0005] Collecting the bone conduction speech signal to be sent and the air conduction speech signal corresponding to the bone conduction speech, and establishing a training set and a test set based on the bone conduction speech signal and the air conduction speech signal;

[0006] Constructing a bone conduction speech codec network based on a diffusion probability model, the bone conduction speech codec network comprising a speech encoder, a residual vector quantizer and a speech decoder, the speech encoder comprising a generative model and a diffusion probability model, wherein the generative model comprises a feature extraction module, a high-frequency reconstruction module and an overall optimization module;

[0007] The bone conduction speech codec network is trained on the divided training set and the network parameters are updated, the gradient value of the network is transmitted by using the back propagation algorithm, the network parameters are repeatedly trained and updated, the generated speech signal is controlled to approximate the air conduction speech signal, so as to obtain the best model parameters, and the training process of the bone conduction speech codec is completed.

[0008] In the deployment stage, the speech encoder and the speech decoder are controlled separately, the bone conduction speech signal in the test set is input into the bone conduction speech codec network loaded with the best model parameters to obtain a binary code stream to be sent;

[0009] The binary code stream is transmitted from the sending end to the receiving end, and the speech decoder is used to decode the binary code stream to synthesize the air conduction speech;

[0010] The continuous air conduction speech is played through a loudspeaker.

[0011] Further, the generated model generates a rough enhanced speech signal from a bone conduction speech signal with a sampling rate of 16 kHz, and inputs the speech into a diffusion probability model for enhancement processing to generate an air conduction speech signal with a sampling rate of 48 kHz, wherein the audio signal with a sampling rate of 48 kHz is divided into five sub-bands by a sub-band filter, namely 0-4 kHz, 4-8 kHz, 8-12 kHz, 12-16 kHz and 16-20 kHz, wherein each sub-band is subjected to improved discrete cosine transform as an input speech feature of a residual vector quantizer.

[0012] Further, the residual vector quantizer includes five residual vector quantizer branches with the same structure and different model parameters, and the speech features generated by the speech encoder are input into the five residual vector quantizer branches for quantization processing to obtain five quantization results.

[0013] Further, the feature extraction module includes two types of convolution: 3x3 convolution+ReLU and 1x1 convolution+ReLU, wherein convolution+ReLU represents convolution followed by ReLU activation function, specifically, 3x3 convolution+ReLU is used when the number of layers is odd, and 1x1 convolution+ReLU is used when the number of layers is even, residual learning is used to integrate hierarchical information and retain shallow information, and the process is represented as

[0014] O kf =R((Conv1(Conv2(O full )))+Conv3(Conv4(O key ))) (1)

[0015] Wherein, R represents the ReLU function, Conv i represents convolution operation, Ofull and O key represent global and local key features, respectively, O kf represents the output information of the feature extraction module.

[0016] Further, the high-frequency reconstruction module is based on a generator model of U-Net structure, adopts symmetrical encoder blocks and decoder blocks, and has a skip connection structure, wherein the layout of the encoder is symmetrical to the decoder, the skip connection is added between each encoder block and its symmetrical decoder block, the skip connection of the outermost layer only connects the necessary speech channels of the output, each of the encoder block and the decoder block includes four blocks and is respectively sandwiched between two regular convolution layers, the encoder block includes four down-sampling layers, wherein the down-sampling multiples are 2, 2, 8 and 8 respectively, and the decoder block is up-sampled in reverse order; the number of channels is doubled each time the down-sampling is performed; the number of channels is halved each time the up-sampling is performed, each decoder block is composed of one up-sampling layer and three residual units, each unit includes one-dimensional convolution with expansion rates of 1, 3 and 9, the encoder block and the decoder block corresponding to each other are composed of the same residual units and realize down-sampling through one-dimensional convolution.

[0017] Further, the overall optimization module in the generation model includes 1 layer of ReLU, 4 layers of Conv+ReLU and 1 layer of Conv; each Conv+ReLU layer is composed of a convolution layer and a ReLU activation function, wherein the filter size of the convolution layer is 3x3, the input and output channel numbers are both 64, the filter size of the final convolution layer is 3x3, and the input and output channel numbers are 64 and 1 respectively, and the process is represented as

[0018] O rb =Cv1(R(Cv2(R(Cv3(R(Cv4(R(Cv5(O hf ))))))))) (2)

[0019] wherein R represents the ReLU function, Cv i represents the convolution operation, O hf represents the output of the high-frequency reconstruction module, O rb represents the output of the feature extraction module.

[0020] Further, the diffusion probability model is a generation trend generation model, which generates complex data from simple noise distribution (such as normal distribution) through iterative Monte Carlo Markov chain sampling. The work of the diffusion probability model includes two processes: forward process and backward process. The forward process is a step-by-step generation mechanism based on the probability model, which generates subsequent states one by one from the initial state according to the time step or sequence dependence, which can be denoted as q(y 1:N |y), which represents the generation of y Nthe joint distribution of y , i.e. n n-1 ) represents the likelihood of the current state y n-1 , given the previous state y n . The fixed noise variance is λ 1:N = [λ1, λ2,..., λ N ] is set from 0.001 to 0.02 by linear interpolation, and the process is composed of the multiplication of multiple Gaussian distributions. The forward process can be represented as

[0021]

[0022] where y represents the input signal, I denotes the identity matrix, which acts as a scaling factor in the equation, ensuring that the covariance matrix of the Gaussian distribution is diagonal and each element is equal to λ n . This setup allows the model to control the noise intensity through different scaling factors, providing greater flexibility in adjusting the variability of the generated data.

[0023] The inverse process is represented by the parameter ω, defined as the reverse process of diffusion, where the model starts by sampling y from a Gaussian distribution n , which represents a random noise. The inverse process is defined as the product of transition probabilities, through which the model gradually removes noise to obtain samples close to the true data. The transition probability p ω (y n-1 | y n ) represents the conditional probability of recovering from the current state y n to the previous state y n-1 . This probability distribution is modeled as a multivariate Gaussian distribution.

[0024]

[0025] where μ ω (y n , n) and are the mean and variance of y n-1 estimated by the model, respectively. The latent variable y n-1 is sampled from the distribution p(y n-1 | y n ).

[0026] The inverse process can be represented as

[0027]

[0028] where ∈ ω is the estimated noise by the model,​ wherein φ n defined as 1-λ n , defined as

[0029] Due to the adoption of the above technical solutions, the scalable bone conduction speech signal transmission method based on a diffusion probability model has the following advantages: a deep learning model is adopted, a neural speech codec architecture suitable for bone conduction speech is proposed, speech enhancement can be realized at the same time of speech coding, and the complexity of use is reduced.

[0030] The idea of scalability is introduced, and the data packets can be dynamically adjusted according to the change of network load in the transmission process. The transmitted data packets are divided into five levels: code1, code2, code3, code4 and code5, wherein code1 is the basic layer, and code2, code3, code4 and code5 are enhancement layers. When the network load is low, the data packets can be transmitted completely. When the network load is high, the data packets are discarded in turn from code5 under the premise of ensuring that the basic layer is not lost, so as to meet the transmission demand of the network. Even if part of the data packets are lost, high-quality speech signals can be decoded, so as to realize the scalable bone conduction speech signal transmission method.

[0031] The residual vector quantizer adopts five residual vector quantizers with the same structure but different model parameters. The speech features generated by the encoder are simultaneously sent to the five residual vector quantizers with the same structure but different model parameters for quantization processing to obtain five quantization results.

[0032] The speech decoder part adopts a high-fidelity speech synthesis generative adversarial network vocoder as the decoder architecture of the scalable neural bone conduction speech codec. The high-fidelity speech synthesis generative adversarial network is one of the most advanced neural vocoders at present, which can realize highly realistic speech synthesis.

[0033] The architecture adopts a feature extraction module. The module first extracts the low-frequency features of the speech signal, and gradually aggregates these features through residual learning, which can enhance the network's attention to the low-frequency signals of the speech and improve the memory ability of the shallow layer to the deep layer. In addition, residual connection is also an effective way to promote the network's feature learning and fusion ability, because it can improve the performance without introducing too many additional parameters.

[0034] The architecture adopts a high-frequency reconstruction module. The module first uses a convolutional network to convert the global features and extracted key features into nonlinearity. Then, residual learning is used to integrate global and local features. Next, the fused features are input into the encoder-decoder network to effectively reconstruct the high-frequency components of the speech signal.

[0035] An overall optimization module is adopted in the architecture to further refine the high frequency part of the speech signal reconstructed by the high frequency reconstruction module, so as to improve the speech quality. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0037] Figure 1 The flow chart of the scalable bone conduction speech signal transmission method based on the diffusion probability model in the embodiment of the present application;

[0038] Figure 2 The architecture schematic diagram of the bone conduction speech codec network in the embodiment of the present application;

[0039] Figure 3 The architecture schematic diagram of the bone conduction speech speech encoder in the embodiment of the present application;

[0040] Figure 4 The architecture schematic diagram of the generator model in the embodiment of the present application;

[0041] Figure 5 The architecture schematic diagram of the feature extraction module in the embodiment of the present application;

[0042] Figure 6 The architecture schematic diagram of the high frequency reconstruction module in the embodiment of the present application

[0043] Figure 7 The architecture schematic diagram of the overall optimization module in the embodiment of the present application;

[0044] Figure 8 The architecture schematic diagram of the diffusion probability model in the embodiment of the present application;

[0045] Figure 9 The STOI objective evaluation index of the decoded air conduction speech signal under the low bit rate condition in the embodiment of the present application;

[0046] Figure 10 The STOI objective evaluation index of the decoded air conduction speech signal under the medium bit rate condition in the embodiment of the present application;

[0047] Figure 11 The STOI objective evaluation index of the decoded air conduction speech signal under the high bit rate condition in the embodiment of the present application;

[0048] Figure 12 Subjective evaluation index (MOS) of the decoded air-conducted speech signal in the embodiment of the present application; DETAILED DESCRIPTION

[0049] In order to make the technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application:

[0050] In order to make the technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application:

[0051] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0052] As Figure 1 shown in a scalable bone-conducted speech signal transmission method based on a diffusion probability model,

[0053] S1: Collect the bone-conducted speech signal to be sent and the air-conducted speech signal corresponding to the bone-conducted speech signal, and establish a training set and a test set based on the bone-conducted speech signal and the air-conducted speech signal.

[0054] In specific implementation, the training set and the test set are obtained from the ESMB data set. The data set includes 128 hours of speech spoken by 131 male and 156 female speakers, each speaker reads about 20 minutes of Chinese speech, the speech sampling frequency is 16 kHz stereo sound, the left sound channel is the air-conducted speech signal, and the right sound channel is the bone-conducted speech signal.

[0055] S2: Constructing a bone conduction speech codec network based on a diffusion probability model, including a speech encoder, a residual vector quantizer and a speech decoder, the speech encoder including a generative model and a diffusion probability model. Wherein the generative model includes a feature extraction module, a high frequency reconstruction module and an overall optimization module. Figure 2 The overall structure of the scalable bone conduction speech codec network based on the diffusion probability model is shown.

[0056] The speech encoder network is shown as Figure 3 The speech encoder includes a generative model and a diffusion probability model, which generates a rough enhanced speech signal from a bone conduction speech signal with a sampling frequency of 16 kHz using the generative model. Then, the speech input is further enhanced by the diffusion model to generate an air conduction speech signal with a sampling frequency of 48 kHz. Then, the audio with a sampling frequency of 48 kHz is divided into five subbands, i.e. 0-4 kHz, 4-8 kHz, 8-12 kHz, 12-16 kHz and 16-20 kHz, by a subband filter. Finally, each subband is modified by a discrete cosine transform, which is used as a speech feature for subsequent residual vector quantization.

[0057] The quantizer part includes five residual vector quantizer branches with the same structure but different model parameters, and the speech features generated by the encoder are simultaneously input into the five residual vector quantizers with the same structure but different model parameters for quantization processing to obtain five quantization results.

[0058] The speech decoder architecture adopts a high-fidelity speech synthesis generative adversarial network vocoder as the decoder architecture of the scalable neural bone conduction speech codec. The high-fidelity speech synthesis generative adversarial network vocoder is one of the most advanced neural vocoders, which can realize highly realistic speech synthesis.

[0059] As shown in Figure 4 The generative model includes a feature extraction module, a high frequency reconstruction module and an overall optimization module. The feature extraction module is shown as Figure 5 The feature extraction module involves two types of convolution: 3x3 convolution+ReLU and 1x1 convolution+ReLU, where convolution+ReLU represents convolution followed by ReLU activation function. Specifically, 3x3 convolution+ReLU is used when the number of layers is odd, while 1x1 convolution+ReLU is used when the number of layers is even. This heterogeneous method reduces the overall computational cost and memory consumption of the network. At the same time, in order to preserve the shallow information, residual learning is used to integrate the hierarchical information, which can be represented as

[0060] O kf =R( ( Conv1(Conv2(O full )))+Conv3(Conv4(Okey (11)

[0061] Where R represents the ReLU function, Conv i Represents the convolution operation, O full and O key O represents global and local key features, respectively. kf This represents the output of the feature extraction module.

[0062] like Figure 6 As shown, the high-frequency reconstruction module is based on a generator model with a U-Net structure, employing a symmetrical encoder-decoder network with skip connections. The encoder layout is symmetrical to the decoder, and skip connections are added between each encoder block and its symmetrical decoder block. The outermost skip connections connect only the necessary speech channels of the output. Both the encoder and decoder contain four blocks, sandwiched between two regular convolutional layers. The encoder has four downsampling layers with downsampling factors of 2, 2, 8, and 8, while the decoder performs upsampling in the reverse order. The number of channels doubles with each downsampling and halves with each upsampling. Each decoder block consists of one upsampling layer and three residual units, each containing a one-dimensional convolution with dilation rates of 1, 3, and 9. The encoder blocks correspond to the decoder blocks, consisting of the same residual units, and downsampling is achieved through one-dimensional convolution.

[0063] like Figure 7 As shown, the overall optimization module contains 1 ReLU layer, 4 Conv+ReLU layers, and 1 Conv layer. Each Conv+ReLU layer consists of a convolutional layer and a ReLU activation function, where the filter size of the convolutional layer is 3×3, and both the input and output channels are 64. The final convolutional layer has a filter size of 3×3, with 64 input channels and 1 output channel. This process can be represented as follows:

[0064] O rb =Cv1(R(Cv2(R(Cv3(R(Cv4(R(Cv5(O hf (12)

[0065] Where R represents the ReLU function, Cv i Represents the convolution operation, O hf O represents the output of the high-frequency reconstruction module. rb This represents the output of the feature extraction module.

[0066] like Figure 8As shown, the diffusion probabilistic model is a generative model that generates trends by iteratively sampling from a simple noise distribution (e.g., a normal distribution) to generate complex data. The working of the diffusion probabilistic model involves two processes: a forward process and a reverse process. The forward process is a step-by-step generation mechanism based on the probabilistic model that generates subsequent states one after the other starting from an initial state according to temporal steps or sequence dependencies, which can be denoted as q(y 1:N |y N ), represents the joint distribution from y1to y given the condition y n , i.e. n-1 n-1 n where the fixed noise variance is λ 1:N = [λ1, λ2,..., λ N ] is set from 0.001 to 0.02 by linear interpolation, and the process is composed of the multiplication of multiple Gaussian distributions. The forward process can be represented as

[0067]

[0068] where y represents the input signal, and I denotes the identity matrix that serves as a scaling factor in the formula to ensure that the covariance matrix of the Gaussian distribution is a diagonal matrix with each element equal to λ n . This setting allows the model to control the noise intensity through different scaling factors, providing greater flexibility in adjusting the variability of the generated data.

[0069] The reverse process is represented by the parameter ω, which defines the inverse process of diffusion. The model starts by sampling y n from a Gaussian distribution , which represents a random noise. The reverse process is defined as the product of transition probabilities, through which the model gradually removes noise to obtain samples close to the true data. The transition probability p ω (y n-1 |y n ) represents the conditional probability of recovering from the current state y n to the previous state y n-1 . This probability distribution is modeled as a multivariate Gaussian distribution.

[0070]

[0071] where μ ω (y n , n) and σ n 2 are the model estimates of y​​n-1 The mean and variance of y. Latent variable y n-1 From distribution p(y) n-1 |y n Sampling in )

[0072] The reverse process can be represented as

[0073]

[0074] Where, ∈ ω It is noise in the model estimation. Where φ n Defined as 1-λ n , Defined as

[0075] The model proposed in this invention operates directly in the time domain, eliminating the need to extract any speech features. The proposed diffusion model uses the diffuse signal... downsampled signal y s and noise level As input, and output the estimated diffusion noise Ten residual layers and 64 channels were used, and a U-Net architecture was introduced after bidirectional dilated convolution in each residual layer. The bidirectional dilated convolution used a kernel size of 3 and a dilation period of [1,2,...,512]×3. The U-Net architecture achieved good results in tasks such as speech super-resolution and speech enhancement. However, to our knowledge, U-Net has not yet been integrated into the field of diffusion probability models. Therefore, to further improve the capabilities of diffusion probability models in speech super-resolution, a novel diffusion model architecture combining both approaches is proposed.

[0076] S3: Construct the loss function for the network, train the scalable bone conduction speech codec network based on the diffusion probability model constructed in S2 on the pre-defined training set, and update the parameters of the proposed network using the loss function. Specifically, the parameter update involves using the backpropagation algorithm to propagate gradients, iterating repeatedly to reduce errors, making the prediction depth approximate the air conduction speech signal, and ultimately obtaining the optimal model parameters.

[0077] In this embodiment, a generative adversarial network training mode is adopted, and two different discriminators are used: a multi-resolution convolutional discriminator that takes a single waveform as input; and an STFT-based discriminator that takes the complex STFT of the input waveform as input.

[0078] The multi-resolution convolutional discriminator uses three discriminators with the same structure applied to audio inputs at different resolutions: original, 2x down-sampled, and 4x down-sampled. Each discriminator consists of one initial normal convolution followed by four grouped convolutions, each group of size 4, down-sampling factor of 4, and channel multiplier of 4, up to 1024 output channels.

[0079] The STFT-based discriminator involves one 2D convolutional layer followed by a series of 2D convolutions, the first with kernel size 7x7 and 32 channels. This is followed by a series of residual blocks. Each block starts with a 3x3 convolution followed by a 3x4 or 4x4 convolution with strides (1,2) or (2,2), where (s t ,s f ) denotes the down-sampling factors along the time and frequency axes. The down-sampling alternates between (1,2) and (2,2) strides, with a total of 6 residual blocks. The number of channels increases gradually with the depth of the network. At the output of the final residual block, the activation shape is T / (H·2 3 )xF / 2 6 , where T is the number of samples in the time domain and F=W / 2 is the number of frequency bins. The last layer employs a fully connected layer (implemented as a 1x F / 2 6 convolution) to pool the frequency bins to obtain a one-dimensional signal in the time domain.

[0080] Let G(x) denote the decoded speech signal, the adversarial loss function for the discriminator is

[0081]

[0082] The loss function for the generator is

[0083]

[0084] The weight values λ adv , λ feat , λ rec are set to 1, 100, and 1, respectively.

[0085]

[0086] where k ∈ {0,...,k} indexes the individual discriminators at different resolutions, with k=0 corresponding to the STFT-based discriminator and k ∈ {1,...,k} denoting the multi-resolution convolutional discriminators (in this embodiment, k=3), T k denotes the output along the time dimension of the kth discriminator.

[0087]

[0088] where L is the number of internal layers, is the output of the discriminator k at the l-th layer at time t, and T k,l denotes the time dimension of the layer.

[0089]

[0090] where S t s (x) represents a mel-spectrogram with a window length of s, a hop length of s / 4, and 64 mel-spectral filter coefficients on the t-th frame,

[0091] S4: In the deployment stage, load the optimal parameter model obtained in the S3 step, and use the scalable bone conduction speech codec network based on the diffusion probability model constructed in S2.

[0092] In the implementation process, the speech encoder and the speech decoder are used separately, the bone conduction speech signal in the ESMB test set is input into the bone conduction speech encoder loaded with the optimal parameters, and the binary code stream to be sent is obtained;

[0093] S5: Transmit the binary code stream via the sending end to the receiving end through the data communication module;

[0094] S6: Decode the binary code stream to synthesize air conduction speech using the speech decoder;

[0095] S7: Play the restored continuous speech signal through the speaker through the speech playing module.

[0096] For the specific implementation of those skilled in the art, the hardware platform used by the application is NVIDIA GeForce RTX 4090 GPU, and the software platform is PyTorch deep learning framework. In the training process, the learning rate is 0.0002, and the adaptive learning rate optimization algorithm with weight decay is used to optimize the network parameters, wherein the decay coefficient of the first order momentum estimation is 0.9, the decay coefficient of the second order momentum estimation is 0.9, and the numerical stability term is 10 -6 .

[0097] In order to verify the effectiveness of the application, a large number of bone conduction speech is tested under three bit rate conditions (low bit rate 3kbps, medium bit rate 6kbps, high bit rate 12kbps) using objective evaluation indicators. In addition, a common bone conduction speech coding method is selected, that is, the bone conduction speech is first enhanced, and then coded and decoded, and the currently popular traditional speech codec and neural speech codec are selected for comparison. Among them, the Scalable Neural BCM Codec is the method of the application, Figures 9 to 11 objective test results,Figure 12 The subjective experimental results are shown in Table 2 and Table 3, respectively. Both of the two experimental results show that the proposed method achieves good performance in the process of bone conduction speech coding at different rates. The method invented in this paper can realize speech enhancement while coding, which reduces the complexity of use. It avoids the problem that the traditional speech codec cannot code speech at a lower bit rate. In addition, the idea of scalability is introduced. The data packets can be dynamically adjusted according to the change of network load during transmission to meet the transmission demand of the network. Even if some data packets are lost, high-quality speech signals can be decoded, so as to realize the scalable bone conduction speech signal transmission method.

[0098] The scalable bone conduction speech signal transmission method based on the diffusion probability model can dynamically adjust the data packets according to the change of network load during transmission. The transmitted data packets are divided into five levels: code1, code2, code3, code4 and code5, wherein code1 is the basic layer, and code2, code3, code4 and code5 are the enhancement layers. When the network load is low, this method can transmit the data packets completely. On the contrary, when the network load is high, the data packets can be discarded in turn from code5 under the premise of ensuring that the basic layer is not lost, so as to meet the transmission demand of the network. Even if some data packets are lost, high-quality speech signals can be decoded, so as to realize the scalable bone conduction speech signal transmission method.

[0099] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A scalable bone conduction speech signal transmission method based on a diffusion probability model, characterized in that... include: Collect the bone conduction speech signal to be transmitted and the corresponding air conduction speech signal, and establish a training set and a test set based on the bone conduction speech signal and the air conduction speech signal; A bone conduction speech codec network based on a diffusion probability model is constructed. The bone conduction speech codec network includes a speech encoder, a residual vector quantizer, and a speech decoder. The speech encoder includes a generation model and a diffusion probability model. The generation model includes a feature extraction module, a high-frequency reconstruction module, and an overall optimization module. The bone conduction speech codec network is trained on the divided training set and the network parameters are updated. The gradient values ​​of the network are transmitted using the backpropagation algorithm. The network parameters are repeatedly trained and updated to control the generated speech signal to approximate the air conduction speech signal, thereby obtaining the optimal model parameters and completing the training process of the bone conduction speech codec. In the deployment phase, the speech encoder and speech decoder are controlled separately, and the bone conduction speech signals from the test set are input into the bone conduction speech codec network loaded with the optimal model parameters to obtain the binary code stream to be sent. The binary code stream is transmitted from the sending end to the receiving end, and the binary code stream is decoded and synthesized into air-conducted speech using a speech decoder; Continuous air-conducted speech is played through a speaker.

2. The scalable bone conduction speech signal transmission method based on a diffusion probability model according to claim 1, characterized in that: The generation model generates a coarsely enhanced speech signal from a bone conduction speech signal with a sampling rate of 16 kHz. The speech input diffusion probability model is then enhanced to generate an air conduction speech signal with a sampling rate of 48 kHz. The 48 kHz audio signal is divided into five sub-bands by a sub-band filter: 0–4 kHz, 4–8 kHz, 8–12 kHz, 12–16 kHz, and 16–20 kHz. Each sub-band undergoes an improved discrete cosine transform and serves as the input speech feature for the residual vector quantizer.

3. The scalable bone conduction speech signal transmission method based on a diffusion probability model according to claim 2, characterized in that: The residual vector quantizer includes five branches with identical structures but different model parameters. The speech features generated by the speech encoder are input into the five residual vector quantizer branches for quantization processing to obtain five quantization results.

4. The scalable bone conduction speech signal transmission method based on a diffusion probability model according to claim 1, characterized in that: The feature extraction module includes two types of convolutions: 3×3 convolution + ReLU and 1×1 convolution + ReLU. Convolution + ReLU means that the convolution is followed by the ReLU activation function. Specifically, 3×3 convolution + ReLU is used when the number of layers is odd, while 1×1 convolution + ReLU is used when the number of layers is even. Residual learning is used to integrate hierarchical information and retain shallow information. This process is represented as follows: THE kf =R((Conv1(Conv2(O full )))+Conv3(Conv4(O key ))) (1) Where R represents the ReLU function, Conv i Represents the convolution operation, O full and O key O represents global and local key features respectively. kf This indicates the output information of the feature extraction module.

5. A scalable bone conduction speech signal transmission method based on a diffusion probability model according to claim 1, characterized in that: The high-frequency reconstruction module is based on a generator model with a U-Net structure. It uses symmetrical encoder blocks and decoder blocks and has a skip connection structure. The encoder layout is symmetrical with the decoder. Skip connections are added between each encoder block and its symmetrical decoder block. The outermost skip connection only connects the necessary speech channels of the output. Each encoder block and decoder block contains four blocks, which are sandwiched between two regular convolutional layers. The encoder block includes four downsampling layers with downsampling factors of 2, 2, 8 and 8, respectively, while the decoder block performs upsampling in the reverse order. Each downsampling operation doubles the number of channels; each upsampling operation halves the number of channels. Each decoder block consists of an upsampling layer and three residual units. Each unit contains a one-dimensional convolution with dilation rates of 1, 3, and 9. The encoder block corresponds to the decoder block and consists of the same residual units. Downsampling is achieved through one-dimensional convolution.

6. A scalable bone conduction speech signal transmission method based on a diffusion probability model according to claim 1, characterized in that: The overall optimization module in the generative model includes one ReLU layer, four Conv+ReLU layers, and one Conv layer. Each Conv+ReLU layer consists of a convolutional layer and a ReLU activation function. The filter size of the convolutional layer is 3×3, with 64 input and 64 output channels. The final convolutional layer has a filter size of 3×3, with 64 input and 1 output channels. This process is represented as follows: O rb =Cv1(R(Cv2(R(Cv3(R(Cv4(R(Cv5(O hf ))))))))) (2) Where R represents the ReLU function, Cv i Represents the convolution operation, O hf O represents the output of the high-frequency reconstruction module. rb This represents the output of the feature extraction module.

Images