Voice signal processing method, neural network training method and equipment

Abstract

The invention discloses a voice signal processing method, a neural network training method and equipment, and is applied to the field of signal processing, and the method comprises the steps: obtaining a first bone conduction voice signal, extracting an excitation parameter from the first bone conduction voice signal, and determining a first transfer function corresponding to the excitation parameter; afterwards, the first transfer function is input into the trained neural network to obtain a second transfer function (namely, the transfer function of the predicted gas conduction voice signal), the trained neural network is obtained by training the neural network by using the training data set based on a target loss function, and finally, according to the second transfer function and the excitation parameter, the gas conduction voice signal is obtained. And obtaining a first air conduction voice signal corresponding to the first bone conduction voice signal. According to the application, based on the trained neural network, the transfer function of the bone conduction voice signal is mapped into the transfer function of the air conduction voice signal, so that tone compensation from the bone conduction voice signal to the air conduction voice signal is realized, the sound listening effect of the bone conduction voice signal is brighter and more comfortable, and the call efficiency is improved.

Description

A method for processing speech signals, a method for training neural networks and a device Technical Field

[0001] The present application relates to the field of signal processing, and in particular to a method for processing speech signals, a method for training a neural network, and a device. Background Art

[0002] In a real-time call scenario, voice signals are transmitted in most cases using air-conduction voice signals. The transmission process can be shown in Figure 1, where user A and user B make a voice call based on their mobile phones. Taking user A speaking and user B listening as an example, the transmission process is as follows: ① User A speaks and sends a voice signal S; ② The conduction microphone on the phone on side A collects the voice signal S; ③ The phone on side A encodes and compresses the voice signal S and transmits it to the phone on side B based on a wireless network; ④ The phone on side B decodes the received signal and restores it to obtain the voice signal S; ⑤ The phone on side B plays the voice signal S based on the air-conduction speaker on the phone on side B.

[0003] When the above-mentioned air-conducted voice signal is transmitted, the voice signal S will collect environmental noise in addition to the voice of the user on side A. If the environmental noise is loud, it will affect the listening experience of the user on side B. Therefore, in some high-noise scenarios, such as mining and rescue operations, bone conduction voice signals are generally used for information transmission. This is because bone conduction microphones collect less ambient noise and have a high audio signal-to-noise ratio, which can greatly ensure call efficiency in these scenarios. The transmission process is shown in Figure 2, where users on side A and side B are making a voice call based on their mobile phones. Taking user A speaking and user B (assuming that side B does not change) listening as an example, the above transmission process is as follows: ① User on side A speaks, emitting a voice signal S1; ② The bone conduction microphone on the head-mounted device on side A (such as headphones) is placed in close contact with the face of user on side A and collects audio signal S2 (also called bone conduction voice signal S2) when user A speaks based on skin vibration. This bone conduction voice signal S2 is sent to the mobile phone on side A (e.g., via a Bluetooth connection between the head-mounted device and the mobile phone); ③ Mobile phone on side A encodes and compresses the bone conduction voice signal S2 and transmits it to mobile phone on side B via a wireless network; ④ Mobile phone on side B decodes the received signal and restores the bone conduction voice signal S2; ⑤ Mobile phone on side B plays the bone conduction voice signal S2 through its air conduction speaker.

[0004] The generation models for both air-conduction and bone-conduction speech signals use a convolutional excitation source with an excitation channel (i.e., excitation source * excitation channel = speech signal). However, the specific generation principles differ: The excitation source for both is the vocal cords, but the excitation channel for air-conduction speech is the pharynx and oral cavity, while the excitation channel for bone-conduction speech is the muscle and bone. These differences in the excitation channels lead to significant differences in the final listening experience: Because the excitation channel for air-conduction speech is airborne, air-conduction speech exhibits minimal distortion and high brightness. However, because the excitation channel for bone-conduction speech is solid and soft, it exhibits significant distortion and a darker brightness than air-conduction speech, making it less comfortable to listen to. More seriously, this can cause the other party (e.g., party B) to hear the voice on party A sound unnatural, and in severe cases, may be unable to distinguish the speaker, thus affecting call efficiency.

[0005] Summary of the Invention

[0006] Embodiments of the present application provide a method for processing voice signals, a method for training a neural network, and a device for mapping the transfer function of a bone conduction voice signal (i.e., a first transfer function) to the transfer function of an air conduction voice signal (i.e., a second transfer function) based on a trained neural network, thereby achieving timbre compensation from the bone conduction voice signal to the air conduction voice signal. This makes the listening effect of the bone conduction voice signal brighter and more comfortable, and can ensure call efficiency in high-noise scenarios such as mining and rescue operations.

[0007] Based on this, the embodiments of the present application provide the following technical solutions:

[0008] In a first aspect, embodiments of the present application provide a method for processing speech signals, which can be used in the field of signal processing. The method comprises: first, obtaining a bone conduction speech signal to be processed, which may be referred to as a first bone conduction speech signal. Next, extracting excitation parameters based on the first bone conduction speech signal. Next, determining a transfer function (also referred to as a first transfer function) of the first bone conduction speech signal based on the first bone conduction speech signal. After obtaining the first transfer function of the first bone conduction speech signal, the first transfer function is input into a trained neural network to output a second transfer function. The second transfer function is the predicted transfer function of the air conduction speech signal, and therefore may also be referred to as a predicted transfer function. Finally, based on the second transfer function output by the neural network and the predetermined excitation parameters, a first air conduction speech signal corresponding to the first bone conduction speech signal is obtained.

[0009] In the above-mentioned embodiment of the present application, based on the trained neural network, the transfer function of the bone conduction voice signal (i.e., the first transfer function) is mapped to the transfer function of the air conduction voice signal (i.e., the second transfer function) to achieve timbre compensation from the bone conduction voice signal to the air conduction voice signal, so that the listening effect of the bone conduction voice signal is brighter and more comfortable, and closer to the auditory perception of the air conduction voice signal.

[0010] In one possible implementation of the first aspect, the trained neural network is obtained by training the neural network using a training dataset based on a target loss function, wherein the training dataset includes multiple training data, the training data including a first true transfer function of a bone conduction speech signal, the first true transfer function being obtained based on an audio signal emitted by a sound source captured by a bone conduction microphone. The output of the neural network is a predicted transfer function, the predicted transfer function corresponding to a second true transfer function of an air conduction speech signal, the second true transfer function being obtained based on the audio signal emitted by the sound source captured by the air conduction microphone.

[0011] In the above-mentioned embodiments of the present application, it is specifically described how the trained neural network is obtained through training. Through the network method, the mapping quality can be improved by increasing the scale of the mapping network and having flexibility.

[0012] In a possible implementation manner of the first aspect, the target loss function may be: an error value between the predicted transfer function and the second true transfer function.

[0013] In the above embodiment of the present application, a method for implementing the target loss function is specifically described, which is simple and easy to operate.

[0014] In a possible implementation of the first aspect, a method for determining the first transfer function of the first bone conduction voice signal based on the first bone conduction voice signal may be: using an excitation parameter, performing a deconvolution operation on the first bone conduction voice signal to obtain the first transfer function of the first bone conduction voice signal.

[0015] In the above-mentioned embodiment of the present application, a method for obtaining the first transfer function is specifically described, which is feasible.

[0016] In a possible implementation of the first aspect, the method for obtaining the first air-conduction voice signal corresponding to the first bone-conduction voice signal based on the second transfer function and the excitation parameter can be: performing a convolution operation on the second transfer function and the excitation parameter to obtain the first air-conduction voice signal corresponding to the first bone-conduction voice signal.

[0017] In the above-mentioned embodiment of the present application, a method for obtaining the first air-conducted speech signal is specifically described, which is feasible.

[0018] In a possible implementation of the first aspect, the first bone conduction voice signal may be obtained by first collecting an audio signal through a bone conduction microphone, and then performing noise reduction on the collected audio signal. For example, noise reduction may be performed using a spectral subtraction noise reduction algorithm to obtain the first bone conduction voice signal.

[0019] In the above embodiment of the present application, the first bone conduction voice signal is subjected to noise reduction processing, so the signal spectrum of the voice signal will be purer.

[0020] In a possible implementation manner of the first aspect, the excitation parameters include a fundamental frequency and harmonics of the fundamental frequency of the first bone conduction voice signal.

[0021] In the above-mentioned embodiments of the present application, the components included in the extracted excitation parameters are specifically described and are feasible.

[0022] A second aspect of an embodiment of the present application provides a neural network training method, comprising: first, collecting audio signals emitted by a sound source (e.g., user A) using a bone conduction microphone and an air conduction microphone over a certain period of time. For example, n (n ≥ 2) bone conduction voice signals and n air conduction voice signals emitted by user A when speaking can be collected. Subsequently, a true transfer function (which may be referred to as a first true transfer function) corresponding to each bone conduction voice signal can be further obtained based on each bone conduction voice signal. For n bone conduction voice signals, a total of n first true transfer functions can be obtained. Similarly, a true transfer function (which may be referred to as a second true transfer function) corresponding to each air conduction voice signal can also be further obtained based on each air conduction voice signal. For n air conduction voice signals, a total of n second true transfer functions can be obtained. The n first true transfer functions are used as a training data set for the neural network, and the neural network is trained using a target loss function until a training termination condition is reached, thereby obtaining a trained neural network. The target loss function is obtained based on the second true transfer function.

[0023] In the above-mentioned embodiments of the present application, it is specifically described how the trained neural network is obtained through training. Through the network method, the mapping quality can be improved by increasing the scale of the mapping network and having flexibility.

[0024] In a possible implementation of the second aspect, the bone conduction microphone and the air conduction microphone are deployed on the same device, and the neural network training process is performed on the device.

[0025] In the above-mentioned embodiment of the present application, the training processes of the bone conduction microphone, the air conduction microphone and the neural network are all performed on the same device, which can realize the function of online training and has flexibility.

[0026] In a possible implementation manner of the second aspect, the same device includes: a head-mounted device, such as headphones.

[0027] In the above-mentioned embodiments of the present application, one form of the same device is specifically described, which is feasible.

[0028] In a possible implementation manner of the second aspect, the target loss function may be: an error value between the predicted transfer function and the second true transfer function.

[0029] In the above embodiment of the present application, a method for implementing the target loss function is specifically described, which is simple and easy to operate.

[0030] In a possible implementation of the second aspect, the conditions for training termination include but are not limited to: the value of the target loss function reaches a preset threshold; or, the target loss function begins to converge; or, the number of training times reaches a preset number of times; or, the duration of training reaches a preset duration; or, an instruction to terminate training is obtained.

[0031] In the above-mentioned embodiments of the present application, several implementation methods for reaching the training termination conditions are specifically described, which can be selected for use based on actual application scenarios and have wide applicability.

[0032] A third aspect of the present application provides an execution device that implements the method of the first aspect or any possible implementation of the first aspect. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0033] A fourth aspect of the present application provides a training device that implements the method of the second aspect or any possible implementation of the second aspect. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functionality.

[0034] The fifth aspect of the embodiment of the present application provides an execution device, which may include a memory, a processor and a bus system, wherein the memory is used to store programs, and the processor is used to call the programs stored in the memory to execute the method of the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0035] The sixth aspect of an embodiment of the present application provides a training device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to call the programs stored in the memory to execute the method of the second aspect of an embodiment of the present application or any possible implementation method of the second aspect.

[0036] In a seventh aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer can execute the method of the above-mentioned first aspect or any possible implementation of the first aspect, or the computer can execute the method of the above-mentioned second aspect or any possible implementation of the second aspect.

[0037] An eighth aspect of an embodiment of the present application provides a computer program, which, when running on a computer, enables the computer to execute the method of the above-mentioned first aspect or any possible implementation of the first aspect, or enables the computer to execute the method of the above-mentioned second aspect or any possible implementation of the second aspect.

[0038] A ninth aspect of the embodiment of the present application provides a chip, which includes at least one processor and at least one interface circuit, the interface circuit is coupled to the processor, the at least one interface circuit is used to perform transceiver functions, and send instructions to at least one processor, the at least one processor is used to run a computer program or instruction, which has the function of implementing the method of the first aspect or any possible implementation of the first aspect, or, it has the function of implementing the method of the second aspect or any possible implementation of the second aspect, the function can be implemented by hardware, or by software, or by a combination of hardware and software, the hardware or software includes one or more modules corresponding to the above functions. In addition, the interface circuit is used to communicate with other modules outside the chip. For example, the interface circuit can send the neural network trained on the chip to a target device (such as headphones, mobile phones, personal computers, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic diagram of a mobile phone-based real-time communication system provided in an embodiment of the present application;

[0040] FIG2 is another schematic diagram of a mobile phone-based real-time communication system according to an embodiment of the present application;

[0041] FIG3 is a schematic diagram of a system architecture of a speech signal processing system provided in an embodiment of the present application;

[0042] FIG4 is a flow chart of a neural network training method according to an embodiment of the present application;

[0043] FIG5 is a diagram illustrating an example of a training process according to an embodiment of the present application;

[0044] FIG6 is a flow chart of a method for processing a speech signal according to an embodiment of the present application;

[0045] FIG7 is a schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0046] FIG8 is a schematic diagram of the structure of a training device provided in an embodiment of the present application;

[0047] FIG9 is a schematic structural diagram of another execution device provided in an embodiment of the present application;

[0048] FIG10 is a schematic structural diagram of another training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] Embodiments of the present application provide a method for processing voice signals, a method for training a neural network, and a device for mapping the transfer function of a bone conduction voice signal (i.e., a first transfer function) to the transfer function of an air conduction voice signal (i.e., a second transfer function) based on a trained neural network, thereby achieving timbre compensation from the bone conduction voice signal to the air conduction voice signal. This makes the listening effect of the bone conduction voice signal brighter and more comfortable, and can ensure call efficiency in high-noise scenarios such as mining and rescue operations.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] To better understand the solutions of the embodiments of the present application, the following first introduces the relevant terms and concepts of neural networks that may be involved in the embodiments of the present application. It should be understood that the explanation of the relevant concepts may be limited by the specific circumstances of the embodiments of the present application, but it does not mean that the present application is limited to only such specific circumstances. The specific circumstances of different embodiments may also vary, and are not specifically limited here.

[0052] (1) Neural Network

[0053] A neural network can be composed of neural units. Specifically, it can be understood as a neural network with an input layer, a hidden layer, and an output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. Among them, a neural network with many hidden layers is called a deep neural network (DNN). The work of each layer in the neural network can be expressed mathematically as To describe, from a physical level, the work of each layer in the neural network can be understood as completing the transformation from input space to output space (i.e., from the row space to the column space of the matrix) through five operations on the input space (a set of input vectors). These five operations include: 1. Dimensionality increase / decrease; 2. Zoom in / out; 3. Rotation; 4. Translation; 5. "Bending". Operations 1, 2, and 3 are represented by Completed, operation 4 is completed by "+b", and operation 5 is implemented by "a()". The word "space" is used here because the object being classified is not a single thing, but a class of things. Space refers to the collection of all individuals of this type of thing, where W is the weight matrix of each layer of the neural network, and each value in the matrix represents the weight value of a neuron in the layer. The matrix W determines the spatial transformation from the input space to the output space described above, that is, the W of each layer of the neural network controls how to transform the space. The purpose of training a neural network is to eventually obtain the weight matrices of all layers of the trained neural network. Therefore, the training process of a neural network is essentially about learning how to control spatial transformation, and more specifically, about learning the weight matrix.

[0054] (2) Loss Function

[0055] In the process of training a neural network, because we want the output of the neural network to be as close as possible to the value we actually want to predict, we can compare the current network's predicted value with the desired target value, and then update the weight matrix of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the neural network). For example, if the network's predicted value is too high, the weight matrix is ​​adjusted to make it predict a lower value, and this adjustment is continued until the neural network can predict the desired target value. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value." This is the loss function or objective function, which is an important equation used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of the neural network becomes a process of minimizing this loss as much as possible.

[0056] (3) Backpropagation algorithm

[0057] During neural network training, the back propagation (BP) algorithm can be used to correct the size of the parameters in the initial neural network model, reducing the reconstruction error loss of the neural network model. Specifically, forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial neural network model, thereby converging the error loss. The BP algorithm is a backpropagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0058] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0059] To facilitate understanding of this solution, the system architecture of the speech signal processing system provided by the embodiment of the present application is first introduced in conjunction with Figure 3. Please refer to Figure 3, which is a schematic diagram of the system architecture of the speech signal processing system provided by the embodiment of the present application. In Figure 3, the speech signal processing system 200 includes an execution device 210, a training device 220, a database 230, a client device 240, a data storage system 250 and a data acquisition device 260 (such as a bone conduction microphone and an air conduction microphone). The execution device 210 includes a computing module 211. Among them, the data acquisition device 260 is used to obtain a large-scale data set required by the user (i.e., a training data set, also referred to as a training set, which includes training data) and store the training set in the database 230. The training device 220 trains the neural network 201 constructed in this application based on the training set maintained in the database 230. The trained neural network 201 obtained by training is then used on the execution device 210. The execution device 210 can call data, code, etc. in the data storage system 250, or store data, instructions, etc. in the data storage system 250. The data storage system 250 may be placed in the execution device 210 , or the data storage system 250 may be an external memory relative to the execution device 210 .

[0060] The trained neural network 201 obtained through training device 220 can be applied to different systems or devices (i.e., execution device 210), specifically, end-side devices or head-mounted devices, such as headphones, mobile phones, tablets, computers, cloud servers, etc. In Figure 6, execution device 210 can be configured with an I / O interface 212 for data exchange with external devices. A "user" can input data into I / O interface 212 through client device 240. For example, client device 240 can be a bone conduction microphone. The bone conduction voice signal collected by the client device is input to execution device 210. Execution device 210 first calculates the true transfer function of the bone conduction voice signal (i.e., the first transfer function described below). This true transfer function is then input to calculation module 211. Calculation module 211 processes the input true transfer function to obtain a predicted transfer function (i.e., the second transfer function described below). This predicted transfer function is then stored on the storage medium of execution device 210 for use in subsequent downstream tasks.

[0061] Furthermore, in some embodiments of the present application, the client device 240 may also be integrated into the execution device 210. For example, when the execution device 210 is a head-mounted device (e.g., headphones), the bone conduction voice signal may be directly acquired through the head-mounted device (e.g., the bone conduction voice signal may be acquired by a bone conduction microphone on the head-mounted device, or the bone conduction voice signal may be received by the head-mounted device from another device, etc., with no limitation on the source of the bone conduction voice signal). The computing module 211 within the head-mounted device then processes the actual transfer function of the bone conduction voice signal to obtain a predicted transfer function, which is then stored. The product form of the execution device 210 and the client device 240 is not limited herein.

[0062] It should also be noted that in some other embodiments of the present application, the data acquisition device 260 and / or the training device 220 can also be integrated into the execution device 210. For example, when the execution device 210 is a head-mounted device (such as headphones), since the timbre and other aspects of the bone conduction voice signal collected when different people wear the head-mounted device may also be different, the data acquisition device 260 and / or the training device 220 can be integrated into the execution device 210. When user A wears the head-mounted device, the data acquisition device 260 (such as a bone conduction microphone) collects the speech of user A, and the training device 220 trains the neural network 201 (the transfer function of the real air conduction voice signal is obtained based on the air conduction voice signal collected by the air conduction microphone). The trained neural network 201 is directly used for subsequent applications of user A; similarly, when user B wears the head-mounted device, the data acquisition device 260 (such as a bone conduction microphone) collects the speech of user B, and the training device 220 trains the neural network 201. The trained neural network 201 is directly used for subsequent applications of user A. This can make the trained neural network 201 more accurate and adaptable when different users use the execution device 210, thus providing flexibility.

[0063] It is worth noting that FIG3 is merely a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in FIG3 , the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210; in FIG3 , the client device 240 is an external device relative to the execution device 210. In other cases, the client device 240 can also be integrated into the execution device 210; in FIG3 , the training device 220 is an external device relative to the execution device 210. In other cases, the training device 220 can also be integrated into the execution device 210; in FIG3 , the data acquisition device 260 is an external device relative to the execution device 210. In other cases, the data acquisition device 260 can also be integrated into the execution device 210, and so on. This application does not impose any limitations on this.

[0064] Since the application of neural networks is generally divided into two stages, namely the training stage and the application stage (also called the inference stage), the specific processes of the neural network training method and the speech signal processing method provided in the embodiments of the present application are described from these two stages below.

[0065] 1. Training Phase

[0066] In the embodiment of the present application, the training phase refers to the process in which the data acquisition device 260 in FIG. 3 collects training data and the training device 220 uses the training data to perform training operations on the neural network 201. Specifically, please refer to FIG. 4 , which is a flow chart of a neural network training method provided in the embodiment of the present application. The method may specifically include the following steps:

[0067] 401. Audio signals emitted by a sound source are collected using a bone conduction microphone and an air conduction microphone respectively to obtain n bone conduction voice signals and n air conduction voice signals, where n≥2.

[0068] First, the audio signals emitted by the sound source (such as user A) can be collected through the bone conduction microphone and the air conduction microphone respectively within a certain period of time. For example, n (n≥2) bone conduction voice signals s(t) and n air conduction voice signals y(t) emitted by user A when speaking can be collected.

[0069] 402. Obtain n first real transfer functions and n second real transfer functions based on the n bone conduction voice signals and the n air conduction voice signals, respectively. The first real transfer functions correspond one-to-one to the bone conduction voice signals, and the second real transfer functions correspond one-to-one to the air conduction voice signals.

[0070] Afterwards, the true transfer function h corresponding to each bone conduction voice signal s(t) can be further obtained based on each bone conduction voice signal s(t). b (t) (can be called the first real transfer function h b (t)), n bone conduction speech signals s(t) can obtain a total of n first true transfer functions h b Similarly, we can further obtain the true transfer function h corresponding to each air-conducted speech signal y(t) based on each air-conducted speech signal y(t). y (t) (can be called the second real transfer function h y (t)), n air-conducted speech signals y(t) can obtain a total of n second real transfer functions h y (t).

[0071] 403. Use the n first true transfer functions as a training data set of a neural network, train the neural network using a target loss function until a training termination condition is reached, and obtain a trained neural network, wherein the target loss function is obtained based on the second true transfer function.

[0072] The n first real transfer functions h b(t) is used as a training data set for the neural network, and the neural network is trained using a target loss function L until a training termination condition is met, thereby obtaining a trained neural network, wherein the target loss function is obtained based on the second true transfer function. As an example, the target loss function L may be an error value between a predicted transfer function output by the neural network and the second true transfer function.

[0073] Specifically, the training process can be: the first true transfer function h b (t) is the input of the neural network, and the output of the neural network is the prediction transfer function h c (t), and then iterate the training until the training termination condition is reached, where the target loss function L = |h c (t)-h y As an example, FIG5 illustrates the relationship between the input data (ie, training data) and output (ie, prediction transfer function) of recurrent neural networks (RNNs).

[0074] It should be noted that, in some embodiments of the present application, the training termination conditions include but are not limited to:

[0075] (1) The target loss function reaches the preset threshold.

[0076] After configuring the target loss function, a threshold value (e.g., 0.03) can be set for the target loss function in advance. During the iterative training of the neural network, after each training, it is determined whether the value of the target loss function obtained in the current round of training reaches the threshold value. If not, the training is continued. If it reaches the preset threshold value, the training is terminated. The values ​​of the network parameters of the neural network determined by the current round of training are used as the network parameter values ​​of the final trained neural network.

[0077] (2) The adjusted objective loss function begins to converge.

[0078] After configuring the target loss function, the neural network can be iteratively trained. If the difference between the value of the target loss function obtained in the current round of training and the value of the target loss function obtained in the previous round of training is within a preset range (e.g., 0.01), the target loss function is considered to have converged and the training can be terminated. The values ​​of the network parameters of the neural network determined by the current round of training are then used as the network parameter values ​​of the final trained neural network.

[0079] (3) The training reaches the preset number of times.

[0080] In this way, the number of iterations for training the neural network can be pre-configured (for example, 100 times). After configuring the target loss function, the neural network can be iteratively trained. After each round of training, the values ​​of the network parameters of the neural network for that round will be stored until the number of training iterations reaches the preset number. Afterwards, the neural networks obtained in each round are verified using test data, and the network parameter value with the best performance is selected as the final network parameter value of the neural network.

[0081] (4) The training duration reaches the preset duration.

[0082] In this way, the iteration time for training the neural network can be pre-configured (for example, 5 minutes). After configuring the target loss function, the neural network can be iteratively trained. After each round of training, the values ​​of the network parameters of the neural network for that round will be stored until the iteration time of training reaches the preset time. Afterwards, the neural networks obtained in each round are verified using test data, and the value of the network parameter with the best performance is selected as the final value of the network parameter of the neural network.

[0083] (5) Obtain the instruction to terminate training.

[0084] In this method, a training switch can be pre-set to trigger the generation of training start and training termination instructions. When the training switch is enabled, the training start instruction is triggered and the neural network begins iterative training. When the training switch is enabled, the training termination instruction is triggered and the neural network stops training. The duration from the training switch being turned on to the training switch being turned off is the training duration of the neural network. After configuring the target loss function, the neural network can be iteratively trained by turning the training switch on or off. After each round of training, the values ​​of the network parameters of the neural network for that round are stored until the training switch is turned off. Afterwards, the neural networks obtained in each round are verified using test data, and the network parameter values ​​with the best performance are selected as the final network parameter values ​​of the neural network.

[0085] It should also be noted that in the embodiments of the present application, n bone conduction voice signals s(t) and n air conduction voice signals y(t) may be collected at the outset, and then the neural network may be iteratively trained based on the collected n bone conduction voice signals s(t) and n air conduction voice signals y(t) (i.e., the required training data is collected first, and then the neural network is trained). Alternatively, the neural network may be trained once each bone conduction voice signal s(t) and air conduction voice signal y(t) are collected. If the neural network trained this time does not meet the training termination condition, the process of collecting training data and executing training again is repeated until the training termination condition is met (i.e., the number of collections is the same as the number of iterations required). Specifically, this application does not limit the order between data collection and training.

[0086] It should be noted that in some embodiments of the present application, the bone conduction microphone and the air conduction microphone can be deployed on the same device, such as a head-mounted device (such as headphones), and the neural network training process can also be performed on this device, that is, the training device is this device. In this case, the neural network training process is an online training process. As an example, the implementation process of the online training process can be:

[0087] ① First, the wearer puts on the head-mounted device (such as headphones).

[0088] ② Enable the online training switch. When the wearer turns on the switch, the head-mounted device is triggered to start training.

[0089] ③ Under the premise of enabling online training, the wearer speaks, and the head-mounted device simultaneously collects the wearer's bone conduction voice signal s(t) and air conduction voice signal y(t) through the bone conduction microphone and the air conduction microphone.

[0090] ④ Enable the online training switch to be turned off. When the wearer turns off the switch, the headset will be triggered to terminate the training (i.e., the training termination condition is to obtain the instruction to terminate the training). The time from the online training switch being turned on to being turned off is the training time of the neural network. During this time, the headset will convert the first real transfer function h corresponding to the bone conduction voice signal s(t) into the real transfer function h. b (t) is the input of the neural network, and the output of the neural network is the prediction transfer function h c (t), and then continuously iterate the training, and save the network parameters of the neural network obtained for the last time when the online training switch is turned off (or the optimal network parameters among all times) as the final network parameters of the neural network.

[0091] It should be noted that in some embodiments of the present application, the online training process of the neural network may not be performed on the same device where the bone conduction microphone and the air conduction microphone are deployed. As an example, assuming that the online training process of the neural network is performed in an online training module and the device is a headset, then the online training module can be deployed on the headset or on other devices, such as a mobile phone, a computer, a cloud server, etc.: a. If deployed on the headset, the system must send the signal of the air conduction microphone to the headset; b. If deployed on other devices, the system must send the signal of the bone conduction microphone to the headset. Therefore, the trained neural network can be saved on the headset or on other devices: a. If saved on the headset, after the other device trains the neural network, the trained neural network needs to be sent to the headset and saved by the headset; b. If saved on other devices, after the headset trains the neural network, the trained neural network can be sent to the other device for saving, and the trained neural network can be obtained from the other device when it is needed.

[0092] It should be noted that, in some embodiments of the present application, the training process of the neural network may also be an offline training process (i.e., the neural network is trained in advance). As an example, the implementation process of the offline training process may be:

[0093] ① Simultaneously collect the audio signals emitted by the sound source through the bone conduction microphone and the air conduction microphone to obtain the bone conduction voice signal s(t) and the air conduction voice signal y(t).

[0094] ② The first real transfer function h corresponding to the bone conduction speech signal s(t) b (t) is the input of the neural network, and the output of the neural network is the prediction transfer function h c (t), the training is iterated continuously until the training termination condition is reached, and the network parameters of the neural network obtained for the last time (or the optimal network parameters among all times) are saved as the final network parameters of the neural network.

[0095] 2. Application Phase

[0096] In the embodiment of the present application, the application phase refers to the process in which the execution device 210 in FIG3 processes the input data using the trained neural network 201. Specifically, please refer to FIG6 , which is a flow chart of a method for processing a speech signal provided in the embodiment of the present application. The method may include the following steps:

[0097] 601. Acquire a first bone conduction voice signal, and extract an excitation parameter based on the first bone conduction voice signal.

[0098] First, a bone conduction speech signal s(t) to be processed is obtained. This bone conduction speech signal s(t) may be referred to as a first bone conduction speech signal s(t). Then, an excitation parameter e(t) is extracted based on the first bone conduction speech signal s(t). The excitation parameter e(t) may include the fundamental frequency of the first bone conduction speech signal s(t) and harmonics of the fundamental frequency, and may be obtained by analyzing the speech signal using a linear predictive coding (LPC) method.

[0099] Optionally, in some embodiments of the present application, the first bone conduction voice signal s(t) may be obtained by first collecting an audio signal x(t) (also referred to as a bone conduction audio signal x(t)) based on a bone conduction microphone, and then performing noise reduction on the audio signal x(t) to obtain the first bone conduction voice signal s(t). As an example, noise reduction can be performed using a spectral subtraction noise reduction algorithm, which specifically includes the following steps: 1) Calculate the noise spectrum of the audio signal x(t): first perform voice activity detection (VAD) analysis on the audio signal x(t), take the non-speech part n(t), and perform fast Fourier transform (FFT) on the non-speech part n(t) to obtain N(w), which is the background noise spectrum; 2) Noise reduction: Perform FFT transform on the audio signal x(t) to obtain X(w), and then subtract the background noise spectrum N(w) to obtain the signal spectrum S(w) of the pure speech signal, that is, S(w) = X(w) - N(w), and then perform inverse fast Fourier transform (IFFT) on S(w) to obtain the first bone conduction speech signal s(t), which is the clean speech signal after noise reduction.

[0100] 602. Determine a first transfer function of the first bone conduction voice signal according to the first bone conduction voice signal.

[0101] Then, the transfer function h of the first bone conduction voice signal s(t) is determined according to the first bone conduction voice signal s(t). b (t), also known as the first transfer function h b Specifically, the obtained excitation parameter e(t) can be used to perform a deconvolution operation on the first bone conduction voice signal s(t) to obtain the first transfer function h of the first bone conduction voice signal s(t). b (t), that is, h b (t) = e -1 (t)*s(t).

[0102] 603. Input the first transfer function into the trained neural network to obtain an output second transfer function, where the second transfer function is the transfer function of the predicted air-conducted speech signal.

[0103] After obtaining the first transfer function h of the first bone conduction speech signal s(t) b (t) After that, the first transfer function h b (t) is input to the trained neural network, thereby outputting the second transfer function h c (t), the second transfer function h c (t) is the transfer function of the predicted air-conducted speech signal, so it can also be called the predicted transfer function h c (t).

[0104] It should be noted that in the embodiments of the present application, the trained neural network is a trained neural network obtained through the above-mentioned training phase. That is, the trained neural network is obtained by training the neural network using a training data set based on a target loss function, wherein the training data set includes multiple training data (including a first true transfer function of a bone conduction speech signal), the first true transfer function is obtained based on the audio signal emitted by the sound source collected by the bone conduction microphone; the output of the neural network is a predicted transfer function, which corresponds to a second true transfer function of the air conduction speech signal, the second true transfer function is obtained based on the audio signal emitted by the sound source collected by the air conduction microphone. Optionally, the target loss function can be the error value between the predicted transfer function output by the neural network and the second true transfer function.

[0105] 604. Obtain a first air-conduction speech signal corresponding to the first bone-conduction speech signal according to the second transfer function and the excitation parameter.

[0106] Finally, according to the second transfer function h output by the neural network c (t) and the excitation parameter e(t) determined in advance, the first air conduction speech signal s'(t) corresponding to the first bone conduction speech signal s(t) is obtained. Specifically, the second transfer function h c (t) and the excitation parameter e(t) are convolved to obtain the first air conduction speech signal s'(t) corresponding to the first bone conduction speech signal s(t), that is, s'(t) = e(t)*h c (t).

[0107] It should be noted that, in some embodiments of the present application, subsequent processing is performed based on the timbre-compensated speech signal (i.e., the first air-conducted speech signal s'(t)), including but not limited to:

[0108] 1) Voice calls can enhance the brightness of the voice after the call, achieve better voice perception quality, ensure that the perception of the other party is not affected, and improve the perceived brightness and comfort of the call voice.

[0109] 2) Voice recognition: Improve the accuracy of voice recognition under bone conduction voice.

[0110] 3) Voiceprint recognition: Improves the accuracy of voiceprint recognition under bone conduction voice.

[0111] Based on the above embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Specifically, referring to FIG7 , FIG7 is a schematic diagram of the structure of an execution device provided in an embodiment of the present application. The execution device 700 may specifically include: an acquisition module 701, a first determination module 702, a calculation module 703, and a second determination module 704. The acquisition module 701 is configured to acquire a first bone conduction voice signal and extract excitation parameters based on the first bone conduction voice signal; the first determination module 702 is configured to determine a first transfer function of the first bone conduction voice signal based on the first bone conduction voice signal; the calculation module 703 is configured to input the first transfer function into a trained neural network to obtain an output second transfer function, which is the transfer function of the predicted air conduction voice signal; and the second determination module 704 is configured to obtain a first air conduction voice signal corresponding to the first bone conduction voice signal based on the second transfer function and the excitation parameters.

[0112] In one possible design, the trained neural network is obtained by training the neural network based on a target loss function using a training data set; the training data set includes multiple training data, and the training data includes a first true transfer function of a bone conduction speech signal, which is obtained based on an audio signal emitted by a sound source collected by a bone conduction microphone; the output of the neural network is a predicted transfer function, which corresponds to a second true transfer function of an air conduction speech signal, which is obtained based on the audio signal emitted by the sound source collected by the air conduction microphone.

[0113] In one possible design, the target loss function includes: an error value between the predicted transfer function and the second true transfer function.

[0114] In one possible design, the first determination module 702 is specifically configured to: perform a deconvolution operation on the first bone conduction voice signal using the excitation parameter to obtain a first transfer function of the first bone conduction voice signal.

[0115] In one possible design, the second determination module 704 is specifically configured to perform a convolution operation on the second transfer function and the excitation parameter to obtain a first air conduction speech signal corresponding to the first bone conduction speech signal.

[0116] In a possible design, the acquisition module 701 is specifically used to: collect an audio signal through a bone conduction microphone, perform noise reduction on the audio signal, and obtain the first bone conduction voice signal.

[0117] In one possible design, the excitation parameters include a fundamental frequency of the first bone conduction voice signal and harmonics of the fundamental frequency.

[0118] It should be noted that the information interaction, execution process, etc. between the modules / units in the execution device 700 are based on the same concept as the method embodiment corresponding to Figure 6 in this application. For specific contents, please refer to the description in the method embodiment shown above in this application, and will not be repeated here.

[0119] An embodiment of the present application further provides a training device. Specifically, refer to FIG8 , which is a schematic diagram of a training device provided by an embodiment of the present application. The training device 800 may specifically include: an acquisition module 801, a calculation module 802, and an iteration module 803. The acquisition module 801 is configured to acquire audio signals emitted by a sound source through a bone conduction microphone and an air conduction microphone, respectively, to obtain n bone conduction voice signals and n air conduction voice signals, where n ≥ 2. The calculation module 802 is configured to obtain n first true transfer functions and n second true transfer functions based on the n bone conduction voice signals and the n air conduction voice signals, respectively, wherein the first true transfer functions correspond one-to-one to the bone conduction voice signals, and the second true transfer functions correspond one-to-one to the air conduction voice signals. The iteration module 803 is configured to use the n first true transfer functions as training data sets for a neural network, train the neural network using a target loss function until a training termination condition is reached, and obtain a trained neural network, wherein the target loss function is obtained based on the second true transfer functions.

[0120] In one possible design, a bone conduction microphone and the air conduction microphone are deployed in the training device 800 .

[0121] In one possible design, training device 800 includes: a head-mounted device.

[0122] In one possible design, the output of the neural network is a predicted transfer function, and the target loss function includes: an error value between the predicted transfer function and the second true transfer function.

[0123] In one possible design, the training termination conditions include: the value of the target loss function reaches a preset threshold; or, the target loss function begins to converge; or, the number of training times reaches a preset number of times; or, the training duration reaches a preset duration; or, an instruction to terminate training is obtained.

[0124] It should be noted that the information interaction, execution process, etc. between the modules / units in the training device 800 are based on the same concept as the method embodiment corresponding to Figure 4 in this application. For specific contents, please refer to the description in the method embodiment shown above in this application, and will not be repeated here.

[0125] Next, another training device provided by an embodiment of the present application will be described. Please refer to FIG. 9 , which is a schematic diagram of the structure of a training device provided by an embodiment of the present application. The training device 800 described in the embodiment corresponding to FIG. 8 can be deployed on the training device 900 to implement the functions of the training device 800 in the embodiment corresponding to FIG. 8 . Specifically, the training device 900 is implemented by one or more servers. The training device 900 may vary significantly due to different configurations or performance. The training device 900 may include one or more central processing units (CPUs) 922 and memories 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memories 932 and storage media 930 may be either ephemeral or persistent storage. The program stored in the storage medium 930 may include one or more modules (not shown), each of which may include a series of instruction operations on the training device 900. Furthermore, the CPU 922 may be configured to communicate with the storage medium 930 to execute the series of instruction operations in the storage medium 930 on the training device 900.

[0126] The training device 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0127] In an embodiment of the present application, the central processing unit 922 is configured to execute the neural network training method described in the embodiment corresponding to FIG4 . For example, the central processing unit 922 may be configured to: first, collect audio signals emitted by a sound source using a bone conduction microphone and an air conduction microphone, respectively, to obtain n bone conduction voice signals and n air conduction voice signals, where n ≥ 2; then, obtain n first true transfer functions and n second true transfer functions based on the n bone conduction voice signals and the n air conduction voice signals, respectively, wherein the first true transfer functions correspond one-to-one to the bone conduction voice signals, and the second true transfer functions correspond one-to-one to the air conduction voice signals; finally, use the n first true transfer functions as a training data set for the neural network, train the neural network using a target loss function until a training termination condition is met, and obtain a trained neural network, wherein the target loss function is obtained based on the second true transfer functions.

[0128] It should be noted that the specific manner in which the central processing unit 922 executes the above steps is based on the same concept as the method embodiment corresponding to Figure 4 in this application, and the technical effects it brings are also the same as the above embodiments of this application. For specific contents, please refer to the description in the method embodiment shown above in this application, and will not be repeated here.

[0129] Next, an execution device provided in an embodiment of the present application is introduced. Please refer to Figure 10. Figure 10 is a structural schematic diagram of an execution device provided in an embodiment of the present application. The execution device 1000 can be specifically manifested as various terminal devices, such as head-mounted devices (such as headphones), mobile phones, tablets, laptops, etc., which are not limited here. Among them, the execution device 700 described in the embodiment corresponding to Figure 7 can be deployed on the execution device 1000 to implement the functions of the execution device 700 in the embodiment corresponding to Figure 7. Specifically, the execution device 1000 may include: a receiver 1001, a transmitter 1002, a processor 1003 and a memory 1004 (wherein the number of processors 1003 in the execution device 1000 can be one or more, and Figure 10 takes one processor as an example), wherein the processor 1003 may include an application processor 10031 and a communication processor 10032. In some embodiments of the present application, the receiver 1001, the transmitter 1002, the processor 1003 and the memory 1004 may be connected via a bus or other means.

[0130] The memory 1004 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1003. A portion of the memory 1004 may also include non-volatile random access memory (NVRAM). The memory 1004 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0131] Processor 1003 controls the operation of execution device 1000. In specific applications, the various components of execution device 1000 are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all of these buses are collectively referred to as a bus system in the figure.

[0132] The method disclosed in the embodiment corresponding to FIG. 6 of the present application can be applied to or implemented by processor 1003. Processor 1003 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the method can be performed by hardware integrated logic circuits or software instructions in processor 1003. The processor 1003 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and can further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1003 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiment corresponding to FIG. 6 of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1004, and processor 1003 reads the information in memory 1004 and, in conjunction with its hardware, completes the steps of the above method.

[0133] Receiver 1001 can be used to receive input digital or character information and generate signal input related to executing relevant settings and function control of device 1000. Transmitter 1002 can be used to output digital or character information through the first interface; transmitter 1002 can also be used to send instructions to the disk pack through the first interface to modify data in the disk pack; transmitter 1002 can also include a display device such as a display screen.

[0134] In one embodiment of the present application, processor 1003 is configured to process the first transfer function of the input first bone conduction speech signal using a trained neural network to obtain a corresponding second transfer function. This trained neural network may be obtained using the training method corresponding to FIG. 4 of the present application. For details, please refer to the description of the method embodiment shown above in the present application and will not be repeated here.

[0135] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for signal processing. When the computer-readable storage medium is run on a computer, the computer executes the steps executed by the training device described in the embodiment shown in Figure 4 above, or the computer executes the steps executed by the execution device described in the embodiment shown in Figure 6 above.

[0136] The training device, execution device, etc. provided in the embodiments of the present application may specifically be a chip, which includes a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit to cause the chip within the training device to perform the steps performed by the training device described in the embodiment shown in FIG. 4 , or to cause the chip within the execution device to perform the steps performed by the execution device described in the embodiment shown in FIG. 6 .

[0137] Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0138] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0139] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or execution equipment, etc.) to execute the methods described in each embodiment of the present application.

[0140] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, training equipment or data center to another website, computer, training equipment or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, data center, etc. that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

Claims

1. A method for processing a speech signal, characterized in that: include: Acquire a first bone conduction voice signal, and extract an excitation parameter based on the first bone conduction voice signal; determining a first transfer function of the first bone conduction voice signal according to the first bone conduction voice signal; Inputting the first transfer function into the trained neural network to obtain a second transfer function as an output, wherein the second transfer function is a transfer function of the predicted air-conducted speech signal; A first air-conduction voice signal corresponding to the first bone-conduction voice signal is obtained according to the second transfer function and the excitation parameter.

2. The method according to claim 1, characterized in that The trained neural network is obtained by training the neural network using a training data set based on a target loss function; The training data set includes a plurality of training data, wherein the training data includes a first true transfer function of a bone conduction speech signal, wherein the first true transfer function is obtained based on an audio signal emitted by a sound source collected by a bone conduction microphone; The output of the neural network is a predicted transfer function, which corresponds to a second real transfer function of the air-conducted speech signal, and the second real transfer function is obtained based on the audio signal emitted by the sound source collected by the air-conduction microphone.

3. The method according to claim 2, characterized in that The objective loss function includes: An error value between the predicted transfer function and the second true transfer function.

4. The method according to any one of claims 1 to 3, characterized in that Determining a first transfer function of the first bone conduction voice signal according to the first bone conduction voice signal includes: A deconvolution operation is performed on the first bone conduction voice signal using the excitation parameter to obtain a first transfer function of the first bone conduction voice signal.

5. The method according to any one of claims 1 to 4, characterized in that The obtaining, according to the second transfer function and the excitation parameter, a first air-conduction voice signal corresponding to the first bone-conduction voice signal comprises: A convolution operation is performed on the second transfer function and the excitation parameter to obtain a first air conduction voice signal corresponding to the first bone conduction voice signal.

6. The method according to any one of claims 1 to 5, characterized in that Acquiring the first bone conduction voice signal includes: Audio signals are collected through bone conduction microphones; Noise reduction is performed on the audio signal to obtain the first bone conduction voice signal.

7. The method according to any one of claims 1 to 6, characterized in that The excitation parameters include a fundamental frequency of the first bone conduction voice signal and harmonics of the fundamental frequency.

8. A method for training a neural network, characterized in that: include: The audio signals emitted by the sound source are collected through a bone conduction microphone and an air conduction microphone respectively, and n bone conduction voice signals and n air conduction voice signals are obtained, where n ≥ 2; Obtaining n first true transfer functions and n second true transfer functions based on the n bone conduction voice signals and the n air conduction voice signals, respectively, wherein the first true transfer functions correspond one-to-one to the bone conduction voice signals, and the second true transfer functions correspond one-to-one to the air conduction voice signals; The n first true transfer functions are used as a training data set of a neural network, and the neural network is trained using a target loss function until a training termination condition is reached to obtain a trained neural network, wherein the target loss function is obtained based on the second true transfer function.

9. The method according to claim 8, characterized in that The bone conduction microphone and the air conduction microphone are deployed in the same device, and the training process of the neural network is performed in the device.

10. The method according to claim 9, characterized in that The device comprises: Head-mounted device.

11. The method according to any one of claims 8 to 10, characterized in that The output of the neural network is a prediction transfer function, and the target loss function includes: An error value between the predicted transfer function and the second true transfer function.

12. The method according to any one of claims 8 to 11, characterized in that The training termination conditions include: The value of the target loss function reaches a preset threshold; or, The objective loss function begins to converge; or, The number of training sessions reaches the preset number; or, The training duration reaches the preset duration; or, Get the instruction to terminate training.

13. An execution device, characterized in that: include: an acquisition module, configured to acquire a first bone conduction voice signal and extract an excitation parameter based on the first bone conduction voice signal; a first determining module, configured to determine a first transfer function of the first bone conduction voice signal according to the first bone conduction voice signal; a calculation module, configured to input the first transfer function into the trained neural network to obtain a second transfer function as an output, wherein the second transfer function is a transfer function of the predicted air-conducted speech signal; The second determination module is used to obtain a first air conduction voice signal corresponding to the first bone conduction voice signal according to the second transfer function and the excitation parameter.

14. The device according to claim 13, characterized in that The trained neural network is obtained by training the neural network using a training data set based on a target loss function; The training data set includes a plurality of training data, wherein the training data includes a first true transfer function of a bone conduction speech signal, wherein the first true transfer function is obtained based on an audio signal emitted by a sound source collected by a bone conduction microphone; The output of the neural network is a predicted transfer function, which corresponds to a second real transfer function of the air-conducted speech signal, and the second real transfer function is obtained based on the audio signal emitted by the sound source collected by the air-conduction microphone.

15. The device according to claim 13, characterized in that The objective loss function includes: An error value between the predicted transfer function and the second true transfer function.

16. The device according to any one of claims 13 to 15, characterized in that The first determining module is specifically configured to: A deconvolution operation is performed on the first bone conduction voice signal using the excitation parameter to obtain a first transfer function of the first bone conduction voice signal.

17. The device according to any one of claims 13 to 16, characterized in that The second determining module is specifically configured to: A convolution operation is performed on the second transfer function and the excitation parameter to obtain a first air conduction voice signal corresponding to the first bone conduction voice signal.

18. The device according to any one of claims 13 to 17, characterized in that The acquisition module is specifically used to: Audio signals are collected through bone conduction microphones; Noise reduction is performed on the audio signal to obtain the first bone conduction voice signal.

19. The device according to any one of claims 13 to 18, characterized in that The excitation parameters include a fundamental frequency of the first bone conduction voice signal and harmonics of the fundamental frequency.

20. A training device, characterized in that include: An acquisition module is used to acquire audio signals emitted by a sound source through a bone conduction microphone and an air conduction microphone, respectively, to obtain n bone conduction voice signals and n air conduction voice signals, where n is greater than or equal to 2; a calculation module, configured to obtain n first real transfer functions and n second real transfer functions based on the n bone conduction voice signals and the n air conduction voice signals, respectively, wherein the first real transfer functions correspond one-to-one to the bone conduction voice signals, and the second real transfer functions correspond one-to-one to the air conduction voice signals; An iterative module is used to use the n first true transfer functions as a training data set of a neural network, train the neural network using a target loss function until a training termination condition is reached, and obtain a trained neural network, wherein the target loss function is obtained based on the second true transfer function.

21. The device according to claim 20, characterized in that The bone conduction microphone and the air conduction microphone are disposed on the training device.

22. The device according to claim 21, characterized in that The training equipment comprises: Head-mounted device.

23. The device according to any one of claims 20 to 22, characterized in that The output of the neural network is a prediction transfer function, and the target loss function includes: An error value between the predicted transfer function and the second true transfer function.

24. The device according to any one of claims 20 to 23, characterized in that The training termination conditions include: The value of the target loss function reaches a preset threshold; or, The objective loss function begins to converge; or, The number of training sessions reaches the preset number; or, The training duration reaches the preset duration; or, Get the instruction to terminate training.

25. An execution device comprising a processor and a memory, wherein the processor is coupled to the memory, wherein: The memory is used to store programs; The processor is configured to execute the program in the memory, so that the execution device executes the method according to any one of claims 1 to 7.

26. A training device comprising a processor and a memory, wherein the processor is coupled to the memory, wherein: The memory is used to store programs; The processor is configured to execute the program in the memory so that the training device executes the method according to any one of claims 8 to 12.

27. A computer-readable storage medium comprising a program, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 12.

28. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 12.

29. A chip, characterized in that: The chip includes a processor and a data interface, and the processor reads instructions stored in a memory through the data interface to execute the method according to any one of claims 1 to 12.

Images