Virtual microphone voice signal determination method and device, equipment, product and medium

Abstract

The invention provides a virtual microphone voice signal determination method and device, equipment, a product and a medium, and relates to the technical field of acoustic signal processing. The method comprises the following steps: performing spherical harmonic transformation on an original sound signal acquired by a microphone array, and determining a spherical harmonic coefficient set; inputting the spherical harmonic coefficient set into a voice signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the voice signal prediction model; performing inverse spherical harmonic transformation on the virtual spherical harmonic coefficient set to determine a virtual microphone voice signal; wherein the voice signal prediction model is obtained based on training of a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set. According to the invention, the defect that the precision of the voice signal of the virtual microphone is insufficient due to the fact that the performance is limited in a complex acoustic environment in the prior art can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of acoustic signal processing, and in particular to a method, device, equipment, product and medium for determining a virtual microphone voice signal. Background Art

[0002] With the rapid development of speech recognition and speech enhancement technologies, multi-channel speech enhancement has become increasingly important in applications such as video conferencing, telecommunications, and hearing aids. Its goal is to improve the quality and clarity of speech captured by microphone arrays in complex noise and reverberation environments, thereby improving the user experience. Traditional speech enhancement techniques usually rely on the number of microphone arrays, with more microphones providing better performance. For example, beamforming technology using N microphones can enhance specific sound sources and suppress interference sources by generating N-1 nulls.

[0003] However, when the number of sound sources S exceeds the number of microphones N, the performance of beamforming technology will be limited. This is especially common in many commercial microphone devices, because many devices are not equipped with enough microphones due to device structure and cost constraints. To solve this problem, virtual microphone technology came into being. Most existing virtual microphone technologies use linear interpolation methods to estimate virtual microphone signals. Linear interpolation methods are computationally efficient and easy to implement, but in the face of complex acoustic environments, especially under the influence of reverberation and diffuse noise, performance is often limited, resulting in insufficient accuracy of virtual microphone speech signals, thereby affecting subsequent speech enhancement effects. Therefore, there is an urgent need for a method that can provide virtual microphone speech signals with higher accuracy and greater robustness. Summary of the invention

[0004] The present invention provides a method, device, equipment, product and medium for determining a virtual microphone speech signal, so as to solve the defect that the prior art is often limited in performance when facing a complex acoustic environment, especially under the influence of reverberation and diffuse noise, resulting in insufficient accuracy of the virtual microphone speech signal, thereby affecting the subsequent speech enhancement effect.

[0005] The present invention provides a method for determining a virtual microphone voice signal, comprising: Perform spherical harmonic transformation on the original sound signal collected by the microphone array to determine the spherical harmonic coefficient set; Inputting the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the speech signal prediction model; Performing an inverse spherical harmonic transform on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal; The speech signal prediction model is obtained by training based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0006] According to a virtual microphone speech signal determination method provided by the present invention, the model parameters of the speech signal prediction model are obtained based on at least one iterative update, and any one of the at least one iterative updates includes: Based on the sample spherical harmonic coefficient set, an initial virtual spherical harmonic coefficient set is predicted; Based on the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label, a loss value is calculated by a loss function; Based on the loss value, the model parameters are updated.

[0007] According to a virtual microphone voice signal determination method provided by the present invention, sound data at the real microphone position is obtained; The sound data at the actual microphone position is subjected to spherical harmonic transformation to determine a set of sample spherical harmonic coefficients.

[0008] According to a method for determining a virtual microphone speech signal provided by the present invention, the loss function is shown in the following formula: ; in, represents the initial virtual spherical harmonic coefficient set predicted by the speech signal prediction model, represents the label of the virtual spherical harmonic coefficient set, N represents the number of samples, and MSE represents the loss function.

[0009] According to a method for determining a virtual microphone speech signal provided by the present invention, the original sound signal includes a target sound signal and a noise signal, and the original sound signal is subjected to spherical harmonic transformation based on the following method to determine a spherical harmonic coefficient set: Determining spatial position information of the target sound signal and the noise signal respectively; The spatial position information of the target sound signal and the noise signal is calculated based on the spherical harmonic basis function to obtain a spherical harmonic coefficient set including the target sound signal and the noise signal.

[0010] According to a virtual microphone speech signal determination method provided by the present invention, any spherical harmonic coefficient in the spherical harmonic coefficient set represents a sound component of a spatial mode.

[0011] The present invention also provides a virtual microphone voice signal determination device, comprising: The spherical harmonic transformation module is used to perform spherical harmonic transformation on the original sound signal collected by the microphone array to determine the spherical harmonic coefficient set.

[0012] A prediction module is used to input the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the speech signal prediction model.

[0013] The inverse spherical harmonic transformation module is used to perform an inverse spherical harmonic transformation on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal.

[0014] The speech signal prediction model is obtained by training based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for determining a virtual microphone voice signal as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for determining a virtual microphone speech signal as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining a virtual microphone speech signal as described above is implemented.

[0018] The virtual microphone speech signal determination method, device, equipment, product and medium provided by the present invention perform spherical harmonic transformation on the original sound signal collected by the microphone array to determine the spherical harmonic coefficient set; input the spherical harmonic coefficient set into the speech signal prediction model to obtain the virtual spherical harmonic coefficient set of the virtual microphone position output by the speech signal prediction model; perform inverse spherical harmonic transformation on the virtual spherical harmonic coefficient set to determine the virtual microphone speech signal; wherein the speech signal prediction model is trained based on the sample spherical harmonic coefficient set and the virtual spherical harmonic coefficient set labels corresponding to the sample spherical harmonic coefficient set; the present invention solves the problem of insufficient accuracy and robustness of traditional virtual microphone technology in complex acoustic environments by combining spherical harmonic transformation with a speech signal prediction model based on deep learning; specifically, firstly The original sound signal is converted into a spherical harmonic coefficient set, which can effectively represent the distribution characteristics of the sound signal in space, especially when processing sounds from different directions, it can capture richer spatial information; secondly, the spherical harmonic coefficient set is input into the speech signal prediction model. The speech signal prediction model is trained to learn how to predict the virtual spherical harmonic coefficient set of the virtual microphone position based on the spherical harmonic coefficient set corresponding to the input actual microphone array. Next, the virtual spherical harmonic coefficient set is converted into a virtual microphone speech signal using an inverse spherical harmonic transform. The setting of the virtual microphone position is usually to estimate the position of the virtual microphone through an algorithm when there are not enough physical microphones, thereby improving the accuracy and quality of sound capture in complex environments, overcoming the performance bottleneck of traditional methods in complex acoustic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of the method for determining a virtual microphone speech signal provided by the present invention.

[0021] Figure 2 It is a structural schematic diagram of a virtual microphone speech signal determination device provided by the present invention.

[0022] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In the field of multi-channel speech enhancement, how to accurately restore and enhance the target speech signal has always been a difficult problem, especially in noisy and reverberant environments. With the advancement of technology, virtual microphone technology, as an effective solution, has achieved initial results in practical applications. This technology simulates more microphone positions to overcome the limitation of the number of physical microphones and improve the quality and clarity of speech signals. However, in complex acoustic environments, the existing technology has several shortcomings, which limits the accuracy of virtual microphone speech signals, thus affecting the speech enhancement effect.

[0025] The applicant of this patent has found in long-term research that existing virtual microphone technologies, such as methods based on linear interpolation and traditional beamforming technologies, can enhance speech signals to a certain extent, but they rely on simplified assumptions, such as plane wave propagation and orthogonality of sound sources. These simplified assumptions are often unable to effectively cope with more complex environmental conditions in the real world, especially under the influence of reverberation and diffuse noise, the performance of these methods drops sharply. Specifically, when the number of sound sources exceeds the number of microphones, or there is a strong reverberation effect in the environment, these traditional methods often lead to insufficient accuracy of the virtual microphone signal, thereby affecting the subsequent speech enhancement effect, and even causing the noise to be unable to be effectively suppressed, and the target speech signal to be severely distorted.

[0026] After further research, the applicant found that although the existing technologies have solved the problem of multi-channel signal enhancement to a certain extent, they still rely on feature extraction methods based on the time-frequency domain and cannot effectively capture spatial information and inter-channel dependencies. This means that when dealing with more complex acoustic environments, the speech enhancement effect of the existing technologies is often affected by factors such as spatial aliasing, noise interference, and reflection, resulting in reduced accuracy in speech signal recovery. Therefore, when facing multi-channel signals, the existing technologies still have significant deficiencies, especially when dealing with reverberation and noise effects in complex environments.

[0027] In view of the above problems, the present invention proposes the following embodiments.

[0028] Figure 1 is a flow chart of a method for determining a virtual microphone speech signal provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 110, performing spherical harmonic transformation on the original sound signal collected by the microphone array to determine a spherical harmonic coefficient set.

[0029] It should be noted that the spherical harmonic transform is a mathematical transformation method used to map data from a Cartesian coordinate system in three-dimensional space to a spherical coordinate system. In acoustic applications, the spherical harmonic transform can describe the spatial patterns in the sound field, especially for the processing of multi-channel audio signals, which can provide higher spatial resolution than traditional methods. In short, the spherical harmonic transform can effectively capture the distribution characteristics of the sound source in three-dimensional space, that is, convert the spatial position information of the original sound signal into a set of spherical harmonic coefficients.

[0030] Here, the original sound signal refers to the sound signal collected from the microphone array at the real position.

[0031] Here, the spherical harmonic coefficient set is the result of spherical harmonic transformation, and each coefficient in the spherical harmonic coefficient set reflects the distribution of the sound field in different spatial modes, where different spatial modes refer to different directivities and frequencies.

[0032] Step 120: input the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the speech signal prediction model.

[0033] The speech signal prediction model is obtained by training based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0034] Here, the speech signal prediction model is usually a deep learning model, which predicts the sound field characteristics of a virtual microphone position through the input spherical harmonic coefficient set based on the mapping relationship learned from a large amount of training data. Virtual microphone means using limited physical microphone signals to generate more virtual position signals through inference and simulate more receiving points. The output of the speech signal prediction model is the virtual spherical harmonic coefficient set corresponding to the virtual microphone position.

[0035] Here, the virtual spherical harmonic coefficient set label is the output target corresponding to the sample spherical harmonic coefficient set. The virtual spherical harmonic coefficient set label represents the expected spherical harmonic coefficients at the virtual microphone position. These virtual spherical harmonic coefficient set labels serve as real target outputs to help the training model learn how to infer the virtual microphone signal from the actual microphone array signal, that is, how to predict the virtual spherical harmonic coefficient set based on the spherical harmonic coefficient set.

[0036] Step 130: Perform an inverse spherical harmonic transform on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal.

[0037] Here, the inverse spherical harmonic transform is the inverse operation of the spherical harmonic transform. The inverse spherical harmonic transform reconstructs the signal in space by convolving the spherical harmonic coefficient set with the spherical harmonic basis function. The result of the inverse spherical harmonic transform is to restore a sound signal containing spatial information, that is, a virtual microphone signal.

[0038] The virtual microphone speech signal determination method provided by the embodiment of the present invention solves the problem of insufficient accuracy and robustness of traditional virtual microphone technology in complex acoustic environments by combining spherical harmonic transformation with a speech signal prediction model based on deep learning. Specifically, the original sound signal is first converted into a spherical harmonic coefficient set, which can effectively represent the distribution characteristics of the sound signal in space, especially when processing sounds from different directions, it can capture richer spatial information. Secondly, the spherical harmonic coefficient set is input into the speech signal prediction model. The speech signal prediction model learns through training how to predict the virtual spherical harmonic coefficient set of the virtual microphone position based on the spherical harmonic coefficient set corresponding to the input actual microphone array. Next, the virtual spherical harmonic coefficient set is converted into a virtual microphone speech signal by using an inverse spherical harmonic transformation. The setting of the virtual microphone position is usually to estimate the position of the virtual microphone through an algorithm when there are not enough physical microphones, thereby improving the accuracy and quality of sound capture in a complex environment, and overcoming the performance bottleneck of traditional methods in complex acoustic environments.

[0039] Based on any of the above embodiments, in the method, the model parameters of the speech signal prediction model are obtained based on at least one iterative update, and any one of the at least one iterative updates includes: Based on the sample spherical harmonic coefficient set, an initial virtual spherical harmonic coefficient set is predicted; Based on the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label, a loss value is calculated by a loss function; Based on the loss value, the model parameters are updated.

[0040] Here, the initial virtual spherical harmonic coefficient set refers to the spherical harmonic coefficient set of the virtual microphone signal predicted by the current parameters. Compared with the real virtual spherical harmonic coefficient set label, it may have a certain error and needs to be continuously optimized through training. The loss function is used to measure the gap between the model prediction result and the real label. Here it is used to measure the gap between the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label. The smaller the loss value, the closer the model's prediction result is to the real value.

[0041] Specifically, based on the loss value, the model parameters are updated using optimization algorithms such as gradient descent. The purpose of this process is to gradually pass the loss value through each layer of the model through back propagation, adjust the parameters of each layer, so that the model's prediction results gradually tend to the true label, and the updated model parameters can provide more accurate predictions in the next iteration, thereby improving the performance of the speech signal prediction model.

[0042] For example, the sample sound signal Perform spherical harmonic transformation based on the following formula to determine the sample spherical harmonic coefficient set : ; Furthermore, the initial virtual spherical harmonic coefficient set is predicted by the following formula: : ; During the training phase, the loss value is calculated by the loss function based on the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label. As shown below: ; Among them, MSE represents the loss function, Represents the label of the virtual spherical harmonic coefficient set.

[0043] Furthermore, the model parameters are updated based on the following formula: : ; in, Represents the learning rate.

[0044] In the test phase, the sound pressure field is reconstructed based on the following formula, that is, the virtual spherical harmonic coefficient set is subjected to an inverse spherical harmonic transformation to determine the virtual microphone speech signal est: ; The virtual microphone speech signal determination method provided by the embodiment of the present invention uses the sample spherical harmonic coefficient set as input, and the speech signal prediction model can predict the initial virtual spherical harmonic coefficient set during the training process. There may be a certain error between the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label, and these errors are quantified by the loss value calculated by the loss function. The size of the loss value directly reflects the gap between the prediction result and the true value. The larger the loss value, the worse the prediction effect of the model, and vice versa. Then, through the optimization algorithm, the parameters of the model are updated according to the calculated loss value. This update process enables the model to gradually adjust its internal parameters, thereby reducing the value of the loss function. As the number of iterations increases, the parameters of the model are continuously optimized, and the prediction results of the virtual spherical harmonic coefficient set gradually tend to the true value. This iterative update process can significantly improve the accuracy and robustness of the model, so that the model can accurately predict the virtual microphone signal, especially in a complex acoustic environment. Specifically, through iterative training, the model can better adapt to different noise and reverberation interference, thereby generating a more accurate virtual spherical harmonic coefficient set, and then determining a more accurate virtual microphone speech signal.

[0045] Based on any of the above embodiments, in this method, the sample spherical harmonic coefficient set is determined based on the following method: Get the sound data at the actual microphone position; The sound data at the actual microphone position is subjected to spherical harmonic transformation to determine a set of sample spherical harmonic coefficients.

[0046] Here, the sound data at the real microphone position refers to the actual sound signal collected by the physical microphone in the microphone array.

[0047] The virtual microphone speech signal determination method provided by the embodiment of the present invention can effectively extract spatial information from the sound signal and form a set of sample spherical harmonic coefficients by acquiring the sound data of the real microphone position and performing spherical harmonic transformation. It provides high-quality training data for the subsequent prediction of virtual microphone signals, can improve the prediction accuracy of virtual microphone signals, and improve the robustness in complex acoustic environments, thereby optimizing the speech enhancement effect.

[0048] Based on any of the above embodiments, in this method, the loss function is shown as follows: ; in, represents the initial virtual spherical harmonic coefficient set predicted by the speech signal prediction model, represents the label of the virtual spherical harmonic coefficient set, N represents the number of samples, and MSE represents the loss function.

[0049] It should be noted that the role of the loss function is to measure the difference between the predicted output of the speech signal prediction model and the actual label, that is, the difference between the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label. The loss function is a key part of the optimization algorithm when training a neural network model. It is used to quantify the error between the model prediction result and the actual result in order to guide the model parameter update.

[0050] The virtual microphone speech signal determination method provided in the embodiment of the present invention adopts a loss function to measure the difference between the prediction result of the speech signal prediction model and the actual label. The loss function plays a key role in optimizing the speech signal prediction model during the training process, and quantifies the error between the initial virtual spherical harmonic coefficient set output by the speech signal prediction model and the virtual spherical harmonic coefficient set label. By calculating the error and continuously adjusting the model parameters, the training process realizes the continuous optimization of the speech signal prediction model, so that the prediction result is closer to the true value.

[0051] Based on any of the above embodiments, in the method, the original sound signal includes a target sound signal and a noise signal, and the original sound signal is spherically harmonically transformed in the following manner to determine a spherical harmonic coefficient set: Determining spatial position information of the target sound signal and the noise signal respectively; The spatial position information of the target sound signal and the noise signal is calculated based on the spherical harmonic basis function to obtain a spherical harmonic coefficient set including the target sound signal and the noise signal.

[0052] Here, spatial position information refers to the directionality and position of the source of the target sound signal and the noise signal. Sound data is usually captured by multiple microphones, and the signal received by each microphone comes from the sound source in a different direction. The spherical harmonic coefficient set is a set of spherical harmonic coefficients including the target sound signal and the noise signal. The spherical harmonic coefficient set contains the performance of the original sound signal in each spatial mode and is an efficient representation of the original sound signal.

[0053] The virtual microphone speech signal determination method provided by the embodiment of the present invention performs spherical harmonic transformation on the spatial position information of the target sound signal and the noise signal and generates a spherical harmonic coefficient set. The spherical harmonic coefficients can effectively describe the spatial distribution characteristics of the sound signal. After the original sound signal is converted into a spherical harmonic coefficient set through spherical harmonic transformation, the spatial information of the sound is retained, so that the subsequent prediction model can use these spatial features to improve the prediction accuracy of the virtual spherical harmonic coefficient set of the subsequently predicted virtual microphone position.

[0054] The virtual microphone voice signal determination device provided by the present invention is described below. The virtual microphone voice signal determination device described below and the virtual microphone voice signal determination method described above can be referenced to each other.

[0055] FIG. 2 is a schematic diagram of the structure of a virtual microphone voice signal determination device provided by the present invention. As shown in FIG. 2 , the virtual microphone voice signal determination device includes: The spherical harmonic transformation module 210 is used to perform spherical harmonic transformation on the original sound signal collected by the microphone array to determine a spherical harmonic coefficient set.

[0056] The prediction module 220 is used to input the spherical harmonic coefficient set into the speech signal prediction model to obtain the virtual spherical harmonic coefficient set of the virtual microphone position output by the speech signal prediction model.

[0057] The inverse spherical harmonic transformation module 230 is used to perform an inverse spherical harmonic transformation on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal.

[0058] The speech signal prediction model is obtained by training based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0059] In one embodiment, the spherical harmonic transformation module 210 is specifically used to determine the spatial position information of the target sound signal and the noise signal respectively; calculate the spatial position information of the target sound signal and the noise signal based on the spherical harmonic basis function to obtain the spherical harmonic coefficients corresponding to the target sound signal and the spherical harmonic coefficients of the noise signal; determine the spherical harmonic coefficient set based on the spherical harmonic coefficients corresponding to the target sound signal and the spherical harmonic coefficients of the noise signal.

[0060] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a method for determining a virtual microphone speech signal, the method comprising: performing a spherical harmonic transformation on the original sound signal acquired by the microphone array to determine a spherical harmonic coefficient set; inputting the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the speech signal prediction model; performing an inverse spherical harmonic transformation on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal; wherein the speech signal prediction model is trained based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0061] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0062] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the virtual microphone speech signal determination method provided by the above-mentioned methods, and the method includes: performing a spherical harmonic transform on the original sound signal collected by the microphone array to determine the spherical harmonic coefficient set; inputting the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of the virtual microphone position output by the speech signal prediction model; performing an inverse spherical harmonic transform on the virtual spherical harmonic coefficient set to determine the virtual microphone speech signal; wherein the speech signal prediction model is trained based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0063] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the virtual microphone speech signal determination method provided by the above-mentioned methods, the method comprising: performing a spherical harmonic transform on the original sound signal collected by a microphone array to determine a spherical harmonic coefficient set; inputting the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of the virtual microphone position output by the speech signal prediction model; performing an inverse spherical harmonic transform on the virtual spherical harmonic coefficient set to determine the virtual microphone speech signal; wherein the speech signal prediction model is trained based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

[0064] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0065] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining a virtual microphone speech signal, characterized in that: include: Perform spherical harmonic transformation on the original sound signal collected by the microphone array to determine the spherical harmonic coefficient set; Inputting the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the speech signal prediction model; Performing an inverse spherical harmonic transform on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal; The speech signal prediction model is obtained by training based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

2. The method for determining a virtual microphone speech signal according to claim 1, characterized in that: The model parameters of the speech signal prediction model are obtained based on at least one iterative update, and any one of the at least one iterative update includes: Based on the sample spherical harmonic coefficient set, an initial virtual spherical harmonic coefficient set is predicted; Based on the initial virtual spherical harmonic coefficient set and the virtual spherical harmonic coefficient set label, a loss value is calculated by a loss function; Based on the loss value, the model parameters are updated.

3. The method for determining a virtual microphone speech signal according to claim 1, characterized in that: The sample spherical harmonic coefficient set is determined based on the following method: Get the sound data at the actual microphone position; The sound data at the actual microphone position is subjected to spherical harmonic transformation to determine a set of sample spherical harmonic coefficients.

4. The method for determining a virtual microphone speech signal according to claim 2, characterized in that: The loss function is shown as follows: ; in, represents the initial virtual spherical harmonic coefficient set predicted by the speech signal prediction model, represents the label of the virtual spherical harmonic coefficient set, N represents the number of samples, and MSE represents the loss function.

5. The method for determining a virtual microphone speech signal according to claim 1, characterized in that: The original sound signal includes a target sound signal and a noise signal, and the original sound signal is subjected to spherical harmonic transformation in the following manner to determine a spherical harmonic coefficient set: Determining spatial position information of the target sound signal and the noise signal respectively; The spatial position information of the target sound signal and the noise signal is calculated based on the spherical harmonic basis function to obtain a spherical harmonic coefficient set including the target sound signal and the noise signal.

6. The method for determining a virtual microphone speech signal according to claim 1, characterized in that: Any spherical harmonic coefficient in the spherical harmonic coefficient set represents a sound component of a spatial mode.

7. A virtual microphone voice signal determination device, characterized in that: include: A spherical harmonic transformation module is used to perform spherical harmonic transformation on the original sound signal collected by the microphone array to determine the spherical harmonic coefficient set; A prediction module, which is used to input the spherical harmonic coefficient set into a speech signal prediction model to obtain a virtual spherical harmonic coefficient set of a virtual microphone position output by the speech signal prediction model; An inverse spherical harmonic transformation module, which is used to perform an inverse spherical harmonic transformation on the virtual spherical harmonic coefficient set to determine a virtual microphone speech signal; The speech signal prediction model is obtained by training based on a sample spherical harmonic coefficient set and a virtual spherical harmonic coefficient set label corresponding to the sample spherical harmonic coefficient set.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the virtual microphone speech signal determination method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining a virtual microphone speech signal according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining a virtual microphone speech signal according to any one of claims 1 to 6 is implemented.

Images